30 June 2014

Processing Frameworks

Technology Does it fit Maturity Ease of use Language Platforms Comments
Hadoop MapReduce ★★★ ★★★ Java Hadoop Very low-level, not re-usable
Spark ★★★ ★★ ★★★ Scala, Java, Python Spark In-memory
Hive ★★★ ★★★ ★★★ HiveQL, Java Hadoop Support planned for Tez
Spark SQL ★★★ ★★ SQL, Scala, Java, Python Spark Successor of Shark
Pig ★★★ ★★★ ★★★ Pig Latin, Java Hadoop Support planned Tez
Spork ★★★ ★★★ Pig Latin, Java Spark  
Cascading/Scalding ★★★ ★★ ★★ Java, Scala Hadoop  
Cascalog ★★★ Clojure Hadoop  
Crunch/SCrunch ★★★ ★★ Java, Scala Hadoop Support planned for Spark and Tez
Pangool ★★★ Java Hadoop  


blog comments powered by Disqus

Fork me on GitHub