30 June 2014

Stream Processing Frameworks

Technology Does it fit Maturity Ease of use Language Comments
Apache Storm ★★★ ★★★ ★★ Clojure originates from Twitter
Apache Spark Streaming ★★★ ★★ ★★★ Scala/Java/Python originates from AMPLab
Apache Samza ★★★ ★★ Scala/Java originates from LinkedIn
Apache S4 ★★★ Java originates from Yahoo!
Spring XD ★★★ ★★ ★★★ Java originates from Pivotal

Cloud-based (XaaS) Offerings

Technology Does it fit Maturity Ease of use API Comments
AWS Kinesis ★★★ ★★ ★★ Java introduced in 11/2013
Google Cloud Dataflow ★★ - ? Java introduced in 06/2014, not yet available

Further Resources

For an introduction into stream processing, have a look at a survey article on Data Stream and Complex Event Processing Systems.

If you’re interested in doing machine learning over streams, check out Yahoo!’s Scalable Advanced Massive Online Analysis (SAMOA).

Getting Started with Storm is a hands-on guide into real-time data analysis with Storm, incl. a real-life example with Node.js and Redis.

The Storm Applied book provides a gentle introduction and in-depth guidance for the advanced.

blog comments powered by Disqus

Fork me on GitHub