19 January 2014

In this post it is shown how to use Trident, Hadoop and Splout SQL together to build a toy example “lambda architecture“. One can learn the basics of Trident, a higher-level API on top of Storm, and Splout SQL, a fast SQL read-only DB for Hadoop. The example architecture is hosted on a github project.

The example simulates counting the number of appearances of hashtags in tweets, by date. The ultimate goal is to solve this simple problem in a fully scalable way, and provide a remote low-latency service for querying the evolution of the counts of a hashtag, including both consolidated and real-time statistics for it.

Read the full post: http://www.datasalt.com/2013/01/an-example-lambda-architecture-using-trident-hadoop-and-splout-sql/.

blog comments powered by Disqus

Fork me on GitHub