An example Lambda Architecture for real-time analysis of hashtags using Trident, Hadoop and Splout SQL
In this post it is shown how to use Trident, Hadoop and Splout SQL together to build a toy example “lambda architecture“. One can learn the basics of Trident, a higher-level API on top of Storm, and Splout SQL, a fast SQL read-only DB for Hadoop. The example architecture is hosted on a github project.
The example simulates counting the number of appearances of hashtags in tweets, by date. The ultimate goal is to solve this simple problem in a fully scalable way, and provide a remote low-latency service for querying the evolution of the counts of a hashtag, including both consolidated and real-time statistics for it.
blog comments powered by Disqus