An example of Lambda Architecture to analyse Twitter's tweets with Spark, Spark-streaming, Cassandra, Kafka, Twitter4j, Akka and Akka-http
This post gives an overview about an article which shows the usage of an "lambda architecture" for a Twitter tweets analysis. It combines reactive frameworks like Spark, Akka and Akka-http to build this kind of architecture.
In this Blog post, we can see realtime twitter's tweets analysis using "lambda architecture". This project basically shows how to easily implement each layer of lambda architecture using SACK(Spark,Akka,Cassandra,Kafka) stack. We have used akka scheduler and Spark-streaming windows time slice to effectively implement batch view and realtime view creation in specified time interval. It uses Twitter4j streaming api to fetch realtime tweets against given keywords and put into kafka. It uses Cassandra to store batch_view, realtime_view and master_dataset. To get source code follow the link: https://github.com/knoldus/Lambda-Arch-Spark
To Read the full Blog post: https://blog.knoldus.com/2017/01/31/twitters-tweets-analysis-using-lambda-architecture/.
- post 6
- Big Data 12
- Lambda 11
- Spark 2
- Spark-streaming 1
- Kafka 3
- Cassandra 2
- Twitter4j 1
- Akka 2
- Akka-http 1
blog comments powered by Disqus