Run StreamsFilterTweets.java and TwitterProducerOptimized.java

See result using:
kafka-console-consumer --bootstrap-server localhost:9092 --topic famous-tweets

Tweets that go into a topic named "twitter-tweets", will be filtered with a tweet that is posted by a user who has 10K followers and they will go into a topic named "famous-tweets"

+ Create app.config to load twitter api tokens

Safe + High throughput by using

compression = snappy,
linger ms = 20,
batch size = 32 * 1024,
acks = all,
idempotence = true,
retries = INT32_MAX
max.in.flight = 5

Create a topic named "twitter-tweets" with below command,

kafka-topics --zookeeper localhost:2181 --create --topic twitter-tweets --partitions 6 --replication-factor 1

and run this application.

Next we will build kafka producer to send tweets to kafka.

https://github.com/twitter/hbc