twitter-stream

Twitter-Kafka Data Pipeline

Requirements :

Apache Kafka 2.6.0 Twitter Developer account ( for API Key, Secret etc.) Apache Zookeeper ( required for Kafka) Oracle JDK 1.8 (64 bit )

How to Run

Provide JVM Argument for TwitterKafkaProducer.java in following order

java TwitterKafkaProducer.java <consumer_key> <consumer_secret> <account_token> <account_secret> <hashtag/term>

The topic and bootstrap servers for Kafka can be configured with option arguments. If using SASL Auth to Kafka, also set username and password. See below for usage.

usage: twitter-stream
 -password <arg>   SASL Auth password
 -servers <arg>    Comma separated list of Kafka bootstrap servers
 -topic <arg>      Kafka topic
 -username <arg>   SASL Auth username

Build Environment :

Eclipse/Intellij Apache Maven

Detailed steps available here - http://saurzcode.in/2015/02/kafka-producer-using-twitter-stream/