Skip to content

Latest commit

 

History

History
18 lines (14 loc) · 619 Bytes

README.md

File metadata and controls

18 lines (14 loc) · 619 Bytes

eth-bigdata

MapReduce

  • Start docker containers from /mapreduce folder docker-compose up.
  • Run ./copy-files.sh.
  • From hadoop namenode run ./compile-and-run.sh.

Spark batch

  • Start docker containers from /spark-batch folder using docker-compose up.
  • Build application using maven build tool.
  • Run ./copy-files.sh
  • Run run.sh with one of the params (org.bigdata.MaxByWeek, org.bigdata.MeanVariance, org.bigdata.MostConnectedAddresses)

Spark streaming

  • Start docker containers from /spark-stream folder using docker-compose up.
  • Build application using maven build tool.
  • Run run.sh