Code

Add Spark Job Launcher tool#9288

Code

Merged

KKcorps merged 10 commits intoapache:masterapache/pinot:masterfrom KKcorps:spark_command_toolKKcorps/incubator-pinot:spark_command_toolCopy head branch name to clipboardSep 7, 2022

+391

Commits

Commits on Aug 24, 2022

• Add Spark data ingestion command

  

  Kartik Khare

  authored and

  Kartik Khare

  committed
  5634a2c

  Copy full SHA for 5634a2c

• Add capability to automatically add classpaths

  

  Kartik Khare

  authored and

  Kartik Khare

  committed
  c3292fb

  Copy full SHA for c3292fb

• Only add spark launcher dependency and remove the rest

  

  Kartik Khare

  authored and

  Kartik Khare

  committed
  3348ea5

  Copy full SHA for 3348ea5

Commits on Aug 25, 2022

• Add support for spark confs

  

  Kartik Khare

  authored and

  Kartik Khare

  committed
  4aeac5a

  Copy full SHA for 4aeac5a

Commits on Aug 26, 2022

• S3 filepaths and plugin filters working

  

  Kartik Khare

  authored and

  Kartik Khare

  committed
  0d4a40e

  Copy full SHA for 0d4a40e

• upgrade spark launcher version

  

  Kartik Khare

  authored and

  Kartik Khare

  committed
  41f9049

  Copy full SHA for 41f9049

• Fix s3 paths

  

  Kartik Khare

  authored and

  Kartik Khare

  committed
  b678399

  Copy full SHA for b678399

Commits on Aug 27, 2022

• Add support for additional args, cleanup description

  

  Kartik Khare

  authored and

  Kartik Khare

  committed
  b7e5cab

  Copy full SHA for b7e5cab

• Fix Scala conflicts by excluding kafka plugins

  

  Kartik Khare

  authored and

  Kartik Khare

  committed
  2ba3254

  Copy full SHA for 2ba3254

Commits on Sep 7, 2022

• Fix linting error

  

  Kartik Khare

  authored and

  Kartik Khare

  committed
  6854682

  Copy full SHA for 6854682