Code

Scala 2.13 Support#8592

Code

Merged

NVnavkumar merged 130 commits intoNVIDIA:branch-23.12NVIDIA/spark-rapids:branch-23.12from NVnavkumar:scala-213NVnavkumar/spark-rapids:scala-213Copy head branch name to clipboardOct 27, 2023

+9,287-384

Commits

Commits on Jun 21, 2023

• Preliminary Scala 2.13 build support, need to fix compilation errors

  Show description for 77e83c3
  NVnavkumar

  committed
  77e83c3

  Copy full SHA for 77e83c3

• Handle 2 simple updates with scalastyle checks

  NVnavkumar

  committed
  7c9a930

  Copy full SHA for 7c9a930

• Update due to requirement of not importing scala.collection.Seq directly

  NVnavkumar

  committed
  5b41add

  Copy full SHA for 5b41add

• One round of code updates for Scala 2.13

  Show description for 02ef278
  NVnavkumar

  committed
  02ef278

  Copy full SHA for 02ef278

Commits on Sep 5, 2023

• Merge branch 'branch-23.10' into scala-213

  NVnavkumar

  committed
  8adb72e

  Copy full SHA for 8adb72e

• Style fixes.

  NVnavkumar

  committed
  ef6a97d

  Copy full SHA for ef6a97d

Commits on Sep 6, 2023

• Update RAPIDS shuffle manager code to parallel use of scala.collection.Seq from Spark 3.4 and higher.

  Show description for a359bec
  NVnavkumar

  committed
  a359bec

  Copy full SHA for a359bec

• Handle usage of Seq in GpuBindReferences

  NVnavkumar

  committed
  65e3601

  Copy full SHA for 65e3601

• Sub partition hash join updates. Follow the recommended scala pattern of converting when returning Seq and using Seq in _* pattern

  NVnavkumar

  committed
  bc5d337

  Copy full SHA for bc5d337

• Handle ArrayStack usage in GpuSortExec and SortUtils

  NVnavkumar

  committed
  ce9054e

  Copy full SHA for ce9054e

• Regex parser fixes.

  NVnavkumar

  committed
  272725e

  Copy full SHA for 272725e

• Parquet Scan Seq usage.

  NVnavkumar

  committed
  cd7a132

  Copy full SHA for cd7a132

• Orc scan Seq usage.

  NVnavkumar

  committed
  ac75ac6

  Copy full SHA for ac75ac6

• Multi file reader Seq usage.

  NVnavkumar

  committed
  c5f7bff

  Copy full SHA for c5f7bff

• Hive CTAS command fixes

  NVnavkumar

  committed
  4b69ec9

  Copy full SHA for 4b69ec9

• miscellaneous scan usage of Seq

  NVnavkumar

  committed
  4714ca6

  Copy full SHA for 4714ca6

• Fix procedure syntax.

  NVnavkumar

  committed
  85c0af9

  Copy full SHA for 85c0af9

Commits on Sep 7, 2023

• Add procedure declaration checker since it's fully deprecated in Scala 2.13 and will cause compilation to fail

  Show description for 144e442
  NVnavkumar

  committed
  144e442

  Copy full SHA for 144e442

• Add scala versions here based on Spark

  NVnavkumar

  committed
  a38a91a

  Copy full SHA for a38a91a

Commits on Sep 8, 2023

• Merge branch 'branch-23.10' into scala-213

  NVnavkumar

  committed
  a9bd68e

  Copy full SHA for a9bd68e

• Move these files to scala 2.12 only compilation

  NVnavkumar

  committed
  8db9a16

  Copy full SHA for 8db9a16

• Restore these to the original scala 2.12 compatible versions

  Show description for 69d7b44
  NVnavkumar

  committed
  69d7b44

  Copy full SHA for 69d7b44

• Merge updates from upstream sort changes.

  NVnavkumar

  committed
  fec9f0c

  Copy full SHA for fec9f0c

Commits on Sep 11, 2023

• Forking Arm because of collection hierarchy handling

  NVnavkumar

  committed
  abf994f

  Copy full SHA for abf994f

Commits on Sep 12, 2023

• Updated forked versions for Scala 2.13

  NVnavkumar

  committed
  577e5a4

  Copy full SHA for 577e5a4

• Updated toSeq usage for Scala 2.13

  NVnavkumar

  committed
  130e52c

  Copy full SHA for 130e52c

• Handle forked scala-2.12 and scala-2.13 directories for build

  NVnavkumar

  committed
  bf24baa

  Copy full SHA for bf24baa

• For Scala 2.13 need to convert Set to Seq when using .map(). This is to avoid an Ordering requirement.

  NVnavkumar

  committed
  220e0d3

  Copy full SHA for 220e0d3

• Need to fork aggregate.scala because of Product with Serialize supertype of case objects.

  NVnavkumar

  committed
  742e043

  Copy full SHA for 742e043

Commits on Sep 19, 2023

• Handle failed implicit SAM conversion for Scala 2.13

  Show description for 6d50c1e
  NVnavkumar

  committed
  6d50c1e

  Copy full SHA for 6d50c1e

Commits on Sep 20, 2023

• rewrite to a pure lambda to make it compile in 2.13

  Show description for d993ec9
  NVnavkumar

  committed
  d993ec9

  Copy full SHA for d993ec9

• Fix 3 more simple compile errors

  Show description for 22e0354
  NVnavkumar

  committed
  22e0354

  Copy full SHA for 22e0354

• Add public override of interface method (in this case the default override from Java is insufficient)

  NVnavkumar

  committed
  cfb4535

  Copy full SHA for cfb4535

• The enqueue method with a single varargs list was removed in Scala 2.13, so need to convert this to a foreach loop instead.

  NVnavkumar

  committed
  7e2cde2

  Copy full SHA for 7e2cde2

Commits on Sep 21, 2023

• Remove unneeded condition for Scala 2.13

  Show description for 8c07562
  NVnavkumar

  committed
  8c07562

  Copy full SHA for 8c07562

• This needs to take in scala.collection.Seq

  NVnavkumar

  committed
  ef58914

  Copy full SHA for ef58914

• Add @SuppressWarnings annotation for Scala 2.13 support

  NVnavkumar

  committed
  3d95050

  Copy full SHA for 3d95050

• Fix Scala 2.13 compiler errors in shuffle-plugin

  Show description for c3467b7
  NVnavkumar

  committed
  c3467b7

  Copy full SHA for c3467b7

• Merge branch 'branch-23.10' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  df0d646

  Copy full SHA for df0d646

• Fix Scala 2.13 compiler issues in udf-compiler

  NVnavkumar

  committed
  7f2fe5d

  Copy full SHA for 7f2fe5d

• Remove duplicate sections

  Show description for e4868bb
  NVnavkumar

  committed
  e4868bb

  Copy full SHA for e4868bb

• Fix delta-lake and datagen compile issues.

  Show description for b041889
  NVnavkumar

  committed
  b041889

  Copy full SHA for b041889

• Update scalatest files to compile with Scala 2.13

  Show description for 4aecd9a
  NVnavkumar

  committed
  4aecd9a

  Copy full SHA for 4aecd9a

Commits on Sep 22, 2023

• Add all actual Scala 2.13 build versions.

  NVnavkumar

  committed
  aecfa13

  Copy full SHA for aecfa13

• Shimming this part of RAPIDS shuffle to handle Spark 3.4+ change that only affects Scala 2.13 builds

  Show description for 8b794ae
  NVnavkumar

  committed
  8b794ae

  Copy full SHA for 8b794ae

• Merge branch 'branch-23.10' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  66162df

  Copy full SHA for 66162df

Commits on Sep 25, 2023

• Merge branch 'branch-23.10' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  282f8d3

  Copy full SHA for 282f8d3

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  2a6b6c6

  Copy full SHA for 2a6b6c6

Commits on Sep 26, 2023

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  ef7e49b

  Copy full SHA for ef7e49b

Commits on Sep 27, 2023

• Fork this code to handle differences in ArrayBuffer with Scala 2.13

  NVnavkumar

  committed
  5bd1619

  Copy full SHA for 5bd1619

• Add Comment to clarify script is copied from Apache Spark

  Show description for 33d3370
  NVnavkumar

  committed
  33d3370

  Copy full SHA for 33d3370

• Change cast to Seq to using toSeq method

  NVnavkumar

  committed
  9363438

  Copy full SHA for 9363438

Commits on Sep 28, 2023

• Handle Map properly for Scala 2.13

  Show description for 7123930
  NVnavkumar

  committed
  7123930

  Copy full SHA for 7123930

• Fix NPE caused by Scala 2.13 Seq changes.

  Show description for 581f022
  NVnavkumar

  committed
  581f022

  Copy full SHA for 581f022

Commits on Sep 29, 2023

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  b1ec78a

  Copy full SHA for b1ec78a

Commits on Oct 3, 2023

• Fix some incorrect versions in pom.xml

  Show description for 2c62f49
  NVnavkumar

  committed
  2c62f49

  Copy full SHA for 2c62f49

Commits on Oct 4, 2023

• Build script updates

  Show description for 5b7fe19
  NVnavkumar

  committed
  5b7fe19

  Copy full SHA for 5b7fe19

Commits on Oct 5, 2023

• Skip decimal tests in arithmetic_ops_test that fail on Scala 2.13 CPU

  Show description for 5fa0c4c
  NVnavkumar

  committed
  5fa0c4c

  Copy full SHA for 5fa0c4c

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  6d2021c

  Copy full SHA for 6d2021c

• Include Scala213 build profiles in dist pom

  Show description for 67e52e9
  NVnavkumar

  committed
  67e52e9

  Copy full SHA for 67e52e9

Commits on Oct 6, 2023

• Revert this change

  Show description for 4c4cc23
  NVnavkumar

  committed
  4c4cc23

  Copy full SHA for 4c4cc23

• Remove commented out code

  Show description for 0973c52
  NVnavkumar

  committed
  0973c52

  Copy full SHA for 0973c52

• Remove commented out code

  Show description for 2499e45
  NVnavkumar

  committed
  2499e45

  Copy full SHA for 2499e45

Commits on Oct 10, 2023

• Address feedback, refactor Arm, and add reference to SPARK JIRA in integration tests.

  NVnavkumar

  committed
  e2901c0

  Copy full SHA for e2901c0

• Refactor aggregate.scala for 2.12/2.13 difference

  Show description for 8d42334
  NVnavkumar

  committed
  8d42334

  Copy full SHA for 8d42334

Commits on Oct 11, 2023

• Refactor GpuSortExec for 2.12/2.13

  NVnavkumar

  committed
  4c0e448

  Copy full SHA for 4c0e448

• Revert 2.13 pom changes

  Show description for 870d19d
  NVnavkumar

  committed
  870d19d

  Copy full SHA for 870d19d

• Refactor GpuSorter out of SortUtils.scala to handle 2.13/2.12 differences

  Show description for f8866f9
  NVnavkumar

  committed
  f8866f9

  Copy full SHA for f8866f9

Commits on Oct 13, 2023

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  85c8fa4

  Copy full SHA for 85c8fa4

Commits on Oct 16, 2023

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  6aab915

  Copy full SHA for 6aab915

• Fix scalastyle issue with importing SeqLike directly

  NVnavkumar

  committed
  06a558f

  Copy full SHA for 06a558f

Commits on Oct 17, 2023

• Fix some style issues.

  NVnavkumar

  committed
  46c414d

  Copy full SHA for 46c414d

• Scala 2.13 build using separate directory

  Show description for d49619e
  NVnavkumar

  committed
  d49619e

  Copy full SHA for d49619e

• Remove the change-scala-version.sh script.

  Show description for 5c6f672
  NVnavkumar

  committed
  5c6f672

  Copy full SHA for 5c6f672

• Support delta lake source import for Scala 2.13

  Show description for 44669c9
  NVnavkumar

  committed
  44669c9

  Copy full SHA for 44669c9

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  3cec238

  Copy full SHA for 3cec238

• Cleanup Delta Lake source import for Scala 2.13

  Show description for 9f41a3a
  NVnavkumar

  committed
  9f41a3a

  Copy full SHA for 9f41a3a

• Fix some pom files for scala 2.13

  Show description for a3ef04d
  NVnavkumar

  committed
  a3ef04d

  Copy full SHA for a3ef04d

• Make build/buildall work with Scala 2.13 to make proper plugin JAR

  Show description for 919ac68
  NVnavkumar

  committed
  919ac68

  Copy full SHA for 919ac68

Commits on Oct 18, 2023

• Scala 2.13 documentation

  Show description for 58a9632
  NVnavkumar

  committed
  58a9632

  Copy full SHA for 58a9632

• Add required profile for scala 2.13 versions.

  Show description for 865d6b8
  NVnavkumar

  committed
  865d6b8

  Copy full SHA for 865d6b8

• Adding pre-merge GitHub workflow for Scala 2.13

  Show description for 95171ec
  NVnavkumar

  committed
  95171ec

  Copy full SHA for 95171ec

• this shouldn't skip the first version

  Show description for 53cc51c
  NVnavkumar

  committed
  53cc51c

  Copy full SHA for 53cc51c

• Fix Scala version on artifactId on these poms

  Show description for 9bcb289
  NVnavkumar

  committed
  9bcb289

  Copy full SHA for 9bcb289

• Fix scala versions in github pre-merge hook

  Show description for 0b93bba
  NVnavkumar

  committed
  0b93bba

  Copy full SHA for 0b93bba

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  7e34de0

  Copy full SHA for 7e34de0

• Shim update for CDH 3.3.2

  Show description for 1ae14cb
  NVnavkumar

  committed
  1ae14cb

  Copy full SHA for 1ae14cb

• Another CDH 3.3.2 shim fix

  Show description for 924fe93
  NVnavkumar

  committed
  924fe93

  Copy full SHA for 924fe93

• Fix bug in profile name

  Show description for 4319799
  NVnavkumar

  committed
  4319799

  Copy full SHA for 4319799

• Shim update for CDH 3.3.2 in tests

  Show description for fd3407e
  NVnavkumar

  committed
  fd3407e

  Copy full SHA for fd3407e

Commits on Oct 19, 2023

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  fbe7838

  Copy full SHA for fbe7838

• Fix premerge line for github scala build files check

  Show description for 6c1c724
  NVnavkumar

  committed
  6c1c724

  Copy full SHA for 6c1c724

• Update clean task to avoid 2.12 pom cleaning out Scala 2.13 files

  Show description for 83680b3
  NVnavkumar

  committed
  83680b3

  Copy full SHA for 83680b3

Commits on Oct 20, 2023

• Handle integration tests with resources

  Show description for 56accae
  NVnavkumar

  committed
  56accae

  Copy full SHA for 56accae

• Add premerge tests for Scala 2.13

  Show description for f467272
  NVnavkumar

  committed
  f467272

  Copy full SHA for f467272

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  f17743a

  Copy full SHA for f17743a

• Fix premerge script syntax error

  Show description for 9c90e17
  NVnavkumar

  committed
  9c90e17

  Copy full SHA for 9c90e17

• Fix premerge script syntax error 2

  Show description for 6d671bd
  NVnavkumar

  committed
  6d671bd

  Copy full SHA for 6d671bd

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  1f3c5ad

  Copy full SHA for 1f3c5ad

• Fix for Databricks refactors that were not merged in properly before

  Show description for 1457ad9
  NVnavkumar

  committed
  1457ad9

  Copy full SHA for 1457ad9

• Merge branch 'scala-213' of github.com:NVnavkumar/spark-rapids into scala-213

  NVnavkumar

  committed
  fcde3eb

  Copy full SHA for fcde3eb

• Add log message for Scala version

  Show description for 046e9b5
  NVnavkumar

  committed
  046e9b5

  Copy full SHA for 046e9b5

Commits on Oct 23, 2023

• Fix premerge profile for Scala 2.13

  Show description for 7011a5a
  NVnavkumar

  committed
  7011a5a

  Copy full SHA for 7011a5a

• Fix Jenkins MVN_URM_MIRROR

  Show description for 6279a06
  NVnavkumar

  committed
  6279a06

  Copy full SHA for 6279a06

• Need these to match up properly

  Show description for 0892562
  NVnavkumar

  committed
  0892562

  Copy full SHA for 0892562

• Revert this change

  Show description for 82456ae
  NVnavkumar

  committed
  82456ae

  Copy full SHA for 82456ae

• Create symlink when handling relative jenkins path

  Show description for 096d775
  NVnavkumar

  committed
  096d775

  Copy full SHA for 096d775

• Fix symlink command

  Show description for ab25ace
  NVnavkumar

  committed
  ab25ace

  Copy full SHA for ab25ace

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  835e29b

  Copy full SHA for 835e29b

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  be09252

  Copy full SHA for be09252

• Fix toSeq issue in GpuSemaphore

  Show description for b8b6463
  NVnavkumar

  committed
  b8b6463

  Copy full SHA for b8b6463

Commits on Oct 24, 2023

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  6277529

  Copy full SHA for 6277529

• Update noSnapshots profile in Scala2.13 for CI purposes.

  NVnavkumar

  committed
  e7301f6

  Copy full SHA for e7301f6

• Update premerge to run Integration tests for Scala 2.13

  Show description for 4c359e7
  NVnavkumar

  committed
  4c359e7

  Copy full SHA for 4c359e7

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  c138485

  Copy full SHA for c138485

• More pom file updates from upstream

  Show description for 92dbb00
  NVnavkumar

  committed
  92dbb00

  Copy full SHA for 92dbb00

Commits on Oct 25, 2023

• Update premerge build script to separate out 2.13 tasks.

  Show description for f52907c
  NVnavkumar

  committed
  f52907c

  Copy full SHA for f52907c

• Fixes for integration tests based on feedback

  Show description for 9fe0e49
  NVnavkumar

  committed
  9fe0e49

  Copy full SHA for 9fe0e49

• Merge branch 'branch-23.12' of github.com:NVIDIA/spark-rapids into scala-213

  NVnavkumar

  committed
  db33a80

  Copy full SHA for db33a80

• Updates for pre-merge CI from Tim

  Show description for ffb45ba
  NVnavkumar

  committed
  ffb45ba

  Copy full SHA for ffb45ba

• signing off

  Show description for 4ed4815
  NVnavkumar

  committed
  4ed4815

  Copy full SHA for 4ed4815

Commits on Oct 26, 2023

• Remove checking scala2.13 folder

  Show description for c88320b
  NvTimLiu

  committed
  c88320b

  Copy full SHA for c88320b

• fix typo

  NvTimLiu

  committed
  5db5520

  Copy full SHA for 5db5520

• Refactor GpuSorter further to keep it in SortUtils and just shim out the Stack implementation

  Show description for eb2dbd3
  NVnavkumar

  committed
  eb2dbd3

  Copy full SHA for eb2dbd3

• remove commented out code and add comment for clarification

  Show description for b375ba3
  NVnavkumar

  committed
  b375ba3

  Copy full SHA for b375ba3

• Fix issue with copyrights

  Show description for e6409c8
  NVnavkumar

  committed
  e6409c8

  Copy full SHA for e6409c8

• use asIsntanceOf[...] to resolve this Scala 2.13/2.12 discrepancy

  Show description for b1e774c
  NVnavkumar

  committed
  b1e774c

  Copy full SHA for b1e774c

• Fix newline end of file

  Show description for bbb6750
  NVnavkumar

  committed
  bbb6750

  Copy full SHA for bbb6750

• Update documentation

  Show description for 4a41342
  NVnavkumar

  committed
  4a41342

  Copy full SHA for 4a41342

• Update documentation 2

  Show description for e0ff07c
  NVnavkumar

  committed
  e0ff07c

  Copy full SHA for e0ff07c