Spark 3.5 update

[pomadchin]

Update up to Spark 3.5 and drop Spark 3.2.

It looks like there are some changes again: #754

[chris-twiner]

yup, but so far I've only found this on the encoder side

java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.ScalaReflection$.isNativeType(Lorg/apache/spark/sql/types/DataType;)Z

	at org.apache.spark.sql.reflection.package$.isNativeType(package.scala:16)
	at frameless.TypedEncoder$$anon$17.toCatalyst(TypedEncoder.scala:343)
	at frameless.TypedEncoder$$anon$19.toCatalyst(TypedEncoder.scala:505)
	at frameless.RecordEncoder.$anonfun$toCatalyst$2(RecordEncoder.scala:155)
	at scala.collection.immutable.List.map(List.scala:297)
	at frameless.RecordEncoder.toCatalyst(RecordEncoder.scala:153)
	at frameless.TypedExpressionEncoder$.apply(TypedExpressionEncoder.scala:28)
	at com.sparkutils.quality.impl.EncodersImplicits.ruleEngineResultExpEnc(SparkHelpers.scala:94)

[chris-twiner]

Seems I missed one forwarded call from the 3.4 support functions/package.scala and the function doesn't exist anymore so the forwarding reflection/package also breaks.

No other compiler issues and after fixing that all tests pass CI. I've kept the functional change commit and build fix separate from the mass of scalafmt changes but it's just 3.5 in build.sbt plus the two above changes and readme. I also had to bump the heap.

PR: #759

[chris-twiner]

fyi - @pomadchin @cchantep - the readme versions aren't correct., also the 0.15 wouldn't include 3.5 support either I've raised #761 to fix that

[chris-twiner]

@pomadchin @cchantep - hihi, any idea when you'll be able to release the support? In case it was thought to go out in 15.0 it's not included in the changes (reflection still calls oisNativeType in the downloaded source). If this has happened due to something in the PR please let me know so I can avoid making the same mistake.

[cchantep]

As merged, it is in snapshot

[chris-twiner]

I can confirm 0.15.0-2-79dfcb7-SNAPSHOT includes the fixes

[chris-twiner]

@pomadchin - given the prep merge and build on #772 would it be possible to release 0.16.0 ?

[pomadchin]

hey @chris-twiner yes! I'll cut a release soon.

[pomadchin]

Released! https://github.com/typelevel/frameless/releases/tag/v0.16.0

[chris-twiner]

@pomadchin - awesome present, many thanks!

I've also tested on DBR 14.0 and 14.1 where it all works swimmingly.

DBR 14.2, alas, does not, I'll raise a ticket when I know what's actually going on (assuming it's just StaticInvoke constructor has a new default param)