scala3: compile akka-http-core and pass all its tests #4097

hughsimpson · 2022-04-08T10:49:00Z

Enables and passes all tests in akka-http-core for scala 3.1.1 and 2.13.8
All files in that module deduplicated between scala-2 and scala-3

References #4079, task "akka-http-core"

…ing values" and "If-Match dispatching"

lightbend-cla-validator · 2022-04-08T10:49:02Z

Hi @hughsimpson,

Thank you for your contribution! We really value the time you've taken to put this together.

Before we proceed with reviewing this pull request, please sign the Lightbend Contributors License Agreement:

https://www.lightbend.com/contribute/cla

hughsimpson · 2022-04-08T10:50:20Z

akka-parsing/src/main/scala-3/akka/parboiled2/support/TailSwitch.scala

-      : TailSwitch[L, T, R] { type Out = TailSwitch0[L, L, T, T, R, HNil] } = `n/a`
+  implicit def tailSwitch[L <: _ :: _, T <: _ :: _, R <: HList]
+  : TailSwitch[L, T, R] { type Out = TailSwitch0[L, L, T, T, R, HNil] } = `n/a`
+  /// Optimisations to reduce compilation times to something tolerable


I couldn't figure out how to solve the general issue, but having some special-case implicits to short-circuit logic in certain common patterns seems to be enough for the usages in akka-http to compile

I've been trying to solve this in a more general way but it looks like this would require some fixes/optimizations in the compiler itself.

Yeah, I had the same feeling... I poked around a little but it looks like the main issue is that the compiler is 'surprisingly slow' at evaluating the compile-time type reduction. Since it blows up so quickly, I guess some resolution stage must be the wrong O...

Anyway apologies for stepping on your toes with this one. I got rather carried away testing that the workaround held good for all the cases we encounter in the parsers 😅

hughsimpson · 2022-04-08T10:57:41Z

Have signed CLA

hughsimpson · 2022-04-08T11:26:31Z

akka-http-core/src/main/scala/akka/http/impl/model/parser/SimpleHeaders.scala

@@ -100,14 +100,14 @@ private[parser] trait SimpleHeaders { this: Parser with CommonRules with CommonA

  // http://tools.ietf.org/html/rfc7233#section-4.2
  def `content-range` = rule {
-    (`byte-content-range` | `other-content-range`) ~ EOI ~> (`Content-Range`(_, _))
+    (`byte-content-range` ~ EOI ~> (`Content-Range`(_, _)) | `other-content-range` ~ EOI ~> (`Content-Range`(_, _)))


The original form is thrown off by the different subtypes, but this doesn't feel too bad an expansion...

Yep, agreed.

akka-http-core/src/main/scala/akka/http/impl/model/parser/UriParser.scala

hughsimpson · 2022-04-08T11:28:53Z

akka-http-core/src/test/scala/akka/testkit/AkkaSpec.scala

@@ -72,7 +75,7 @@ abstract class AkkaSpec(_system: ActorSystem)

  def this() = this(ActorSystem(AkkaSpec.getCallerName(getClass), AkkaSpec.testConf))

-  val log: LoggingAdapter = Logging(system, this.getClass)
+  val log: LoggingAdapter = Logging(system, this.getClass.asInstanceOf[Class[Any]])


I don't really understand this one TBQH...

I tried variations but also failed to come up something better. It seems some automatic conversions wrt to Java wildcard types don't work as well (or at least different) in Scala 3 than before.

project/Dependencies.scala

jrudolph · 2022-04-11T07:53:16Z

Wow, great that you tackled this, @hughsimpson! I'm going to have a thorough look later.

jrudolph

Looking great. The only thing that probably needs fixing is the error message assertions in tests to make sure they work for both Scala 2 and Scala 3. I'll have a look what's going on there.

jrudolph · 2022-04-11T10:43:37Z

akka-http-core/src/main/scala/akka/http/impl/model/parser/SimpleHeaders.scala

@@ -100,14 +100,14 @@ private[parser] trait SimpleHeaders { this: Parser with CommonRules with CommonA

  // http://tools.ietf.org/html/rfc7233#section-4.2
  def `content-range` = rule {
-    (`byte-content-range` | `other-content-range`) ~ EOI ~> (`Content-Range`(_, _))
+    (`byte-content-range` ~ EOI ~> (`Content-Range`(_, _)) | `other-content-range` ~ EOI ~> (`Content-Range`(_, _)))


Yep, agreed.

akka-http-core/src/main/scala/akka/http/impl/model/parser/UriParser.scala

akka-http-core/src/test/scala/akka/http/impl/model/parser/HttpHeaderSpec.scala

project/Dependencies.scala

akka-parsing/src/main/scala-3/akka/parboiled2/support/TailSwitch.scala

jrudolph · 2022-04-11T11:52:19Z

akka-http-core/src/test/scala/akka/testkit/AkkaSpec.scala

@@ -72,7 +75,7 @@ abstract class AkkaSpec(_system: ActorSystem)

  def this() = this(ActorSystem(AkkaSpec.getCallerName(getClass), AkkaSpec.testConf))

-  val log: LoggingAdapter = Logging(system, this.getClass)
+  val log: LoggingAdapter = Logging(system, this.getClass.asInstanceOf[Class[Any]])


I tried variations but also failed to come up something better. It seems some automatic conversions wrt to Java wildcard types don't work as well (or at least different) in Scala 3 than before.

jrudolph

LGTM, no blockers, if anything, it's the sorry shape of testing in the Scala 3 branch...

Thanks a lot, @hughsimpson, of getting this important milestone in place!

prolativ · 2022-04-11T12:42:01Z

@jrudolph @hughsimpson it looks I've found the core reason of the problem with TailSwitch scala/scala3#14903

jrudolph · 2022-04-11T12:45:34Z

@jrudolph @hughsimpson it looks I've found the core reason of the problem with TailSwitch lampepfl/dotty#14903

Great work. While the current solution works for akka-http, there are probably other parboiled2 parsers where the workaround still wouldn't be enough, so good to make progress on a more general solution!

Hugh Simpson added 9 commits April 8, 2022 09:42

most tests passing for akka-http-core

7c2deeb

two failing tests: "survive parsing a URI with thousands of query str…

a8ba2c9

…ing values" and "If-Match dispatching"

scala 2.13 passing tests

1589f06

dedup some more

89629e8

continue to dedup

644e0d4

dedup uri parser

97a973a

dedup remaining parsers

414cfe4

dedup HttpServerSpec

0e8cce5

dedup UriSpec

5fb5b44

hughsimpson commented Apr 8, 2022

View reviewed changes

hughsimpson changed the title ~~Compile akka http core~~ Compile akka-http-core Apr 8, 2022

hughsimpson commented Apr 8, 2022

View reviewed changes

akka-http-core/src/main/scala/akka/http/impl/model/parser/UriParser.scala Show resolved Hide resolved

revert some experimental changes

f84d23a

hughsimpson commented Apr 8, 2022

View reviewed changes

Hugh Simpson added 9 commits April 8, 2022 12:34

revert some other gratuitous changes

4fdec01

dedup HttpHeaderSpec, move failing test into HttpHeaderAuxSpec

d91f738

rm imports

3f1ae36

further minimisation of diff

90abc67

null has to be explicitly typed for the test-case to fail in scala 3

745ac4a

rm ws diff

dcd51f9

revert import diff

37e35e0

fmt akka-parsing

248f3f8

bumping scala3 version to 3.1.2 should also improve compilation times

689f0ce

hughsimpson commented Apr 8, 2022

View reviewed changes

project/Dependencies.scala Show resolved Hide resolved

jrudolph reviewed Apr 11, 2022

View reviewed changes

jrudolph approved these changes Apr 11, 2022

View reviewed changes

jrudolph changed the title ~~Compile akka-http-core~~ scala3: compile akka-http-core and pass all its tests Apr 11, 2022

jrudolph merged commit 0a979b8 into akka:scala-3 Apr 11, 2022

jrudolph mentioned this pull request Apr 11, 2022

Ongoing Scala 3 migration effort #4079

Merged

22 tasks

jrudolph mentioned this pull request Apr 11, 2022

Scala 3: very slow / hanging rule compilation for ~ chaining subrules sirthias/parboiled2#360

Closed

hughsimpson deleted the compile_akka-http-core branch April 13, 2022 20:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

scala3: compile akka-http-core and pass all its tests #4097

scala3: compile akka-http-core and pass all its tests #4097

hughsimpson commented Apr 8, 2022 •

edited

Loading

lightbend-cla-validator commented Apr 8, 2022

hughsimpson Apr 8, 2022 •

edited

Loading

prolativ Apr 8, 2022

hughsimpson Apr 8, 2022 •

edited

Loading

hughsimpson Apr 10, 2022

hughsimpson commented Apr 8, 2022

hughsimpson Apr 8, 2022

jrudolph Apr 11, 2022

hughsimpson Apr 8, 2022

jrudolph Apr 11, 2022

jrudolph commented Apr 11, 2022

jrudolph left a comment

jrudolph Apr 11, 2022

jrudolph Apr 11, 2022

jrudolph left a comment

prolativ commented Apr 11, 2022

jrudolph commented Apr 11, 2022

scala3: compile akka-http-core and pass all its tests #4097

scala3: compile akka-http-core and pass all its tests #4097

Conversation

hughsimpson commented Apr 8, 2022 • edited Loading

lightbend-cla-validator commented Apr 8, 2022

hughsimpson Apr 8, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hughsimpson Apr 8, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hughsimpson commented Apr 8, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrudolph commented Apr 11, 2022

jrudolph left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jrudolph left a comment

Choose a reason for hiding this comment

prolativ commented Apr 11, 2022

jrudolph commented Apr 11, 2022

hughsimpson commented Apr 8, 2022 •

edited

Loading

hughsimpson Apr 8, 2022 •

edited

Loading

hughsimpson Apr 8, 2022 •

edited

Loading