Problems with jsoniter compilation after 3.3.1

[Kordyjan]

Compiler version

Everything after 3.3.1.
3.3.2-RC1-bin-20230530-28915c4-NIGHTLY - 3.3.2-RC1-bin-20230630-c629090-NIGHTLY - xcheck-macros error
from 3.3.2-RC1-bin-20230703-0a21ecf-NIGHTLY - typer crash

Reproduction

git clone --branch v2.24.4 --depth 1 https://github.com/plokhotnyuk/jsoniter-scala.git
cd jsoniter-scala
sbt
++3.3.2-RC1-bin-20230630-c629090-NIGHTLY!
jsoniter-scala-macrosJVM/Test/compile

Output (click arrow to expand)

Up to 3.3.2-RC1-bin-20230630-c629090-NIGHTLY - xcheck-macros assertion fail about missing position
From 3.3.2-RC1-bin-20230703-0a21ecf-NIGHTLY:

[error] java.lang.AssertionError: assertion failed
[error] scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:11)
[error] dotty.tools.dotc.core.Types$LazyRef.ref(Types.scala:3111)
[error] dotty.tools.dotc.core.Types$Type.dealias1(Types.scala:1433)
[error] dotty.tools.dotc.core.Types$Type.dealias(Types.scala:1441)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:171)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:169)
[error] dotty.tools.dotc.core.Types$TypeAccumulator.op$proxy25$1(Types.scala:6246)
[error] dotty.tools.dotc.core.Types$TypeAccumulator.foldArgs$3(Types.scala:6246)
[error] dotty.tools.dotc.core.Types$TypeAccumulator.foldOver(Types.scala:6250)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:198)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:169)
[error] dotty.tools.dotc.core.Types$TypeAccumulator.foldOver(Types.scala:6284)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:198)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:169)
[error] dotty.tools.dotc.core.Types$TypeAccumulator.foldOver(Types.scala:6284)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:198)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:172)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:169)
[error] dotty.tools.dotc.core.Types$TypeAccumulator.op$proxy25$1(Types.scala:6246)
[error] dotty.tools.dotc.core.Types$TypeAccumulator.foldArgs$3(Types.scala:6246)
[error] dotty.tools.dotc.core.Types$TypeAccumulator.foldOver(Types.scala:6250)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:198)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:169)
[error] dotty.tools.dotc.core.Types$TypeAccumulator.foldOver(Types.scala:6284)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:198)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:169)
[error] dotty.tools.dotc.core.Types$TypeAccumulator.foldOver(Types.scala:6284)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:198)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:172)
[error] dotty.tools.dotc.typer.Inferencing$IsFullyDefinedAccumulator.apply(Inferencing.scala:169)
[...]

[nicolasstucki]

Up to 3.3.2-RC1-bin-20230705-13f877b-NIGHTLY - xcheck-macros assertion fail about missing position

I did not manage to reproduce this one. I get the same failure that is in the details.

[Kordyjan]

I fixed the ranges in the original comment.

[WojciechMazur]

It might be possible that it's related to #18257 based on JsonCodecMakerNewTypeSpec test which defined recursive jsons. Altough there is was either failing or crashing with stack overflow depending on the complexity of match type.

[nicolasstucki]

The missing position was fixed in #18178.

[nicolasstucki]

I have no insight on the LazyRef.ref assertion failure.

[WojciechMazur]

The compilation crash is still present in the latest 3.5.0-RC1-nightly. The code that triggers the crash is a single test https://github.com/plokhotnyuk/jsoniter-scala/blob/84d57daeada31c0cce2631489e6d7c71e57be37e/jsoniter-scala-macros/shared/src/test/scala-3/com/github/plokhotnyuk/jsoniter_scala/macros/JsonCodecMakerNewTypeSpec.scala#L97-L205
One of the reasons for the crash is the used recursive match type - this kind of constructs are not allowed. It's reproduction can be found in here #18257 (comment) - however what's important in this form it leads to infinite loop instead of crash.
The one thing that Scala compiler should do is to detect and break the cycle early. The recursive match type detection works only for simpler types, but in the complex ones, as the one in jsoniter, we never get any hints from compiler what goes wrong.

#Edit
Actually in many cases the match type recursion would be reported as error, however, there are some cases when we get crash with no errors, eg. this one leads to stack overflow and no context about recursion being involved when the recursion match is defined in call-by-name block

import scala.collection.mutable
trait JsonValueCodec[T]
trait JsonReader
trait JsonWriter

def verifySerDeser[T](codec: JsonValueCodec[T], obj: T, json: String): Unit = ???
def test(block: => Unit) = ???

def Test = test{
    type JsonPrimitive = String | Int | Double | Boolean | None.type
    type Rec[JA[_], JO[_], A] = A match { // FIXME: remove this workaround after adding support of recursive types
      case JsonPrimitive => JsonPrimitive | JA[Rec[JA, JO, JsonPrimitive]] | JO[Rec[JA, JO, JsonPrimitive]]
      case _ => A | JA[Rec[JA, JO, A]] | JO[Rec[JA, JO, A]]
    }

    type Json = Rec[[A] =>> mutable.Buffer[A], [A] =>> mutable.Map[String, A], JsonPrimitive]
    type JsonObject = mutable.Map[String, Json]
    type JsonArray = mutable.Buffer[Json]
    
    val jsonCodec: JsonValueCodec[Json] = new JsonValueCodec[Json] {
      val nullValue: Json = None
    }

    def obj(values: (String, Json)*): Json = ???
    def arr(values: Json*): Json = mutable.ArrayBuffer[Json](values: _*)

    verifySerDeser(
      jsonCodec,
      arr("VVV", 1.2, true, obj("WWW" -> None, "XXX" -> 777)),
      """["VVV",1.2,true,{"WWW":null,"XXX":777}]"""
    )
  }