From b342884268700db91dcdd0187087472b4861165e Mon Sep 17 00:00:00 2001 From: Tobias Roeser Date: Wed, 13 Dec 2023 12:54:11 +0100 Subject: [PATCH 1/4] Add `ScalaModule.scalacHelp` command ## Motivation As the Scala compiler is most likely only used via the build tool, and there is probably not the exact scalac version installed on the system, it can get tricky to find the right options for a specific Scala version. ## Usage This adds a new `scalacHelp` command to the `ScalaModule` with which one can get the scalac built-in help with Mill. ``` > mill foo.scalacHelp Output of scalac version: 2.13.11 with options: -help -Xlint:help Usage: scalac Standard options: -Dproperty=value Pass -Dproperty=value directly to the runtime system. -J Pass directly to the runtime system. -P:: Pass an option to a plugin -V Print a synopsis of verbose options. [false] -W Print a synopsis of warning options. [false] -Werror Fail the compilation if there are any warnings. [false] -X Print a synopsis of advanced options. [false] -Y Print a synopsis of private options. [false] -bootclasspath Override location of bootstrap class files. -classpath Specify where to find user class files. -d destination for generated classfiles. -dependencyfile Set dependency tracking file. -deprecation Emit warning and location for usages of deprecated APIs. See also -Wconf. [false] -encoding Specify character encoding used by source files. -explaintypes Explain type errors in more detail. [false] -extdirs Override location of installed extensions. -feature Emit warning and location for usages of features that should be imported explicitly. See also -Wconf. [false] -g: Set level of generated debugging info. (none,source,line,[vars],notailcalls) -help Print a synopsis of standard options [false] -javabootclasspath Override java boot classpath. -javaextdirs Override java extdirs classpath. -language: Enable or disable language features -no-specialization Ignore @specialize annotations. [false] -nobootcp Do not use the boot classpath for the scala jars. [false] -nowarn Generate no warnings. [false] -opt: Enable optimizations: `-opt:local`, `-opt:inline:`; `-opt:help` for details. -opt-inline-from: Patterns for classfile names from which to allow inlining, `help` for details. -opt-warnings: Enable optimizer warnings, `help` for details. -print Print program with Scala-specific features removed. [false] -release: Compile for a version of the Java API and target class file. (8,9,10,[11]) -rootdir The absolute path of the project root directory, usually the git/scm checkout. Used by -Wconf. -sourcepath Specify location(s) of source files. -toolcp Add to the runner classpath. -unchecked Enable additional warnings where generated code depends on assumptions. See also -Wconf. [false] -uniqid Uniquely tag all identifiers in debugging output. [false] -usejavacp Utilize the java.class.path in classpath resolution. [false] -usemanifestcp Utilize the manifest in classpath resolution. [false] -verbose Output messages about what the compiler is doing. [false] -version Print product version and exit. [false] @ A text file containing compiler arguments (options and source files) [false] Deprecated settings: -optimize Enables optimizations. [false] deprecated: Since 2.12, enables -opt:inline:**. This can be dangerous. -target: Target platform for object files. ([8],9,10,11) deprecated: Use -release instead to compile against the correct platform API. ``` The command also accepts arguments which will be passed directly to scalac. ``` > mill foo.scalacHelp -Xlint:help Output of scalac version: 2.13.11 with options: -Xlint:help Enable recommended warnings adapted-args An argument list was modified to match the receiver. nullary-unit `def f: Unit` looks like an accessor; add parens to look side-effecting. inaccessible Warn about inaccessible types in method signatures. infer-any A type argument was inferred as Any. missing-interpolator A string literal appears to be missing an interpolator id. doc-detached When running scaladoc, warn if a doc comment is discarded. private-shadow A private field (or class parameter) shadows a superclass field. type-parameter-shadow A local type parameter shadows a type already in scope. poly-implicit-overload Parameterized overloaded implicit methods are not visible as view bounds. option-implicit Option.apply used an implicit view. delayedinit-select Selecting member of DelayedInit. package-object-classes Class or object defined in package object. stars-align In a pattern, a sequence wildcard `_*` should match all of a repeated parameter. strict-unsealed-patmat Pattern match on an unsealed class without a catch-all. constant Evaluation of a constant arithmetic expression resulted in an error. unused Enable -Wunused:imports,privates,locals,implicits,nowarn. nonlocal-return A return statement used an exception for flow control. implicit-not-found Check @implicitNotFound and @implicitAmbiguous messages. serial @SerialVersionUID on traits and non-serializable classes. valpattern Enable pattern checks in val definitions. eta-zero Usage `f` of parameterless `def f()` resulted in eta-expansion, not empty application `f()`. eta-sam A method reference was eta-expanded but the expected SAM type was not annotated @FunctionalInterface. deprecation Enable -deprecation and also check @deprecated annotations. byname-implicit Block adapted by implicit with by-name parameter. recurse-with-default Recursive call used default argument. unit-special Warn for specialization of Unit in parameter position. multiarg-infix Infix operator was defined or used with multiarg operand. implicit-recursion Implicit resolves to an enclosing definition. universal-methods Require arg to is/asInstanceOf. No Unit receiver. numeric-methods Dubious usages, such as `42.isNaN`. arg-discard -Wvalue-discard for adapted arguments. int-div-to-float Warn when an integer division is converted (widened) to floating point: `(someInt / 2): Double`. Default: All choices are enabled by default. ``` ## Implementation I first tried to just pass the "-help" options to the zinc worker, but I found it does not output the exected built-in help. Hence, I just load the scala compiler in an isolated classloader. This is a fire-and-forget attempt, as it is most likely rarely used. --- scalalib/src/mill/scalalib/ScalaModule.scala | 42 +++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/scalalib/src/mill/scalalib/ScalaModule.scala b/scalalib/src/mill/scalalib/ScalaModule.scala index 2071117b7ce..7f82373fd97 100644 --- a/scalalib/src/mill/scalalib/ScalaModule.scala +++ b/scalalib/src/mill/scalalib/ScalaModule.scala @@ -7,11 +7,12 @@ import mill.util.{Jvm, Util} import mill.util.Jvm.createJar import mill.api.Loose.Agg import mill.scalalib.api.{CompilationResult, Versions, ZincWorkerUtil} - import mainargs.Flag import mill.scalalib.bsp.{BspBuildTarget, BspModule, ScalaBuildTarget, ScalaPlatform} import mill.scalalib.dependency.versions.{ValidVersion, Version} +import scala.reflect.internal.util.ScalaClassLoader + /** * Core configuration required to compile a single Scala compilation target */ @@ -87,6 +88,45 @@ trait ScalaModule extends JavaModule with TestModule.ScalaModuleBase { outer => ) } + /** + * Print the scala compile built-in help output. + * This is equivalent to running `scalac -help` + * + * @param args The option to pass to the scala compiler, e.g. "-Xlint:help". Default: "-help" + */ + def scalacHelp( + @mainargs.arg(doc = + """The option to pass to the scala compiler, e.g. "-Xlint:help". Default: "-help"""" + ) + args: String* + ): Command[Unit] = T.command { + val sv = scalaVersion() + + // TODO: do we need to handle compiler plugins? + val options: Seq[String] = if (args.isEmpty) Seq("-help") else args + T.log.info( + s"""Output of scalac version: ${sv} + | with options: ${options.mkString(" ")}""".stripMargin + ) + + val mainClassName = + if (sv.startsWith("2.")) "scala.tools.nsc.Main" + else "dotty.tools.MainGenericRunner" + + // Zinc isn't outputting any help with `-help` options, so we ask the compiler directly + val cp = scalaCompilerClasspath() + val cl = ScalaClassLoader.fromURLs(cp.toSeq.map(_.path.toNIO.toUri().toURL())) + val mainClass = cl.loadClass("scala.tools.nsc.Main") + val mainMethod = mainClass.getMethod("process", Seq(classOf[Array[String]]): _*) + val exitVal = mainMethod.invoke(null, options.toArray) + exitVal match { + case true | java.lang.Boolean.TRUE => Result.Success(()) + case false | java.lang.Boolean.FALSE => Result.Failure("Could not invoke scala compiler") + case x => Result.Failure(s"Got unexpected return type from the scala compile: ${x}") + } + () + } + /** * Allows you to make use of Scala compiler plugins. */ From 7d778e66febcd41448cf5a155fd439e9c0f5bc0c Mon Sep 17 00:00:00 2001 From: Tobias Roeser Date: Wed, 13 Dec 2023 14:30:45 +0100 Subject: [PATCH 2/4] Added support for Scala 3.x and older 2.x versions --- scalalib/src/mill/scalalib/ScalaModule.scala | 48 ++++++++++++++------ 1 file changed, 35 insertions(+), 13 deletions(-) diff --git a/scalalib/src/mill/scalalib/ScalaModule.scala b/scalalib/src/mill/scalalib/ScalaModule.scala index 7f82373fd97..3e9766942cd 100644 --- a/scalalib/src/mill/scalalib/ScalaModule.scala +++ b/scalalib/src/mill/scalalib/ScalaModule.scala @@ -106,25 +106,47 @@ trait ScalaModule extends JavaModule with TestModule.ScalaModuleBase { outer => val options: Seq[String] = if (args.isEmpty) Seq("-help") else args T.log.info( s"""Output of scalac version: ${sv} - | with options: ${options.mkString(" ")}""".stripMargin + | with options: ${options.mkString(" ")} + |""".stripMargin ) - val mainClassName = - if (sv.startsWith("2.")) "scala.tools.nsc.Main" - else "dotty.tools.MainGenericRunner" - // Zinc isn't outputting any help with `-help` options, so we ask the compiler directly val cp = scalaCompilerClasspath() val cl = ScalaClassLoader.fromURLs(cp.toSeq.map(_.path.toNIO.toUri().toURL())) - val mainClass = cl.loadClass("scala.tools.nsc.Main") - val mainMethod = mainClass.getMethod("process", Seq(classOf[Array[String]]): _*) - val exitVal = mainMethod.invoke(null, options.toArray) - exitVal match { - case true | java.lang.Boolean.TRUE => Result.Success(()) - case false | java.lang.Boolean.FALSE => Result.Failure("Could not invoke scala compiler") - case x => Result.Failure(s"Got unexpected return type from the scala compile: ${x}") + + def handleResult(trueIsSuccess: Boolean): PartialFunction[Any, Result[Unit]] = { + val ok = Result.Success(()) + val fail = Result.Failure("The compiler exited with errors (exit code 1)") + + { + case true | java.lang.Boolean.TRUE => if(trueIsSuccess) ok else fail + case false | java.lang.Boolean.FALSE => if(trueIsSuccess) fail else ok + case null if sv.startsWith("2.") => + // Scala 2.11 and earlier return `Unit` and require use to use the result value, + // which we don't want to implement for just a simple help output of an very old compiler + Result.Success(()) + case x => Result.Failure(s"Got unexpected return type from the scala compiler: ${x}") + } + } + + if (sv.startsWith("2.")) { + // Scala 2.x + val mainClass = cl.loadClass("scala.tools.nsc.Main") + val mainMethod = mainClass.getMethod("process", Seq(classOf[Array[String]]): _*) + val exitVal = mainMethod.invoke(null, options.toArray) + handleResult(true)(exitVal) + } else { + // Scala 3.x + val mainClass = cl.loadClass("dotty.tools.dotc.Main") + val mainMethod = mainClass.getMethod("process", Seq(classOf[Array[String]]): _*) + val resultClass = cl.loadClass("dotty.tools.dotc.reporting.Reporter") + val hasErrorsMethod = resultClass.getMethod("hasErrors") + val exitVal = mainMethod.invoke(null, options.toArray) + exitVal match { + case r if resultClass.isInstance(r) => handleResult(false)(hasErrorsMethod.invoke(r)) + case x => Result.Failure(s"Got unexpected return type from the scala compiler: ${x}") + } } - () } /** From 6ee988e2f6e9995cd388c95f2ca5db29f9f54831 Mon Sep 17 00:00:00 2001 From: Tobias Roeser Date: Wed, 13 Dec 2023 14:47:59 +0100 Subject: [PATCH 3/4] scalafmt --- scalalib/src/mill/scalalib/ScalaModule.scala | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scalalib/src/mill/scalalib/ScalaModule.scala b/scalalib/src/mill/scalalib/ScalaModule.scala index 3e9766942cd..1a4756ab0ab 100644 --- a/scalalib/src/mill/scalalib/ScalaModule.scala +++ b/scalalib/src/mill/scalalib/ScalaModule.scala @@ -119,8 +119,8 @@ trait ScalaModule extends JavaModule with TestModule.ScalaModuleBase { outer => val fail = Result.Failure("The compiler exited with errors (exit code 1)") { - case true | java.lang.Boolean.TRUE => if(trueIsSuccess) ok else fail - case false | java.lang.Boolean.FALSE => if(trueIsSuccess) fail else ok + case true | java.lang.Boolean.TRUE => if (trueIsSuccess) ok else fail + case false | java.lang.Boolean.FALSE => if (trueIsSuccess) fail else ok case null if sv.startsWith("2.") => // Scala 2.11 and earlier return `Unit` and require use to use the result value, // which we don't want to implement for just a simple help output of an very old compiler From f0f9624c9543f9cae79a08322b8dff11ea8d98fc Mon Sep 17 00:00:00 2001 From: Tobias Roeser Date: Wed, 13 Dec 2023 17:55:46 +0100 Subject: [PATCH 4/4] Close classloader after use --- scalalib/src/mill/scalalib/ScalaModule.scala | 63 ++++++++++---------- 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/scalalib/src/mill/scalalib/ScalaModule.scala b/scalalib/src/mill/scalalib/ScalaModule.scala index 1a4756ab0ab..9b852a892ea 100644 --- a/scalalib/src/mill/scalalib/ScalaModule.scala +++ b/scalalib/src/mill/scalalib/ScalaModule.scala @@ -12,6 +12,7 @@ import mill.scalalib.bsp.{BspBuildTarget, BspModule, ScalaBuildTarget, ScalaPlat import mill.scalalib.dependency.versions.{ValidVersion, Version} import scala.reflect.internal.util.ScalaClassLoader +import scala.util.Using /** * Core configuration required to compile a single Scala compilation target @@ -112,39 +113,39 @@ trait ScalaModule extends JavaModule with TestModule.ScalaModuleBase { outer => // Zinc isn't outputting any help with `-help` options, so we ask the compiler directly val cp = scalaCompilerClasspath() - val cl = ScalaClassLoader.fromURLs(cp.toSeq.map(_.path.toNIO.toUri().toURL())) - - def handleResult(trueIsSuccess: Boolean): PartialFunction[Any, Result[Unit]] = { - val ok = Result.Success(()) - val fail = Result.Failure("The compiler exited with errors (exit code 1)") - - { - case true | java.lang.Boolean.TRUE => if (trueIsSuccess) ok else fail - case false | java.lang.Boolean.FALSE => if (trueIsSuccess) fail else ok - case null if sv.startsWith("2.") => - // Scala 2.11 and earlier return `Unit` and require use to use the result value, - // which we don't want to implement for just a simple help output of an very old compiler - Result.Success(()) - case x => Result.Failure(s"Got unexpected return type from the scala compiler: ${x}") + Using.resource(ScalaClassLoader.fromURLs(cp.toSeq.map(_.path.toNIO.toUri().toURL()))) { cl => + def handleResult(trueIsSuccess: Boolean): PartialFunction[Any, Result[Unit]] = { + val ok = Result.Success(()) + val fail = Result.Failure("The compiler exited with errors (exit code 1)") + + { + case true | java.lang.Boolean.TRUE => if (trueIsSuccess) ok else fail + case false | java.lang.Boolean.FALSE => if (trueIsSuccess) fail else ok + case null if sv.startsWith("2.") => + // Scala 2.11 and earlier return `Unit` and require use to use the result value, + // which we don't want to implement for just a simple help output of an very old compiler + Result.Success(()) + case x => Result.Failure(s"Got unexpected return type from the scala compiler: ${x}") + } } - } - if (sv.startsWith("2.")) { - // Scala 2.x - val mainClass = cl.loadClass("scala.tools.nsc.Main") - val mainMethod = mainClass.getMethod("process", Seq(classOf[Array[String]]): _*) - val exitVal = mainMethod.invoke(null, options.toArray) - handleResult(true)(exitVal) - } else { - // Scala 3.x - val mainClass = cl.loadClass("dotty.tools.dotc.Main") - val mainMethod = mainClass.getMethod("process", Seq(classOf[Array[String]]): _*) - val resultClass = cl.loadClass("dotty.tools.dotc.reporting.Reporter") - val hasErrorsMethod = resultClass.getMethod("hasErrors") - val exitVal = mainMethod.invoke(null, options.toArray) - exitVal match { - case r if resultClass.isInstance(r) => handleResult(false)(hasErrorsMethod.invoke(r)) - case x => Result.Failure(s"Got unexpected return type from the scala compiler: ${x}") + if (sv.startsWith("2.")) { + // Scala 2.x + val mainClass = cl.loadClass("scala.tools.nsc.Main") + val mainMethod = mainClass.getMethod("process", Seq(classOf[Array[String]]): _*) + val exitVal = mainMethod.invoke(null, options.toArray) + handleResult(true)(exitVal) + } else { + // Scala 3.x + val mainClass = cl.loadClass("dotty.tools.dotc.Main") + val mainMethod = mainClass.getMethod("process", Seq(classOf[Array[String]]): _*) + val resultClass = cl.loadClass("dotty.tools.dotc.reporting.Reporter") + val hasErrorsMethod = resultClass.getMethod("hasErrors") + val exitVal = mainMethod.invoke(null, options.toArray) + exitVal match { + case r if resultClass.isInstance(r) => handleResult(false)(hasErrorsMethod.invoke(r)) + case x => Result.Failure(s"Got unexpected return type from the scala compiler: ${x}") + } } } }