Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extended mill init to support an existing Maven project #3756

Merged
merged 71 commits into from
Nov 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
ab80850
Extended `mill init` to support an existing Maven project
ajaychandran Oct 16, 2024
9cfa813
Renamed variable
ajaychandran Oct 17, 2024
6b1d573
Added magic import to generated build.mill
ajaychandran Oct 17, 2024
0e1dbe6
Added provided dependencies to compileIvyDeps
ajaychandran Oct 17, 2024
6c4ad41
Extended `mill init` to support an existing Maven project
ajaychandran Oct 16, 2024
b60147e
Renamed variable
ajaychandran Oct 17, 2024
db77ae6
Added magic import to generated build.mill
ajaychandran Oct 17, 2024
e28b83f
Added provided dependencies to compileIvyDeps
ajaychandran Oct 17, 2024
6fc9eb2
Merge remote-tracking branch 'origin/main'
ajaychandran Oct 17, 2024
229e95c
Added magic import to resolve base module in package build files
ajaychandran Oct 18, 2024
4b7948c
Fix test
ajaychandran Oct 18, 2024
6e7061e
Refactored out code generation
ajaychandran Oct 18, 2024
76e4bd5
Merge branch 'com-lihaoyi:main' into main
ajaychandran Oct 23, 2024
b784532
Merge branch 'com-lihaoyi:main' into main
ajaychandran Oct 25, 2024
d16915f
Merge branch 'com-lihaoyi:main' into main
ajaychandran Oct 28, 2024
3b5d140
Merge branch 'com-lihaoyi:main' into main
ajaychandran Nov 1, 2024
69273e8
Merge branch 'com-lihaoyi:main' into main
ajaychandran Nov 5, 2024
3658abb
- Addressed review comments
ajaychandran Nov 5, 2024
50f87b9
Renamed exception
ajaychandran Nov 5, 2024
70e4614
Fixed formatting
ajaychandran Nov 5, 2024
84098b7
Renamed symbolic operators
ajaychandran Nov 6, 2024
a71c46f
Added test case for avaje-config
ajaychandran Nov 6, 2024
34ac1e0
Added location of compile errors
ajaychandran Nov 6, 2024
dab05d0
Separated variable declarations
ajaychandran Nov 6, 2024
1d6e756
Changed implicit parameters to arguments.
ajaychandran Nov 6, 2024
acae1a8
Cleaned folder names
ajaychandran Nov 6, 2024
d5778a7
Cleaned object names
ajaychandran Nov 6, 2024
dd41e49
Merge branch 'com-lihaoyi:main' into main
ajaychandran Nov 7, 2024
52a11a0
Replaced arrow assertions with assert
ajaychandran Nov 6, 2024
f092903
Format build files generated in tests
ajaychandran Nov 6, 2024
50fe2ac
Removed support for build-helper-maven-plugin
ajaychandran Nov 6, 2024
86501ff
Aligned inheritance scheme with existing capabilities
ajaychandran Nov 7, 2024
4785683
Merge branch 'com-lihaoyi:main' into main
ajaychandran Nov 11, 2024
7b15c28
Added test class for owner project and cleaned existing tests
ajaychandran Nov 7, 2024
b0d45da
Reordered parameters
ajaychandran Nov 7, 2024
17d8e92
Improved generator algorithm
ajaychandran Nov 9, 2024
0bb1824
Annotated compile tasks
ajaychandran Nov 9, 2024
d641e2a
Replaced System.lineSeparator()
ajaychandran Nov 9, 2024
65ddee1
Renamed method and updated Scaladoc
ajaychandran Nov 9, 2024
5622b78
Simplified build IR
ajaychandran Nov 9, 2024
76b21c6
Added feature flag to compact build tree
ajaychandran Nov 11, 2024
ee93973
Updated comments for JPMS errors
ajaychandran Nov 11, 2024
4df7e8d
Added documentation with example
ajaychandran Nov 11, 2024
e3aa816
Refactored integration tests
ajaychandran Nov 11, 2024
8a229d8
Removed compile tasks for "pom" modules
ajaychandran Nov 11, 2024
ff40e9e
Merge branch 'com-lihaoyi:main' into main
ajaychandran Nov 11, 2024
46cd3b2
Improved integration tests
ajaychandran Nov 11, 2024
c3bcb72
Updated documentation
ajaychandran Nov 11, 2024
2ab9b05
- Fix error due to use of /* in command
ajaychandran Nov 11, 2024
f0b487f
Updated test golden files
ajaychandran Nov 11, 2024
75e704f
Compare lines instead of content to avoid file EOL differences
ajaychandran Nov 12, 2024
7018dde
Merge remote-tracking branch 'upstream/main'
ajaychandran Nov 12, 2024
895ffe2
Removed default parameter value
ajaychandran Nov 12, 2024
a01a8d2
Suppressed test assertion to avoid CI failure
ajaychandran Nov 12, 2024
fe6cc7b
Cleanup
ajaychandran Nov 12, 2024
b0839c8
Escape single backslash to handle Windows paths
ajaychandran Nov 12, 2024
e8683d9
Use raw interpolator for single backslashes
ajaychandran Nov 12, 2024
88cdcf6
debug
lihaoyi Nov 14, 2024
fbffb2a
use pprint literalize
lihaoyi Nov 14, 2024
0c56337
cleanup
lihaoyi Nov 14, 2024
e309e9b
docs
lihaoyi Nov 14, 2024
3449411
docs
lihaoyi Nov 14, 2024
0d2ddc0
fixtests
lihaoyi Nov 14, 2024
639c2a4
docs
lihaoyi Nov 14, 2024
6010bce
docs
lihaoyi Nov 14, 2024
219cef4
docs
lihaoyi Nov 14, 2024
ef66ca9
docs
lihaoyi Nov 14, 2024
9cb7be5
.
lihaoyi Nov 14, 2024
47a71fc
.
lihaoyi Nov 14, 2024
c08d6d0
.
lihaoyi Nov 14, 2024
a0a0d2c
.
lihaoyi Nov 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions build.mill
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,14 @@ object Deps {
val sonatypeCentralClient = ivy"com.lumidion::sonatype-central-client-requests:0.3.0"
val kotlinVersion = "2.0.21"
val kotlinCompiler = ivy"org.jetbrains.kotlin:kotlin-compiler:$kotlinVersion"
val mavenVersion = "3.9.9"
val mavenEmbedder = ivy"org.apache.maven:maven-embedder:$mavenVersion"
val mavenResolverVersion = "1.9.22"
val mavenResolverConnectorBasic = ivy"org.apache.maven.resolver:maven-resolver-connector-basic:$mavenResolverVersion"
val mavenResolverSupplier = ivy"org.apache.maven.resolver:maven-resolver-supplier:$mavenResolverVersion"
val mavenResolverTransportFile = ivy"org.apache.maven.resolver:maven-resolver-transport-file:$mavenResolverVersion"
val mavenResolverTransportHttp = ivy"org.apache.maven.resolver:maven-resolver-transport-http:$mavenResolverVersion"
val mavenResolverTransportWagon = ivy"org.apache.maven.resolver:maven-resolver-transport-wagon:$mavenResolverVersion"

object RuntimeDeps {
val dokkaVersion = "1.9.20"
Expand Down
1 change: 1 addition & 0 deletions dist/package.mill
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ object `package` extends RootModule with build.MillPublishJavaModule {

def testTransitiveDeps = build.runner.testTransitiveDeps() ++ Seq(
build.main.graphviz.testDep(),
build.main.maven.testDep(),
build.runner.linenumbers.testDep(),
build.scalalib.backgroundwrapper.testDep(),
build.contrib.bloop.testDep(),
Expand Down
2 changes: 2 additions & 0 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@
** xref:cli/flags.adoc[]
** xref:cli/alternate-installation.adoc[]
** xref:cli/builtin-commands.adoc[]
* Migrating to Mill
** xref:migrating/maven.adoc[]
// This section gives a tour of the various user-facing features of Mill:
// library deps, out folder, queries, tasks, etc.. These are things that
// every Mill user will likely encounter, and are touched upon in the various
Expand Down
155 changes: 155 additions & 0 deletions docs/modules/ROOT/pages/migrating/maven.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
= Migrating From Maven to Mill
:page-aliases: Migrating_A_Maven_Build_to_Mill.adoc
:icons: font

include::partial$gtag-config.adoc[]

The Mill `init` command can be used to convert a Maven build to Mill. This has
xref:#limitations[limitations] and is not intended to reliably migrate 100% of
Maven builds out there in the wild, but is instead meant to provide the basic
scaffolding of a Mill build for you to further refine and update manually.

Each Maven module with a `pom.xml` is converted to a Mill `build.mill`/`package.mill`
file containing a top-level `MavenModule`. A nested `test` module is defined if both:

* `src/test` exists
* a supported xref:javalib/testing.adoc[test framework] is detected (for a _tests only_
module with test sources in `src/main/java`)


Again, note that `mill init` importing Maven builds is best effort.
This means that while small projects can be expected to complete without issue:

include::partial$example/javalib/migrating/1-maven-complete.adoc[]

More larger projects often require some manual tweaking in order to work:

include::partial$example/javalib/migrating/2-maven-incomplete.adoc[]

Nevertheless, even for larger builds `mill init` automates most of the tedious
busy-work of writing `build.mill`/`package.mill` files, and makes it much quicker
to get a working Mill build for any existing Maven project.


== Capabilities

The conversion

* handles deeply nested modules
* captures project metadata
* configures dependencies for scopes:
** compile
** provided
** runtime
** test
* configures testing frameworks:
** JUnit 4
** JUnit 5
** TestNG
* configures multiple, compile and test, resource directories

=== Command line arguments
.name of generated base module trait defining project metadata settings
[source,sh]
----
./mill init --base-module MyModule
----
.name of generated nested test module (defaults to `test`)
[source,sh]
----
./mill init --test-module test
----
.name of generated companion object defining constants for dependencies
[source,sh]
----
./mill init --deps-object Deps
----
.capture properties defined in `pom.xml` for publishing
[source,sh]
----
./mill init --publish-properties
----
.merge build files generated for a multi-module build
[source,sh]
----
./mill init --merge
----

.use cache for Maven repository system
[source,sh]
----
./mill init --cache-repository
----
.process Maven plugin executions and configurations
[source,sh]
----
./mill init --process-plugins
----

=== Verified projects

The conversion has been tested with the following projects:

* https://github.com/fusesource/jansi/archive/refs/tags/jansi-2.4.1.zip[jansi]
[source,sh]
----
./mill init --base-module JansiModule --deps-object Deps --cache-repository --process-plugins
----

* https://github.com/davidmoten/geo/archive/refs/tags/0.8.1.zip[geo] (multi-module build)
[source,sh]
----
./mill init --base-module GeoModule --deps-object Deps --merge --cache-repository --process-plugins
----

Post `init`, the following tasks were executed successfully:

* `compile`
* `test`
* `publishLocal`

[#limitations]
== Limitations

The conversion does not support

* build extensions
* build profiles
* non-Java (native) sources

Maven plugin support is limited to

* https://maven.apache.org/plugins/maven-compiler-plugin/[maven-compiler-plugin]

[TIP]
====
These limitations can be overcome by:

* configuring equivalent Mill xref:extending/contrib-plugins.adoc[contrib]
or xref:extending/thirdparty-plugins.adoc[third party] plugins
* defining custom xref:extending/writing-plugins.adoc[plugins]
* defining custom xref:fundamentals/tasks.adoc[tasks]
* defining custom xref:fundamentals/cross-builds.adoc[cross modules]
====

== FAQ

.How to fix compilation errors in generated build files?

This could happen if a module and task name collision occurs. Either rename the module or enclose the name in backticks.


.How to fix JPMS `module not found` compilation errors?

Set https://github.com/tfesenko/Java-Modules-JPMS-CheatSheet#how-to-export-or-open-a-package[additional command line options]
for dependencies.


.How to fix test compilation errors?

* The test framework configured may be for an unsupported version; try upgrading the
corresponding dependencies.
* Mill does not add `provided` dependencies to the transitive dependencies of the nested
test module; specify the dependencies again, in one of `ivyDeps`, `compileIvyDeps`, `runIvyDeps`, in the test module.


29 changes: 29 additions & 0 deletions example/javalib/migrating/1-maven-complete/build.mill
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
/** Usage

> rm build.mill # remove any existing build file

> git init .
> git remote add -f origin https://github.com/davidmoten/geo.git
> git checkout 0.8.1 # example multi-module Java project using JUnit4

> ./mill init
converting Maven build
writing Mill build file to geo/package.mill
writing Mill build file to geo-mem/package.mill
writing Mill build file to build.mill
init completed, run "mill resolve _" to list available tasks

> ./mill __.compile
compiling 9 Java sources to .../out/geo/compile.dest/classes ...
compiling 2 Java sources to .../out/geo-mem/compile.dest/classes ...
compiling 10 Java sources to .../out/geo/test/compile.dest/classes ...
done compiling
compiling 1 Java source to .../out/geo-mem/test/compile.dest/classes ...
done compiling

> ./mill __.test # all tests pass immediately
Test run com.github.davidmoten.geo.GeoHashTest finished: 0 failed, 0 ignored, 66 total, ...
Test run com.github.davidmoten.geo.DirectionTest finished: 0 failed, 0 ignored, 1 total, ...
...

*/
24 changes: 24 additions & 0 deletions example/javalib/migrating/2-maven-incomplete/build.mill
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
/** Usage

> rm build.mill # remove any existing build file

> git init .
> git remote add -f origin https://github.com/dhatim/fastexcel.git
> git checkout 0.18.4
> # pom.xml has custom profiles for JPMS that mill init does not support
> # https://github.com/dhatim/fastexcel/blob/de56e786a1fe29351e2f8dc1d81b7cdd9196de4a/pom.xml#L251

> ./mill init
converting Maven build
writing Mill build file to fastexcel-writer/package.mill
writing Mill build file to fastexcel-reader/package.mill
writing Mill build file to e2e/package.mill
writing Mill build file to build.mill
init completed, run "mill resolve _" to list available tasks

> ./mill -k __.compile # Compilation needs manual tweaking to pass
error: ...fastexcel-reader/src/main/java/module-info.java:3:32: module not found: org.apache.commons.compress
error: ...fastexcel-reader/src/main/java/module-info.java:4:27: module not found: com.fasterxml.aalto
error: ...fastexcel-writer/src/main/java/module-info.java:2:14: module not found: opczip

*/
1 change: 1 addition & 0 deletions example/package.mill
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ object `package` extends RootModule with Module {
object dependencies extends Cross[ExampleCrossModuleJava](build.listIn(millSourcePath / "dependencies"))
object testing extends Cross[ExampleCrossModuleJava](build.listIn(millSourcePath / "testing"))
object linting extends Cross[ExampleCrossModuleJava](build.listIn(millSourcePath / "linting"))
object migrating extends Cross[ExampleCrossModuleJava](build.listIn(millSourcePath / "migrating"))
object publishing extends Cross[ExampleCrossModuleJava](build.listIn(millSourcePath / "publishing"))
object web extends Cross[ExampleCrossModule](build.listIn(millSourcePath / "web"))
}
Expand Down
Loading
Loading