Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tidy up init command impl, make it unpack into current folder #3626

Merged
merged 57 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
29851e7
modifying init to fetch examples from releases page instead of using …
pawelsadlo Sep 20, 2024
68260e3
renaming init -> initmodule
pawelsadlo Sep 20, 2024
eaa5402
fix integration tests
pawelsadlo Sep 20, 2024
bf479c0
format
pawelsadlo Sep 20, 2024
838dab0
fixing tests
pawelsadlo Sep 21, 2024
2947bb9
fix
pawelsadlo Sep 21, 2024
40216f7
fix
pawelsadlo Sep 21, 2024
fdcef78
fix
pawelsadlo Sep 21, 2024
f72e879
fix
pawelsadlo Sep 21, 2024
605089a
Update build.mill
pawelsadlo Sep 21, 2024
185fdf9
fix
pawelsadlo Sep 21, 2024
e5b9756
Merge remote-tracking branch 'ps_mill/list-examples' into list-examples
pawelsadlo Sep 21, 2024
bd33abf
fix
pawelsadlo Sep 21, 2024
039afcc
fix
pawelsadlo Sep 21, 2024
c7623ac
fix
pawelsadlo Sep 21, 2024
17c1b81
fix
pawelsadlo Sep 21, 2024
4b0bcbb
Update build.mill
pawelsadlo Sep 21, 2024
2ea02db
fix
pawelsadlo Sep 23, 2024
d4997a7
Merge remote-tracking branch 'ps_mill/list-examples' into list-examples
pawelsadlo Sep 23, 2024
a746cda
fix
pawelsadlo Sep 23, 2024
ecf5a65
format
pawelsadlo Sep 23, 2024
83244ce
sorting exampleList
pawelsadlo Sep 26, 2024
86b30ea
reverting forVersion param from examplePathsWithArtifactName
pawelsadlo Sep 26, 2024
52fb129
add show-all param handling
pawelsadlo Sep 26, 2024
c1d3bf8
format
pawelsadlo Sep 26, 2024
0eb5c0d
Merge branch 'main' into list-examples
pawelsadlo Sep 26, 2024
f5a52ed
Update MainModule.scala
pawelsadlo Sep 26, 2024
fc84b10
fix sorting
pawelsadlo Sep 26, 2024
45b45ef
Merge remote-tracking branch 'ps_mill/list-examples' into list-examples
pawelsadlo Sep 26, 2024
5badd4c
adding test for overriding directory while downloading example,
pawelsadlo Sep 27, 2024
505c588
Merge remote-tracking branch 'refs/remotes/origin/main' into list-exa…
pawelsadlo Sep 27, 2024
7081e4e
fixing download url
pawelsadlo Sep 27, 2024
38da55f
Revert "fixing download url"
pawelsadlo Sep 27, 2024
4731e07
adjusting for CI
pawelsadlo Sep 27, 2024
eadbab0
Revert "adjusting for CI"
pawelsadlo Sep 27, 2024
6b87184
parsing millLastTag from version
pawelsadlo Sep 28, 2024
5bc0e1b
Merge branch 'main' into list-examples
pawelsadlo Sep 28, 2024
738493d
Merge branch 'main' into list-examples
lihaoyi Sep 30, 2024
b067dbe
Merge branch 'main' into 3583
lihaoyi Sep 30, 2024
bb1d96c
merge
lihaoyi Sep 30, 2024
fd9a276
package.mill
lihaoyi Sep 30, 2024
542f339
format
lihaoyi Sep 30, 2024
ba9d622
reorg
lihaoyi Sep 30, 2024
ef8c3a0
repro line prefix double newline issue
lihaoyi Sep 30, 2024
28d0611
fix for LinePrefixOutputStream double newline
lihaoyi Sep 30, 2024
e3a0b69
fmt
lihaoyi Sep 30, 2024
7849933
.
lihaoyi Sep 30, 2024
23e6030
wip
lihaoyi Sep 30, 2024
f1af93f
wip
lihaoyi Sep 30, 2024
15fb1c3
merrge
lihaoyi Sep 30, 2024
168d8da
.
lihaoyi Sep 30, 2024
900f388
.
lihaoyi Sep 30, 2024
53ec8a3
.
lihaoyi Sep 30, 2024
4a6ee54
.
lihaoyi Sep 30, 2024
8769d5f
.
lihaoyi Sep 30, 2024
b44f73c
.
lihaoyi Sep 30, 2024
2dc541f
fix -w infinite loop
lihaoyi Sep 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion main/init/package.mill
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import scala.util.matching.Regex
object `package` extends RootModule with build.MillPublishScalaModule {
def moduleDeps = Seq(build.main)

override def resources = T.sources {
override def resources = T {
super.resources() ++ Seq(exampleList())
}

Expand Down
83 changes: 44 additions & 39 deletions main/init/src/mill/init/InitModule.scala
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,8 @@ package mill.init
import mainargs.{Flag, arg}
import mill.api.IO
import mill.define.{Discover, ExternalModule}
import mill.util.Util.download
import mill.{Command, Module, T}

import java.io.IOException
import java.util.UUID
import scala.util.{Failure, Success, Try, Using}

object InitModule extends ExternalModule with InitModule {
Expand Down Expand Up @@ -36,52 +33,60 @@ trait InitModule extends Module {
): Command[Seq[String]] =
T.command {
usingExamples { examples =>
val result: Try[(Seq[String], String)] = exampleId match {
exampleId match {
case None =>
val exampleIds: Seq[ExampleId] = examples.map { case (exampleId, _) => exampleId }
def fullMessage(exampleIds: Seq[ExampleId]) =
msg + "\n\n" + exampleIds.mkString("\n") + "\n\n" + msg
if (showAll.value)
Success((exampleIds, fullMessage(exampleIds)))
if (showAll.value) (exampleIds, fullMessage(exampleIds))
else {
val toShow = List("basic", "builds", "web")
val filteredIds =
exampleIds.filter(_.split('/').lift.apply(1).exists(toShow.contains))
Success((filteredIds, fullMessage(filteredIds)))
val filteredIds = exampleIds
.filter(_.split('/').lift.apply(1).exists(toShow.contains))

(filteredIds, fullMessage(filteredIds))
}

case Some(value) =>
val result: Try[(Seq[String], String)] = for {
url <- examples.toMap.get(value).toRight(new Exception(
moduleNotExistMsg(value)
)).toTry
extractedDirName = {
val zipName = url.split('/').last
if (zipName.toLowerCase.endsWith(".zip")) zipName.dropRight(4) else zipName
}
downloadDir = T.workspace
downloadPath = downloadDir / extractedDirName
_ <- if (os.exists(downloadPath)) Failure(new IOException(
directoryExistsMsg(downloadPath.toString)
))
else Success(())
path <-
Try({
val tmpName = UUID.randomUUID().toString + ".zip"
val downloaded = download(url, os.rel / tmpName)(downloadDir)
val unpacked = IO.unpackZip(downloaded.path, os.rel)(downloadDir)
Try(os.remove(downloaded.path))
unpacked
}).recoverWith(ex =>
Failure(
new IOException(s"Couldn't download example: [$value];\n ${ex.getMessage}")
)
)
} yield (Seq(path.path.toString()), s"Example downloaded to [$downloadPath]")
val url = examples.toMap.get(value).getOrElse(sys.error(moduleNotExistMsg(value)))
println(s"Downloading example from $url...")
val zipName = url.split('/').last
val extractedDirName = zipName.stripSuffix(".zip")
val downloaded = os.temp(requests.get(url))
println(s"Unpacking example...")
val unpackPath = IO.unpackZip(downloaded, os.rel)
val extractedPath = T.dest / extractedDirName
val conflicting = for {
p <- os.walk(extractedPath)
val rel = p.relativeTo(extractedPath)
if os.exists(T.workspace / rel)
} yield rel

if (conflicting.nonEmpty) {
throw new Exception(
"Unable to unpack example because it conflicts with existing file: " +
conflicting.mkString(", ")
)
}

for (p <- os.walk(extractedPath)) {
println(p.relativeTo(extractedPath))
}

// Remove any existing bootstrap script since the example will come with one
os.remove(T.workspace / "mill")
os.copy.apply(extractedPath, T.workspace, mergeFolders = true)

// Make sure the `./mill` launcher is executable
os.perms.set(T.workspace / "mill", "rwxrwxrwx")

result
(
Seq(unpackPath.path.toString()),
s"Example download and unpacked to [${T.workspace}]; " +
"See `build.mill` for an explanation of this example and instructions on how to use it"
)
}
result
}.flatten match {
} match {
case Success((ret, msg)) =>
T.log.outputStream.println(msg)
ret
Expand Down
21 changes: 0 additions & 21 deletions main/init/test/src/mill/init/InitModuleTests.scala
Original file line number Diff line number Diff line change
Expand Up @@ -46,27 +46,6 @@ object InitModuleTests extends TestSuite {
assert(results.failing.keyCount == 1)
assert(errStream.toString.contains(initmodule.moduleNotExistMsg(nonExistingModuleId)))
}
test("mill init errors if directory already exist") {
type ExampleId = String
type ExampleUrl = String
val examplesList =
Using(initmodule.getClass.getClassLoader.getResourceAsStream("exampleList.txt")) {
examplesSource =>
val reader = upickle.default.reader[Seq[(ExampleId, ExampleUrl)]]
val examples: Seq[(ExampleId, ExampleUrl)] =
upickle.default.read(examplesSource)(reader)
examples
}.get
val exampleId = examplesList.head._1
val targetDir = examplesList.head._2.dropRight(4).split("/").last // dropping .zip

val targetPath = initmodule.millSourcePath / targetDir
os.makeDir(targetPath)
val results = evaluator.evaluator.evaluate(Agg(initmodule.init(Some(exampleId))))
assert(results.failing.keyCount == 1)
assert(errStream.toString.contains(initmodule.directoryExistsMsg(targetPath.toString())))

}
}
}
}
Loading