Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve SBT caching during code generation #1499

Merged
merged 13 commits into from
May 18, 2024
Merged

Improve SBT caching during code generation #1499

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
70ebc75
Improve cache in code generation
majk-p Apr 22, 2024
4fcd2dd
introduce PathRef for hashed directories
majk-p Apr 23, 2024
bd719c1
restore cache directory to factory generated
majk-p Apr 24, 2024
cd141bd
restore public api for CodegenArgs
majk-p Apr 24, 2024
73d938f
add missing headers
majk-p Apr 25, 2024
df5f6d1
adjust paths in scripted tests
majk-p Apr 26, 2024
5bfb52f
un-sort transforms
majk-p Apr 26, 2024
99254fe
Don't nest resources
kubukoz Apr 26, 2024
75588b1
Merge branch 'series/0.18' into codegen-cache-improvements
kubukoz May 17, 2024
3deb068
Fix new test
kubukoz May 17, 2024
70709ae
Deleting output files won't trigger regenerating them
kubukoz May 17, 2024
32bd0a2
Add CHANGELOG entry
kubukoz May 17, 2024
acfd9dd
Move PathRef to sbt
kubukoz May 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 9 additions & 5 deletions modules/codegen-plugin/src/smithy4s/codegen/JsonConverters.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,13 +61,17 @@ private[smithy4s] object JsonConverters {
)

// format: off
type GenTarget = List[os.Path] :*: os.Path :*: os.Path :*: Set[FileType] :*: Boolean:*: Option[Set[String]] :*: Option[Set[String]] :*: List[String] :*: List[String] :*: List[String] :*: List[os.Path] :*: LNil
type GenTarget = List[os.Path] :*: String :*: String :*: Set[FileType] :*: Boolean:*: Option[Set[String]] :*: Option[Set[String]] :*: List[String] :*: List[String] :*: List[String] :*: List[os.Path] :*: LNil
kubukoz marked this conversation as resolved.
Show resolved Hide resolved
// format: on

// `output` and `resourceOutput` are intentionally serialized as strings
// instead paths. This is to avoid hashing the directories that are generated by smithy4s anyway
// See https://github.com/disneystreaming/smithy4s/issues/1495 for reference on this decision
implicit val codegenArgsIso = LList.iso[CodegenArgs, GenTarget](
{ ca: CodegenArgs =>
("specs", ca.specs) :*:
("output", ca.output) :*:
("resourceOutput", ca.resourceOutput) :*:
("output", ca.output.toString) :*:
("resourceOutput", ca.resourceOutput.toString) :*:
("skip", ca.skip) :*:
("discoverModels", ca.discoverModels) :*:
("allowedNS", ca.allowedNS) :*:
Expand All @@ -92,8 +96,8 @@ private[smithy4s] object JsonConverters {
(_, localJars) :*: LNil =>
CodegenArgs(
specs,
output,
resourceOutput,
os.Path(output),
os.Path(resourceOutput),
skip,
discoverModels,
allowedNS,
Expand Down
14 changes: 8 additions & 6 deletions modules/codegen-plugin/src/smithy4s/codegen/Smithy4sCodegenPlugin.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -403,8 +403,8 @@ object Smithy4sCodegenPlugin extends AutoPlugin {
(inputDirs ++ generatedFiles)
.filter(_.exists())
.toList
kubukoz marked this conversation as resolved.
Show resolved Hide resolved
val outputPath = (conf / smithy4sOutputDir).value
val resourceOutputPath = (conf / smithy4sResourceDir).value
val outputPath = (conf / smithy4sOutputDir).value / "smithy4s"
val resourceOutputPath = (conf / smithy4sResourceDir).value / "smithy4s"
kubukoz marked this conversation as resolved.
Show resolved Hide resolved
val allowedNamespaces =
(conf / smithy4sAllowedNamespaces).?.value.map(_.toSet)
val excludedNamespaces =
Expand All @@ -424,7 +424,7 @@ object Smithy4sCodegenPlugin extends AutoPlugin {

val filePaths = inputFiles.map(_.getAbsolutePath())
val codegenArgs = CodegenArgs(
filePaths.map(os.Path(_)).toList,
filePaths.sorted.map(os.Path(_)).toList,
output = os.Path(outputPath),
resourceOutput = os.Path(resourceOutputPath),
skip = skipSet,
Expand All @@ -433,8 +433,8 @@ object Smithy4sCodegenPlugin extends AutoPlugin {
excludedNS = excludedNamespaces,
repositories = res,
dependencies = List.empty,
transformers = transforms,
localJars = localJars
transformers = transforms.sorted,
localJars = localJars.sorted
)

val cached =
Expand All @@ -443,14 +443,16 @@ object Smithy4sCodegenPlugin extends AutoPlugin {
) {
Function.untupled {
Tracked.lastOutput[(Boolean, CodegenArgs), Seq[File]](
s.cacheStoreFactory.make("output")
s.cacheDirectory / "smithy4s-output"
) { case ((inputChanged, args), outputs) =>
if (inputChanged || outputs.isEmpty) {
s.log.debug("Regenerating managed sources")
kubukoz marked this conversation as resolved.
Show resolved Hide resolved
val resPaths = smithy4s.codegen.Codegen
.generateToDisk(args)
.toList
resPaths.map(path => new File(path.toString))
} else {
s.log.debug("Using cached version of outputs")
outputs.getOrElse(Seq.empty)
}
}
Expand Down
Loading