Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase build performance w/ Gradle upgrade to 7.4 #539

Merged
merged 2 commits into from
Mar 2, 2022

Conversation

am357
Copy link
Contributor

@am357 am357 commented Mar 2, 2022

Issue #, if available: 538

Description of changes:
See the attached issue for motivation.

This commit contains the following changes:

  1. Required settings for parallelizing test executions for each sub-project.
  2. Changes for removing blocking deprecated Gradle settings for upgrading Gradle to 7.4
  3. Gradle 7.4 upgrade:
    In addition to other fixes, to be able to use features like File System Watching by default:
    https://docs.gradle.org/current/userguide/performance.html#file_system_watching
  4. Enable parallel Gradle build and build cache:
    https://docs.gradle.org/current/userguide/build_cache.html
  5. Plugin upgrade:
    As part of upgrade removing the dependency to jcenter repository.

With these changes, apart from parallelizing builds and test executions, our major gain is in builds after the first build. This is because as appose to the current situation, the subsequent build times are dependent to the amount of incremental changes we make between each build. For example in the following, the second build is after making no changes to the first build:

➜  partiql-lang-kotlin git:(improve-build-performance) ./gradlew build

> Task :lang:generatePigDomains
pig: universe file: /Users/maymandi/workplace/maymandi/partiql-lang-kotlin/lang/resources/org/partiql/type-domains/partiql.ion
pig: output file  : /Users/maymandi/workplace/maymandi/partiql-lang-kotlin/lang/src/org/partiql/lang/domains/partiql-domains.kt
pig: parsing the universe...
pig: permuting domains...
pig: applying Kotlin pre-processing
pig: applying the Kotlin template...
pig: universe generation complete!

> Task :lang:compileTestKotlin
...
Daemon will be stopped at the end of the build after running out of JVM memory
...
BUILD SUCCESSFUL in 2m 38s
49 actionable tasks: 27 executed, 22 up-to-date

➜  partiql-lang-kotlin git:(improve-build-performance) ./gradlew build
...

BUILD SUCCESSFUL in 10s

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

This commit contains the following changes:
1. Required settings for parallelizing test executions for each subproject
2. Settings for removing deprecated gradle settings for upgrading gradle to 7.4
3. Gradle 7.4 upgrade
4. Enable parallel gradle build
5. Upgrading the plugins
   As part of upgrade removing the dependency to jcenter repository
@@ -1,7 +1,7 @@
#!/usr/bin/env sh
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes in this file are auto-generated by Gradle 7.4 upgrade.

@am357 am357 marked this pull request as ready for review March 2, 2022 18:38
Copy link
Contributor

@jpschorr jpschorr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Several build.gradle files have:

        xml.required.set(false)
        html.required.set(true)
        xml.required.set(false)

Where the two xml... lines seem to be duplicates.

cli/build.gradle Outdated Show resolved Hide resolved
examples/build.gradle Outdated Show resolved Hide resolved
lang/build.gradle Outdated Show resolved Hide resolved
testscript/build.gradle Outdated Show resolved Hide resolved
pts/build.gradle Outdated Show resolved Hide resolved
Copy link
Member

@alancai98 alancai98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice improvement! I was getting ~5 min for each local build previously. Now I'm getting around ~3.5 min.

Have the same comment Josh had provided in his review. Otherwise looks good

cli/build.gradle Outdated Show resolved Hide resolved
Copy link
Member

@alancai98 alancai98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@am357 am357 merged commit 4684e9e into main Mar 2, 2022
@am357 am357 deleted the improve-build-performance branch March 2, 2022 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants