Update Gradle to 5.5 version #4567

chetanmeh · 2019-07-24T10:30:16Z

Updates Gradle to 5.5 version and also updates Gradle Scoverage plugin to 3.1.5

Description

For enabling running and building on JDK 11 (#4217) we need to move to Gradle 5.x version which officially supports JDK 11. In doing so we would need to update the Gradle Scoverage plugin to 3.1.5 as existing version was not compatible

Benefits

Enables building on JDK 11
Gradle 5.x has better support for Maven publishing. We would now be needing that to publish the standalone jar

Scoverage Version Impact

Newer soverage plugin has a different model of working and does not produce a jar of instrumented classes which impacts our logic for coverage collection for code running within container (#3685). To make it work we now copy the instrumented classes within container and remove existing packaged jars from the lib

Related issue and scope

I opened an issue to propose and discuss this change (Update to Java 11 #4217 )

My changes affect the following components

Types of changes

Bug fix (generally a non-breaking change which closes an issue).
Enhancement or new feature (adds new functionality).
Breaking change (a bug fix or enhancement which changes existing behavior).

Checklist:

I signed an Apache CLA.
I reviewed the style guides and followed the recommendations (Travis CI will check :).
I added tests to cover my changes.
My changes require further changes to the documentation.
I updated the documentation where necessary.

scoverage now does not create scoverage specific jars. So now we copy the instrumented classes into the docker container and remove the existing jars

codecov-io · 2019-08-06T07:40:03Z

Codecov Report

Merging #4567 into master will decrease coverage by 5.75%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #4567      +/-   ##
==========================================
- Coverage   84.52%   78.76%   -5.76%     
==========================================
  Files         183      183              
  Lines        8252     8252              
  Branches      558      558              
==========================================
- Hits         6975     6500     -475     
- Misses       1277     1752     +475

Impacted Files	Coverage Δ
...core/database/cosmosdb/RxObservableImplicits.scala	`0% <0%> (-100%)`	⬇️
...ore/database/cosmosdb/cache/CacheInvalidator.scala	`0% <0%> (-100%)`	⬇️
...core/database/cosmosdb/CosmosDBArtifactStore.scala	`0% <0%> (-95.89%)`	⬇️
...tabase/cosmosdb/cache/CacheInvalidatorConfig.scala	`0% <0%> (-94.74%)`	⬇️
...sk/core/database/cosmosdb/CosmosDBViewMapper.scala	`0% <0%> (-92.6%)`	⬇️
...e/database/cosmosdb/cache/ChangeFeedListener.scala	`0% <0%> (-86.67%)`	⬇️
...e/database/cosmosdb/cache/KafkaEventProducer.scala	`0% <0%> (-76.48%)`	⬇️
...whisk/core/database/cosmosdb/CosmosDBSupport.scala	`0% <0%> (-74.08%)`	⬇️
...a/org/apache/openwhisk/common/ExecutorCloser.scala	`0% <0%> (-66.67%)`	⬇️
...abase/cosmosdb/CosmosDBArtifactStoreProvider.scala	`4% <0%> (-52%)`	⬇️
... and 7 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ce45d54...f663b0e. Read the comment docs.

chetanmeh · 2019-08-06T08:49:59Z

tests/build.gradle

+     * Task to generate coverage xml report. Requires the
+     * tests to be executed prior to its invocation
+     */
+    task reportCoverage(type: ScoverageReport) {


The new version now creates task like report<Test Task Name>Scoverage as entry task. This task in turn adapts the compile task to use the instrumented classes. This works fine for conventional cases where tests are part of same module.

However in our case as we have a separate test module and further to capture coverage of code running within container (as part of integration test) we copy the coverage metrics files manually we cannot use default tasks.

Hence the report task duplicate bit of logic from actual task and clones the config from generated reportTestCoverage task

chetanmeh · 2019-08-06T08:53:32Z

tests/build.gradle

@@ -250,8 +255,33 @@ gradle.projectsEvaluated {
            showStandardStreams = true
            exceptionFormat = 'full'
        }
+        maxHeapSize = "1024m" //Gradle 5.5 defaults to 512MB which is low


With Gradle 5.0 it now defaults to 512M of heap. Due to this OOM was seen in test runs on Travis (which has 7 GB memory). Hence specifying the size explicitly to 1024 MB now

All workers, including compilers and test executors, now start with 512MB of heap. The previous default was 1/4th of physical memory. Large projects may have to increase this setting on the relevant tasks, e.g. JavaCompile or Test.

chetanmeh · 2019-08-06T08:54:28Z

gradle/wrapper/gradle-wrapper.properties

@@ -17,6 +17,6 @@

 distributionBase=GRADLE_USER_HOME
 distributionPath=wrapper/dists
-distributionUrl=https\://services.gradle.org/distributions/gradle-4.10.2-bin.zip
+distributionUrl=https\://services.gradle.org/distributions/gradle-5.5.1-all.zip


Switching to 5.5.1 and also to all mode so as to enable code assists in IDE

chetanmeh · 2019-08-06T08:55:37Z

settings.gradle

@@ -43,13 +43,6 @@ gradle.ext.scalafmt = [
    config: new File(rootProject.projectDir, '.scalafmt.conf')
 ]

-gradle.ext.scoverage = [


Now scoverage plugin internally adapts the classpath with required dependencies. So no need to explicitly add them

chetanmeh · 2019-08-06T08:56:45Z

tests/build.gradle

- * Aggregates the scoverage xml reports from various modules into a
- * single report
- */
-task aggregateCoverage(type: JavaExec, dependsOn: reportCoverage) {


This task was never used as we rely on codecov to perform actual aggregation

chetanmeh · 2019-08-06T11:18:51Z

This PR is now ready for review

chetanmeh · 2019-08-09T04:27:54Z

@dgrove-oss @rabbah Can you review this PR. Once merged we can then work on publishing Maven artifacts from core repo (#4579).

Not sure if any PG run is needed as it would impact the build process

rabbah · 2019-08-09T15:37:08Z

gradlew.bat

@@ -30,7 +30,7 @@ set APP_BASE_NAME=%~n0
 set APP_HOME=%DIRNAME%

 @rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
-set DEFAULT_JVM_OPTS=-Dfile.encoding=UTF-8
+set DEFAULT_JVM_OPTS=-Dfile.encoding=UTF-8 "-Xmx64m" "-Xms64m"


is this too low?

Yeah but thats the current default in there generated files

If a different heap size is appropriate, we can change it. It's fine for us to diverge from the upstream gradlew[.bat] as needed.

For now build seems to work fine. If we see any issue then we can change. I believe Gradle wrapper internally forks a new jvm process so it needs bare minimum memory for its work

rabbah · 2019-08-09T15:58:31Z

No issues on my side using this patch.

dgrove-oss

Looks fine to me. Only comment was on heapsize argument in default JVM args in gradlew; feel free to deviate from upstream version to set a default appropriate to this project if you want.

Updates Gradle to 5.5 to enable work with JDK 11. Along with that it also upgrades the scoverage plugin as the older version does not work with Gradle 5. Due to various changes in how scoverage works we needed to adapt the way how instrumented classes are included in test runs

chetanmeh and others added 11 commits August 2, 2019 16:09

Move to scoverage 3.1.5 and adapt existing config

e1e5965

Remove aggregate report task

5f3d842

Fix version

5d83ee1

Remove scoverage entries

fecf2df

Add repo entry for all projects as scoverage plugins get applied to all

73bfd96

pass project list as method param

0649a7c

Update to Gradle 5.5

5adcdb0

Add license header

f058465

Work with instrumented classes instead of instrumented jars

e12d981

scoverage now does not create scoverage specific jars. So now we copy the instrumented classes into the docker container and remove the existing jars

Trim whitespace

f9ae903

Copy the instrumented classes to classes dir

0b7944d

chetanmeh force-pushed the gradle-update-5x branch from 8992b77 to 0b7944d Compare August 2, 2019 10:43

chetanmeh added 4 commits August 2, 2019 16:38

Adapt for scoverage changes

57bfa51

Fix module name

9b99b26

Increase memory size for test

e4c30e7

Adapt report coverage tasks

bb7b5b8

chetanmeh added 3 commits August 6, 2019 14:07

Change maxHeap to 1024

70601a6

Update to latest 5.5.1 release

4abb5b3

Match to conventional license header using http url

832c3d3

chetanmeh commented Aug 6, 2019

View reviewed changes

chetanmeh marked this pull request as ready for review August 6, 2019 08:57

chetanmeh added 3 commits August 6, 2019 15:04

Enable encoding again

0056977

Remove unneeded config change

972d6d7

Disable logs from Kafka Server used in cache invalidator tests

f663b0e

chetanmeh mentioned this pull request Aug 9, 2019

Publish OpenWhisk artifacts to Maven #4579

Open

chetanmeh requested a review from dgrove-oss August 9, 2019 04:28

rabbah approved these changes Aug 9, 2019

View reviewed changes

dgrove-oss approved these changes Aug 12, 2019

View reviewed changes

chetanmeh merged commit d9de97d into apache:master Aug 12, 2019

chetanmeh deleted the gradle-update-5x branch August 12, 2019 16:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update Gradle to 5.5 version #4567

Update Gradle to 5.5 version #4567

chetanmeh commented Jul 24, 2019

codecov-io commented Aug 6, 2019 •

edited

Loading

chetanmeh Aug 6, 2019 •

edited

Loading

chetanmeh Aug 6, 2019

chetanmeh Aug 6, 2019

chetanmeh Aug 6, 2019

chetanmeh Aug 6, 2019

chetanmeh commented Aug 6, 2019

chetanmeh commented Aug 9, 2019 •

edited

Loading

rabbah Aug 9, 2019

chetanmeh Aug 12, 2019

dgrove-oss Aug 12, 2019

chetanmeh Aug 12, 2019

rabbah commented Aug 9, 2019

dgrove-oss left a comment

Update Gradle to 5.5 version #4567

Update Gradle to 5.5 version #4567

Conversation

chetanmeh commented Jul 24, 2019

Description

Benefits

Scoverage Version Impact

Related issue and scope

My changes affect the following components

Types of changes

Checklist:

codecov-io commented Aug 6, 2019 • edited Loading

Codecov Report

chetanmeh Aug 6, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chetanmeh commented Aug 6, 2019

chetanmeh commented Aug 9, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rabbah commented Aug 9, 2019

dgrove-oss left a comment

Choose a reason for hiding this comment

codecov-io commented Aug 6, 2019 •

edited

Loading

chetanmeh Aug 6, 2019 •

edited

Loading

chetanmeh commented Aug 9, 2019 •

edited

Loading