Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump up ZK version to 3.6.3 #9612

Merged
merged 1 commit into from
Oct 19, 2022
Merged

Bump up ZK version to 3.6.3 #9612

merged 1 commit into from
Oct 19, 2022

Conversation

jackjlli
Copy link
Member

@jackjlli jackjlli commented Oct 17, 2022

In LinkedIn the ZK version has already been bumped up to 3.6.3 and it's working as expected.
This PR bumps up ZK version to 3.6.3 in the Apache Pinot repo.

Issue: #9431

@codecov-commenter
Copy link

codecov-commenter commented Oct 17, 2022

Codecov Report

Merging #9612 (ccfdba0) into master (edf0c01) will increase coverage by 33.00%.
The diff coverage is n/a.

@@              Coverage Diff              @@
##             master    #9612       +/-   ##
=============================================
+ Coverage     37.00%   70.00%   +33.00%     
- Complexity      198     5325     +5127     
=============================================
  Files          1941     1939        -2     
  Lines        103572   103671       +99     
  Branches      15703    15727       +24     
=============================================
+ Hits          38331    72580    +34249     
+ Misses        62048    25992    -36056     
- Partials       3193     5099     +1906     
Flag Coverage Δ
integration1 25.85% <ø> (-0.01%) ⬇️
integration2 24.56% <ø> (-0.01%) ⬇️
unittests1 67.35% <ø> (?)
unittests2 15.60% <ø> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ache/pinot/core/operator/docidsets/OrDocIdSet.java 86.36% <0.00%> (-13.64%) ⬇️
...nction/DistinctCountBitmapAggregationFunction.java 47.15% <0.00%> (-8.30%) ⬇️
...query/runtime/operator/MailboxReceiveOperator.java 76.66% <0.00%> (-2.65%) ⬇️
...ces/PinotSegmentUploadDownloadRestletResource.java 53.94% <0.00%> (-0.83%) ⬇️
...che/pinot/broker/routing/BrokerRoutingManager.java 85.51% <0.00%> (-0.56%) ⬇️
...ntroller/helix/core/PinotHelixResourceManager.java 71.79% <0.00%> (-0.24%) ⬇️
...pache/pinot/core/query/utils/idset/EmptyIdSet.java 25.00% <0.00%> (ø)
...anager/realtime/SegmentBuildTimeLeaseExtender.java 63.23% <0.00%> (ø)
...ocessing/transformer/RecordTransformerFactory.java
...segment/processing/filter/RecordFilterFactory.java
... and 1017 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@siddharthteotia
Copy link
Contributor

FYI @walterddr @Jackie-Jiang

@@ -107,6 +107,10 @@
<groupId>net.sf.jopt-simple</groupId>
<artifactId>jopt-simple</artifactId>
</dependency>
<dependency>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this pom change related to zk version upgrade?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately yes, the metric-core module is required. Otherwise, the following exception would be thrown:

Error:  org.apache.pinot.plugin.stream.kafka20.KafkaPartitionLevelConsumerBackwardCompatibilityTest.setUp  Time elapsed: 0.906 s  <<< FAILURE!
java.lang.NoClassDefFoundError: com/codahale/metrics/Reservoir
	at org.apache.zookeeper.metrics.impl.DefaultMetricsProvider$DefaultMetricsContext.lambda$getSummary$2(DefaultMetricsProvider.java:126)
	at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705)
	at org.apache.zookeeper.metrics.impl.DefaultMetricsProvider$DefaultMetricsContext.getSummary(DefaultMetricsProvider.java:122)
	at org.apache.zookeeper.server.ServerMetrics.<init>(ServerMetrics.java:74)
	at org.apache.zookeeper.server.ServerMetrics.<clinit>(ServerMetrics.java:44)
	at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:246)
	at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:285)
	at org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:494)
	at org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:665)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:758)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:130)
	at org.apache.pinot.plugin.stream.kafka20.utils.EmbeddedZooKeeper.<init>(EmbeddedZooKeeper.java:42)
	at org.apache.pinot.plugin.stream.kafka20.utils.MiniKafkaCluster.<init>(MiniKafkaCluster.java:54)
	at org.apache.pinot.plugin.stream.kafka20.KafkaPartitionLevelConsumerTest.setUp(KafkaPartitionLevelConsumerTest.java:63)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:108)
	at org.testng.internal.Invoker.invokeConfigurationMethod(Invoker.java:523)
	at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:224)
	at org.testng.internal.Invoker.invokeConfigurations(Invoker.java:146)
	at org.testng.internal.TestMethodWorker.invokeBeforeClassMethods(TestMethodWorker.java:166)
	at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:105)
	at org.testng.TestRunner.privateRun(TestRunner.java:744)
	at org.testng.TestRunner.run(TestRunner.java:602)
	at org.testng.SuiteRunner.runTest(SuiteRunner.java:380)
	at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:375)
	at org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
	at org.testng.SuiteRunner.run(SuiteRunner.java:289)
	at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
	at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
	at org.testng.TestNG.runSuitesSequentially(TestNG.java:1301)
	at org.testng.TestNG.runSuitesLocally(TestNG.java:1226)
	at org.testng.TestNG.runSuites(TestNG.java:1144)
	at org.testng.TestNG.run(TestNG.java:1115)
	at org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:136)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:112)
	at org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:99)
	at org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:145)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
	at org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
Caused by: java.lang.ClassNotFoundException: com.codahale.metrics.Reservoir
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 44 more

I can try restrict the scope of that dependency to test only, as I can build it locally when running tests is disabled.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing it test scope works. It seems ServerCnxnFactory is only used in test classes, so we should be good with the current changes in this PR.

@siddharthteotia siddharthteotia merged commit ff5f939 into master Oct 19, 2022
@Jackie-Jiang Jackie-Jiang deleted the bump-up-zk-version branch October 19, 2022 23:16
@Jackie-Jiang Jackie-Jiang added the release-notes Referenced by PRs that need attention when compiling the next release notes label Oct 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-notes Referenced by PRs that need attention when compiling the next release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants