[Release 3.0] Bump gradle 8.10.2 / JDK23 / 3.0.0.0-alpha1 on SQL plugin #3319

peterzhuamazon · 2025-02-12T21:03:34Z

Description

[Release 3.0] Bump gradle 8.10.2 / JDK23 / 3.0.0.0-alpha1 on SQL plugin

Change the normal import Client to transport.Client.
Change the master term to clustermanager.
Update to new com.gradleup.shadow id for shadow plugin.

Related Issues

Closes #3315
#1880

Check List

~~[ ] New functionality includes testing.~~
~~[ ] New functionality has been documented.~~
~~[ ] New functionality has javadoc added.~~
~~[ ] New functionality has a user manual doc added.~~
~~[ ] API changes companion pull request created.~~
Commits are signed per the DCO using --signoff.
~~[ ] Public documentation issue/PR created.~~

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Peter Zhu <[email protected]>

peterzhuamazon · 2025-02-12T21:03:52Z

Pending geospatial:

[Release 3.0] Bump gradle 8.10.2 / JDK 23 / 3.0.0.0-alpha1 support on geospatial geospatial#723


* What went wrong:
Could not determine the dependencies of task ':integ-test:integTest'.
> Could not resolve all files for configuration ':integ-test:zipArchive'.
   > Could not resolve org.opensearch.plugin:geospatial:3.0.0.0-alpha1-SNAPSHOT.
     Required by:

peterzhuamazon · 2025-02-12T22:09:46Z

Now pending ml:

Fix test failure after lucene upgrade to 10 ml-commons#3426


* What went wrong:
Execution failed for task ':opensearch:compileJava'.
> Could not resolve all files for configuration ':opensearch:compileClasspath'.
   > Could not resolve org.opensearch:opensearch-ml-client:3.0.0.0-alpha1-SNAPSHOT.
     Required by:
         project :opensearch

Signed-off-by: Peter Zhu <[email protected]>

peterzhuamazon · 2025-02-13T17:49:31Z

Seems issues with jdk23 not very sure:



        Caused by:
        java.lang.IllegalStateException: Failed to load interface org.mockito.plugins.MockMaker implementation declared in java.lang.CompoundEnumeration@216c22ce
            at org.mockito.internal.configuration.plugins.PluginInitializer.loadImpl(PluginInitializer.java:56)
            at org.mockito.internal.configuration.plugins.PluginLoader.loadPlugin(PluginLoader.java:65)
            at org.mockito.internal.configuration.plugins.PluginLoader.loadPlugin(PluginLoader.java:49)
            at org.mockito.internal.configuration.plugins.PluginRegistry.<init>(PluginRegistry.java:29)
            at org.mockito.internal.configuration.plugins.Plugins.<clinit>(Plugins.java:26)
            at org.mockito.internal.MockitoCore.<clinit>(MockitoCore.java:71)
            at org.mockito.Mockito.<clinit>(Mockito.java:1741)
            at org.mockito.junit.jupiter.MockitoExtension.beforeEach(MockitoExtension.java:155)
            ... 2 more

            Caused by:
            java.lang.reflect.InvocationTargetException
                at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:501)
                at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:485)
                at org.mockito.internal.configuration.plugins.PluginInitializer.loadImpl(PluginInitializer.java:51)
                ... 9 more

                Caused by:
                org.mockito.exceptions.base.MockitoInitializationException:
                Could not initialize inline Byte Buddy mock maker.

                It appears as if your JDK does not supply a working agent attachment mechanism.
                Java               : 23
                JVM vendor name    : Eclipse Adoptium
                JVM vendor version : 23.0.2+7
                JVM name           : OpenJDK 64-Bit Server VM
                JVM version        : 23.0.2+7
                JVM info           : mixed mode, sharing
                OS name            : Linux
                OS version         : 5.10.233-204.894.amzn2int.x86_64
                    at app//org.mockito.internal.creation.bytebuddy.InlineDelegateByteBuddyMockMaker.<init>(InlineDelegateByteBuddyMockMaker.java:254)
                    at app//org.mockito.internal.creation.bytebuddy.InlineByteBuddyMockMaker.<init>(InlineByteBuddyMockMaker.java:23)
                    ... 12 more

                    Caused by:
                    java.lang.IllegalStateException: Failed to resolve the class file version of the current VM: Unknown Java version: 0
                        at net.bytebuddy.ClassFileVersion$VersionLocator$Unresolved.resolve(ClassFileVersion.java:667)
                        at net.bytebuddy.ClassFileVersion.ofThisVm(ClassFileVersion.java:349)
                        at org.mockito.internal.PremainAttachAccess.getInstrumentation(PremainAttachAccess.java:36)
                        at org.mockito.internal.creation.bytebuddy.InlineDelegateByteBuddyMockMaker.<clinit>(InlineDelegateByteBuddyMockMaker.java:138)
                        ... 13 more

peterzhuamazon · 2025-02-13T18:27:51Z

See it here:

                java.lang.IllegalArgumentException: Java 23 (67) is not supported by the current version of Byte Buddy which officially supports Java 22 (66) - update Byte Buddy or set net.bytebuddy.experimental as a VM property

Will update 1.14.19 (1.14.16 supports JDK23)

sql-3.0.0.0-alpha1-jdk23-error-log.txt

Signed-off-by: Peter Zhu <[email protected]>

peterzhuamazon · 2025-02-13T19:01:39Z

Now JDK23 has the same errors on integTest as JDK21:
sql-3.0.0.0-alpha1-jdk23-error-log2.txt

peterzhuamazon · 2025-02-13T19:25:40Z

Adding @noCharger to take a look at integTest as he is the release manager for SQL.

Thanks!

noCharger

One major isssue from the log:

»  java.lang.IllegalArgumentException: No enum constant org.opensearch.action.search.SearchPhaseName.CREATE_PIT

> Task :integ-test:integTest FAILED

This is tracking opensearch-project/OpenSearch#16750
Bugfix opensearch-project/OpenSearch#16781

noCharger · 2025-02-13T20:41:38Z

Some other IT failed, which might be transient:

REPRODUCE WITH: gradlew ':integ-test:integTest' --tests "org.opensearch.sql.legacy.AggregationIT.termsWithMissing" -Dtests.seed=FF8898E23121F455 -Dtests.security.manager=false -Dtests.locale=os -Dtests.timezone=CET -Druntime.java=21

AggregationIT > termsWithMissing FAILED
    java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([FF8898E23121F455:CAA7D9E22B98F403]:0)
        at org.junit.Assert.fail(Assert.java:87)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.junit.Assert.assertTrue(Assert.java:53)
        at org.opensearch.sql.legacy.AggregationIT.termsWithMissing(AggregationIT.java:449)

Signed-off-by: Peter Zhu <[email protected]>

peterzhuamazon · 2025-02-14T21:29:00Z

Seems like same issues as this one:

[3.0] Update neural-search for OpenSearch 3.0 compatibility neural-search#1141 (comment)

peterzhuamazon · 2025-02-15T00:00:04Z

New Failures on the build test, not integTest anymore after updating security snapshots:


2025-02-14T23:24:14.7130535Z > Task :doctest:doctest FAILED
2025-02-14T23:24:14.7131137Z ======================================================================
2025-02-14T23:24:14.7131683Z FAIL: /__w/sql/sql/doctest/../docs/user/dql/functions.rst
2025-02-14T23:24:14.7132155Z Doctest: functions.rst
2025-02-14T23:24:14.7132566Z ----------------------------------------------------------------------
2025-02-14T23:24:14.7133076Z Traceback (most recent call last):
2025-02-14T23:24:14.7133808Z   File "/usr/local/lib/python3.9/doctest.py", line 2204, in runTest
2025-02-14T23:24:14.7134247Z     raise self.failureException(self.format_failure(new.getvalue()))
2025-02-14T23:24:14.7134627Z AssertionError: Failed doctest test for functions.rst
2025-02-14T23:24:14.7135377Z   File "/__w/sql/sql/doctest/../docs/user/dql/functions.rst", line 0
2025-02-14T23:24:14.7135803Z 
2025-02-14T23:24:14.7136011Z ----------------------------------------------------------------------
2025-02-14T23:24:14.7136977Z File "/__w/sql/sql/doctest/../docs/user/dql/functions.rst", line 4328, in functions.rst
2025-02-14T23:24:14.7137355Z Failed example:
2025-02-14T23:24:14.7137826Z     sql_cmd.process("select id, title, author, _score from books where score(query('title:Pooh House', default_operator='AND'), 2.0)")Expected:
2025-02-14T23:24:14.7138341Z     fetched rows / total rows = 1/1
2025-02-14T23:24:14.7138618Z     +----+--------------------------+----------------------+-----------+
2025-02-14T23:24:14.7138942Z     | id | title                    | author               | _score    |
2025-02-14T23:24:14.7139677Z     |----+--------------------------+----------------------+-----------|
2025-02-14T23:24:14.7143292Z     | 1  | The House at Pooh Corner | Alan Alexander Milne | 1.5884793 |
2025-02-14T23:24:14.7144141Z     +----+--------------------------+----------------------+-----------+
2025-02-14T23:24:14.7144644Z Got:
2025-02-14T23:24:14.7144929Z     fetched rows / total rows = 1/1
2025-02-14T23:24:14.7145415Z     +----+--------------------------+----------------------+----------+
2025-02-14T23:24:14.7145949Z     | id | title                    | author               | _score   |
2025-02-14T23:24:14.7146469Z     |----+--------------------------+----------------------+----------|
2025-02-14T23:24:14.7147028Z     | 1  | The House at Pooh Corner | Alan Alexander Milne | 0.722036 |
2025-02-14T23:24:14.7147574Z     +----+--------------------------+----------------------+----------+
2025-02-14T23:24:14.7148102Z ----------------------------------------------------------------------
2025-02-14T23:24:14.7148810Z File "/__w/sql/sql/doctest/../docs/user/dql/functions.rst", line 4336, in functions.rst
2025-02-14T23:24:14.7149431Z Failed example:
2025-02-14T23:24:14.7150471Z     sql_cmd.process("select id, title, author, _score from books where score(query('title:Pooh House', default_operator='AND'), 5.0) OR score(query('title:Winnie', default_operator='AND'), 1.5)")Expected:
2025-02-14T23:24:14.7151608Z     fetched rows / total rows = 2/2
2025-02-14T23:24:14.7152057Z     +----+--------------------------+----------------------+-----------+
2025-02-14T23:24:14.7152577Z     | id | title                    | author               | _score    |
2025-02-14T23:24:14.7153091Z     |----+--------------------------+----------------------+-----------|
2025-02-14T23:24:14.7153825Z     | 1  | The House at Pooh Corner | Alan Alexander Milne | 3.9711983 |
2025-02-14T23:24:14.7154427Z     | 2  | Winnie-the-Pooh          | Alan Alexander Milne | 1.1581701 |
2025-02-14T23:24:14.7155942Z     +----+--------------------------+----------------------+-----------+
2025-02-14T23:24:14.7156427Z Got:
2025-02-14T23:24:14.7156714Z     fetched rows / total rows = 2/2
2025-02-14T23:24:14.7157171Z     +----+--------------------------+----------------------+------------+
2025-02-14T23:24:14.7157716Z     | id | title                    | author               | _score     |
2025-02-14T23:24:14.7158270Z     |----+--------------------------+----------------------+------------|
2025-02-14T23:24:14.7158913Z     | 1  | The House at Pooh Corner | Alan Alexander Milne | 1.8050902  |
2025-02-14T23:24:14.7159554Z     | 2  | Winnie-the-Pooh          | Alan Alexander Milne | 0.52644086 |
2025-02-14T23:24:14.7160175Z     +----+--------------------------+----------------------+------------+
2025-02-14T23:24:14.7160563Z 
2025-02-14T23:24:14.7160571Z 
2025-02-14T23:24:14.7160798Z ----------------------------------------------------------------------

cc: @noCharger

Thanks!

noCharger · 2025-02-18T19:44:35Z

sql_cmd.process("select id, title, author, _score from books where score(query('title:Pooh House', default_operator='AND'), 2.0)")

This score discrepancy was replicated on my local with this PR. opensearch-project/OpenSearch#17306 is the likely reason, if the non-alpha1 snapshot does not contain this PR. Let's make the necessary updates to the functions.rst document. Also ping @msfroh to confirm if there are any adverse effects / regression could be introduced by this score function change.

peterzhuamazon · 2025-02-18T19:53:12Z

sql_cmd.process("select id, title, author, _score from books where score(query('title:Pooh House', default_operator='AND'), 2.0)")

This score discrepancy was replicated on my local with this PR. opensearch-project/OpenSearch#17306 is the likely reason, if the non-alpha1 snapshot does not contain this PR. Let's make the necessary updates to the functions.rst document. Also ping @msfroh to confirm if there are any adverse effects / regression could be introduced by this score function change.

The alpha1 version will eventually become non-alpha1 during GA, so I think we can start with the update.

Signed-off-by: Peter Zhu <[email protected]>

peterzhuamazon · 2025-02-19T00:50:15Z

Still seeing:

» WARN ][o.o.a.s.TransportSearchAction] [integTest-0] onPhaseEnd listener [org.opensearch.action.search.SearchRequestStats@514c3086] failed
»  java.lang.IllegalArgumentException: No enum constant org.opensearch.action.search.SearchPhaseName.CREATE_PIT

And more:

 2> REPRODUCE WITH: ./gradlew ':integ-test:integTest' --tests "org.opensearch.sql.legacy.AggregationIT.termsWithMissing" -Dtests.seed=BF805D501F710CAA -Dtests.security.manager=false -Dtests.locale=ksb-TZ -Dtests.timezone=America/Yakutat -Druntime.java=21
  2> java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([BF805D501F710CAA:8AAF1C5005C80CFC]:0)
        at org.junit.Assert.fail(Assert.java:87)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.junit.Assert.assertTrue(Assert.java:53)
        at org.opensearch.sql.legacy.AggregationIT.termsWithMissing(AggregationIT.java:449)

opensearch-project/OpenSearch#16781

noCharger · 2025-02-19T06:37:10Z

Still seeing:

» WARN ][o.o.a.s.TransportSearchAction] [integTest-0] onPhaseEnd listener [org.opensearch.action.search.SearchRequestStats@514c3086] failed
»  java.lang.IllegalArgumentException: No enum constant org.opensearch.action.search.SearchPhaseName.CREATE_PIT

And more:

 2> REPRODUCE WITH: ./gradlew ':integ-test:integTest' --tests "org.opensearch.sql.legacy.AggregationIT.termsWithMissing" -Dtests.seed=BF805D501F710CAA -Dtests.security.manager=false -Dtests.locale=ksb-TZ -Dtests.timezone=America/Yakutat -Druntime.java=21
  2> java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([BF805D501F710CAA:8AAF1C5005C80CFC]:0)
        at org.junit.Assert.fail(Assert.java:87)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.junit.Assert.assertTrue(Assert.java:53)
        at org.opensearch.sql.legacy.AggregationIT.termsWithMissing(AggregationIT.java:449)

opensearch-project/OpenSearch#16781

I reproduced this test failure locally and raised an issue in the core repository. opensearch-project/OpenSearch#17391; There's no query plan change on sql plugin (verified via explain API).

@peterzhuamazon It would be helpful if you could test following those steps by sending dsl queries to the alpha1 cluster.

peterzhuamazon · 2025-02-21T21:26:45Z

Retry after opensearch-project/OpenSearch#17418.

noCharger

Thanks for the change! Can we also briefly mention the import changes in PR description?

peterzhuamazon · 2025-02-21T22:38:05Z

Thanks for the change! Can we also briefly mention the import changes in PR description?

Updated, thanks!

peterzhuamazon · 2025-02-21T22:38:12Z

Thanks for the change! Can we also briefly mention the import changes in PR description?

Updated, thanks!

Bump gradle 8.10.2 / JDK23 / 3.0.0.0-alpha1 on SQL plugin

c515a35

Signed-off-by: Peter Zhu <[email protected]>

peterzhuamazon added release v3.0.0 labels Feb 12, 2025

peterzhuamazon self-assigned this Feb 12, 2025

peterzhuamazon mentioned this pull request Feb 12, 2025

Bump dashboards query workbench to version 3.0.0.0-alpha1 opensearch-project/dashboards-query-workbench#444

Draft

1 task

peterzhuamazon changed the title ~~Bump gradle 8.10.2 / JDK23 / 3.0.0.0-alpha1 on SQL plugin~~ [Release 3.0] Bump gradle 8.10.2 / JDK23 / 3.0.0.0-alpha1 on SQL plugin Feb 12, 2025

peterzhuamazon mentioned this pull request Feb 12, 2025

[RELEASE] Release version 3.0.0 opensearch-project/opensearch-build#3747

Open

66 tasks

peterzhuamazon added 2 commits February 13, 2025 01:46

Update imports

afea919

Signed-off-by: Peter Zhu <[email protected]>

Update more imports and github actions with jdk23

5f283e6

Signed-off-by: Peter Zhu <[email protected]>

peterzhuamazon marked this pull request as ready for review February 13, 2025 01:55

peterzhuamazon requested review from ps48, kavithacm, derek-ho, joshuali925, dai-chen, YANG-DB, mengweieric, Swiddis, penghuo, seankao-az, MaxKsyunz, Yury-Fridlyand, anirudha, forestmvey, acarbonetto, GumpacG, ykmr1224 and LantaoJin as code owners February 13, 2025 01:55

peterzhuamazon requested a review from noCharger as a code owner February 13, 2025 01:55

peterzhuamazon added 2 commits February 13, 2025 02:08

More imports

c992c92

Signed-off-by: Peter Zhu <[email protected]>

Apply spotless

3556d49

Signed-off-by: Peter Zhu <[email protected]>

Update byte-buddy to 1.14.19 to support JDK23

1d766ac

Signed-off-by: Peter Zhu <[email protected]>

noCharger reviewed Feb 13, 2025

View reviewed changes

peterzhuamazon mentioned this pull request Feb 13, 2025

[Bugfix] Fix IllegalArgumentException thrown when creating a PIT opensearch-project/OpenSearch#16781

Merged

3 tasks

Update bwc test

8df7993

Signed-off-by: Peter Zhu <[email protected]>

Update doctest scores

1cd7634

Signed-off-by: Peter Zhu <[email protected]>

peterzhuamazon mentioned this pull request Feb 19, 2025

[BUG] No enum constant org.opensearch.action.search.SearchPhaseName.CREATE_PIT opensearch-project/OpenSearch#16750

Closed

noCharger approved these changes Feb 21, 2025

View reviewed changes

rishabh6788 approved these changes Feb 21, 2025

View reviewed changes

peterzhuamazon merged commit 28275b8 into opensearch-project:main Feb 21, 2025
22 checks passed

peterzhuamazon deleted the bump-3.0.0.0-alpha1 branch February 21, 2025 22:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Release 3.0] Bump gradle 8.10.2 / JDK23 / 3.0.0.0-alpha1 on SQL plugin #3319

[Release 3.0] Bump gradle 8.10.2 / JDK23 / 3.0.0.0-alpha1 on SQL plugin #3319

peterzhuamazon commented Feb 12, 2025 •

edited

Loading

peterzhuamazon commented Feb 12, 2025 •

edited

Loading

peterzhuamazon commented Feb 12, 2025

peterzhuamazon commented Feb 13, 2025

peterzhuamazon commented Feb 13, 2025 •

edited

Loading

peterzhuamazon commented Feb 13, 2025

peterzhuamazon commented Feb 13, 2025

noCharger left a comment

noCharger commented Feb 13, 2025

peterzhuamazon commented Feb 14, 2025

peterzhuamazon commented Feb 15, 2025 •

edited

Loading

noCharger commented Feb 18, 2025 •

edited

Loading

peterzhuamazon commented Feb 18, 2025

peterzhuamazon commented Feb 19, 2025

noCharger commented Feb 19, 2025 •

edited

Loading

peterzhuamazon commented Feb 21, 2025

noCharger left a comment •

edited

Loading

peterzhuamazon commented Feb 21, 2025

peterzhuamazon commented Feb 21, 2025

[Release 3.0] Bump gradle 8.10.2 / JDK23 / 3.0.0.0-alpha1 on SQL plugin #3319

[Release 3.0] Bump gradle 8.10.2 / JDK23 / 3.0.0.0-alpha1 on SQL plugin #3319

Conversation

peterzhuamazon commented Feb 12, 2025 • edited Loading

Description

Related Issues

Check List

peterzhuamazon commented Feb 12, 2025 • edited Loading

peterzhuamazon commented Feb 12, 2025

peterzhuamazon commented Feb 13, 2025

peterzhuamazon commented Feb 13, 2025 • edited Loading

peterzhuamazon commented Feb 13, 2025

peterzhuamazon commented Feb 13, 2025

noCharger left a comment

Choose a reason for hiding this comment

noCharger commented Feb 13, 2025

peterzhuamazon commented Feb 14, 2025

peterzhuamazon commented Feb 15, 2025 • edited Loading

noCharger commented Feb 18, 2025 • edited Loading

peterzhuamazon commented Feb 18, 2025

peterzhuamazon commented Feb 19, 2025

noCharger commented Feb 19, 2025 • edited Loading

peterzhuamazon commented Feb 21, 2025

noCharger left a comment • edited Loading

Choose a reason for hiding this comment

peterzhuamazon commented Feb 21, 2025

peterzhuamazon commented Feb 21, 2025

peterzhuamazon commented Feb 12, 2025 •

edited

Loading

peterzhuamazon commented Feb 12, 2025 •

edited

Loading

peterzhuamazon commented Feb 13, 2025 •

edited

Loading

peterzhuamazon commented Feb 15, 2025 •

edited

Loading

noCharger commented Feb 18, 2025 •

edited

Loading

noCharger commented Feb 19, 2025 •

edited

Loading

noCharger left a comment •

edited

Loading