Types removal - forward-compatible code for bulk API change coming in 7.0

[markharwood]

This rolling upgrade test now tolerates (but does not require) type removal warnings.
Mainly adding here to 6.x to make the master builds for my bulk types removal PR (36549) pass.

[elasticmachine]

Pinging @elastic/es-search

[jtibshirani]

@markharwood do you have the reproduction line for the failing test motivating this change? I would like to dig into it a bit to make sure I understand. In the meantime I'll start to review #36549 so as not to hold you up.

[markharwood]

I think the repro line was this one:

./gradlew :x-pack:qa:rolling-upgrade:with-system-key:v6.7.0#oneThirdUpgradedTestRunner -Dtests.seed=7EE10D0C67FF5836 -Dtests.class=org.elasticsearch.upgrades.IndexingIT -Dtests.method="testIndexing" -Dtests.security.manager=true -Dtests.locale=ar-BH -Dtests.timezone=US/Pacific-New -Dcompiler.java=11 -Druntime.java=8

Sadly it never reproduced on my macbook - it always failed with a different "timed out waiting for a connection" type error so I could only get it to fail in CI which made chasing it more tedious

[jtibshirani]

Thanks for the line, unfortunately I also wasn't able to reproduce the issue. After taking a look, I'm still having trouble understanding how the 6.x version of this test could be affecting the results of the test on master, since I think only the master test should be running (even if some of the nodes in the test are on 6.x). I wonder if it's worth catching up with the core infra team to see if they can shed light on the failures you've been seeing?

[markharwood]

@nik9000 Can you shed any light on the above?
I know you were involved with IndexingIT and deprecation warnings checks so can you explain why a PR in master would fail in the way described? The master server issues type removal warnings and the and the test client expects them but we see the RestClient barfing because of unexpected warnings - see here

My only thought was the test is running a 6.x client (which currently has no warnings expectations) against the 7.0 server.

[jtibshirani]

@markharwood looking over the linked build failures, could the issue be that we need to add expectWarnings to the x-pack version of IndexingIT as well? Here's the bulk call, which I didn't see modified yet in your PR: https://github.com/elastic/elasticsearch/blob/master/x-pack/qa/rolling-upgrade/src/test/java/org/elasticsearch/upgrades/IndexingIT.java#L138

[nik9000]

I'm still having trouble understanding how the 6.x version of this test could be affecting the results of the test on master, since I think only the master test should be running (even if some of the nodes in the test are on 6.x)

Unless something changed in the past few weeks, I think this statement is accurate. Our tests "look back" on previous versions.

[markharwood]

Closed as redundant - underlying issue was identified in #37105 (comment)