[CI] SqlSearchIT testAllTypesWithRequestToUpgradedNodes failing

[mark-vieira]

This is busted in our BWC tests with node versions >= 7.10.0 and < 7.14.0. SqlCompatIT.testNullsOrderBeforeMissingOrderSupportQueryingNewNode is also failing on the same builds with the same error.

Build scan:
https://gradle-enterprise.elastic.co/s/ihwhafl6ggimq/tests/:x-pack:plugin:sql:qa:mixed-node:v7.12.1%23mixedClusterTest/org.elasticsearch.xpack.sql.qa.mixed_node.SqlSearchIT/testAllTypesWithRequestToUpgradedNodes

Reproduction line:
./gradlew ':x-pack:plugin:sql:qa:mixed-node:v7.12.1#mixedClusterTest' -Dtests.class="org.elasticsearch.xpack.sql.qa.mixed_node.SqlSearchIT" -Dtests.method="testAllTypesWithRequestToUpgradedNodes" -Dtests.seed=8B155910105B6087 -Dtests.bwc=true -Dtests.locale=en-ZA -Dtests.timezone=Europe/Warsaw -Druntime.java=8

Applicable branches:
7.x

Reproduces locally?:
Yes

Failure history:
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.xpack.sql.qa.mixed_node.SqlSearchIT&tests.test=testAllTypesWithRequestToUpgradedNodes

Failure excerpt:

org.elasticsearch.client.ResponseException: method [POST], host [http://127.0.0.1:39823], URI [_sql], status line [HTTP/1.1 500 Internal Server Error]
{"error":{"root_cause":[{"type":"transport_serialization_exception","reason":"Failed to deserialize response from handler [org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.action.ActionListenerResponseHandler@1196680f/org.elasticsearch.action.support.TransportAction$1@554a8ab8]"}],"type":"transport_serialization_exception","reason":"Failed to deserialize response from handler [org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler/org.elasticsearch.action.ActionListenerResponseHandler@1196680f/org.elasticsearch.action.support.TransportAction$1@554a8ab8]","caused_by":{"type":"e_o_f_exception","reason":null}},"status":500}

  at __randomizedtesting.SeedInfo.seed([8B155910105B6087:F06FF849C06FEA2D]:0)
  at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:326)
  at org.elasticsearch.client.RestClient.performRequest(RestClient.java:296)
  at org.elasticsearch.client.RestClient.performRequest(RestClient.java:270)
  at org.elasticsearch.xpack.sql.qa.mixed_node.SqlSearchIT.runSql(SqlSearchIT.java:316)
  at org.elasticsearch.xpack.sql.qa.mixed_node.SqlSearchIT.lambda$assertAllTypesWithNodes$19(SqlSearchIT.java:303)
  at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:1027)
  at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:1012)
  at org.elasticsearch.xpack.sql.qa.mixed_node.SqlSearchIT.assertAllTypesWithNodes(SqlSearchIT.java:303)
  at org.elasticsearch.xpack.sql.qa.mixed_node.SqlSearchIT.testAllTypesWithRequestToUpgradedNodes(SqlSearchIT.java:205)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-2)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:498)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
  at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
  at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
  at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
  at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
  at java.lang.Thread.run(Thread.java:748)

[elasticmachine]

Pinging @elastic/es-ql (Team:QL)

[martijnvg]

I will mute this test for now to reduce noise.