sql/logictest: TestLogic failed

[cockroach-teamcity]

sql/logictest.TestLogic failed with artifacts on master @ b8f13f26ed9647428ce3130c129adc913c60316c:

=== RUN   TestLogic
    test_log_scope.go:79: test logs captured to: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic112937844
    test_log_scope.go:80: use -show-logs to present logs inline
=== CONT  TestLogic
    logic.go:3555: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestLogic112937844
--- FAIL: TestLogic (2438.72s)
=== RUN   TestLogic/5node-disk/distsql_union/Except
    logic.go:2690: 
        
        testdata/logic_test/distsql_union:332: (SELECT y FROM xyz ORDER BY y) EXCEPT ALL (SELECT y FROM xyz ORDER BY y)
        expected success, but found
        (58C01) inbox communication error: rpc error: code = Canceled desc = context canceled
        inbox.go:328: in Next()
    logic.go:2416: 
         pq: inbox communication error: rpc error: code = Canceled desc = context canceled
    logic.go:1962: 
         testdata/logic_test/distsql_union:336: too many errors encountered, skipping the rest of the input
--- done: testdata/logic_test/distsql_union with config 5node-disk: 35 tests, 3 failures
--- total progress: 156339 statements/queries
--- progress: testdata/logic_test/hash_join_dist: 16 statements/queries
--- done: testdata/logic_test/hash_join_dist with config 5node-disk: 18 tests, 0 failures
            --- FAIL: TestLogic/5node-disk/distsql_union/Except (0.02s)
=== RUN   TestLogic/5node-disk/distsql_union
=== PAUSE TestLogic/5node-disk/distsql_union
=== CONT  TestLogic/5node-disk/distsql_union
        --- FAIL: TestLogic/5node-disk/distsql_union (1.44s)
=== RUN   TestLogic/5node-disk
    --- FAIL: TestLogic/5node-disk (0.00s)

Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)Parameters in this failure:

• GOFLAGS=-parallel=2

Same failure on other branches

• sql/logictest: TestLogic failed #63262 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-20.2]
• sql/logictest: TestLogic failed #61657 sql/logictest: TestLogic failed [C-test-failure O-robot branch-release-21.1]
• sql/logictest: TestLogic failed #58269 sql/logictest: TestLogic failed [C-test-failure O-robot T-sql-schema branch-release-20.1]

/cc @cockroachdb/sql-queries

This test on roachdash | Improve this report!

[yuzefovich]

I originally thought this issue was another instance of #72017, however there we have no inbound stream error and here we have a cancellation error. For the no inbound stream error I think it's possible that the recent change to max_running_flows to be dependent on the CPU count is to blame, so I want to be on lookout for those, but the context cancellation error in this issue seems benign to me, so I'll just close it and wait for more failures like this before investigating.

[yuzefovich]

This can be rarely reproduced on master even with the revert of #73887. I think it started happening because of 62ea0c6.

[yuzefovich]

Never mind, I reproduced this with #72998 reverted, but that PR might be making the reproduction more likely. My current thinking is that maybe the way vectorizedInboundStreamHandler.Timeout is somehow broken because that seems like to be affected the most by #72998, yet I don't see the problem with the implementation.