Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test: TestTenantLogicCCL_plpgsql_txn failed [3node-tenant tests are flaky with autocommit_before_ddl=true] #140172

Closed
cockroach-teamcity opened this issue Jan 30, 2025 · 1 comment · Fixed by #140400
Closed

pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test: TestTenantLogicCCL_plpgsql_txn failed [3node-tenant tests are flaky with autocommit_before_ddl=true] #140172

cockroach-teamcity opened this issue Jan 30, 2025 · 1 comment · Fixed by #140400
Assignees
@rafiss
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. P-1 Issues/test failures with a fix SLA of 1 month T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Jan 30, 2025 •
edited by cockroach-jira-scripts
Loading

pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test.TestTenantLogicCCL_plpgsql_txn failed with artifacts on master @ db0c794615911572c9c873e8c30ac9cd6d8b59f6:

=== RUN   TestTenantLogicCCL_plpgsql_txn
    test_log_scope.go:165: test logs captured to: /artifacts/tmp/_tmp/647c645250a3b6a1f6a8541f942b2509/logTestTenantLogicCCL_plpgsql_txn1939806204
    test_log_scope.go:76: use -show-logs to present logs inline
[23:08:01] setting distsql_workmem='25576B';
[23:08:01] setting distsql_workmem='25576B';
    logic.go:4425: -- test log scope end --
test logs left over in: /artifacts/tmp/_tmp/647c645250a3b6a1f6a8541f942b2509/logTestTenantLogicCCL_plpgsql_txn1939806204
--- FAIL: TestTenantLogicCCL_plpgsql_txn (10.96s)
=== RUN   TestTenantLogicCCL_plpgsql_txn/set_timestamp
    logic.go:3112: let $start_ts = 2025-01-30T23:08:04.023349Z
    logic.go:3112: let $ts_1 = 2025-01-30T23:08:03.023349Z
    logic.go:3112: let $ts_2 = 2025-01-30T23:08:02.023349Z
    logic.go:3112: let $ts_3 = 2025-01-30T23:08:01.023349Z
    logic.go:3112: let $ts_4 = 2025-01-30T23:08:00.023349Z
    logic.go:3112: let $ts_1 = 2025-01-30T23:08:04.194163Z
    logic.go:3112: let $ts_2 = 2025-01-30T23:08:04.227391Z
    logic.go:3112: let $ts_3 = 2025-01-30T23:08:04.255618Z
    logic.go:2273: 
         
        /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/5931/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test_/3node-tenant_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/logictestccl/testdata/logic_test/plpgsql_txn:804: 
        expected success, but found
        (40001) restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_CLIENT_REJECT): "sql txn" meta={id=37aa2bcd key=/Tenant/10/Table/3/1/105/2/1 iso=Serializable pri=0.00288878 epo=0 ts=1738278484.284389934,0 min=1738278484.284389934,0 seq=16} lock=true stat=PENDING rts=1738278484.284389934,0 wto=false gul=1738278484.784389934,0
[23:08:10] --- done: /home/roach/.cache/bazel/_bazel_roach/c5a4e7d36696d9cd970af2045211a7df/sandbox/processwrapper-sandbox/5931/execroot/com_github_cockroachdb_cockroach/bazel-out/aarch64-fastbuild/bin/pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test_/3node-tenant_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/logictestccl/testdata/logic_test/plpgsql_txn with config 3node-tenant: 73 tests, 1 failures
[23:08:11] --- total progress: 73 statements
--- total: 73 tests, 1 failures
    --- FAIL: TestTenantLogicCCL_plpgsql_txn/set_timestamp (6.42s)
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

Jira issue: CRDB-47020
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-queries SQL Queries Team labels Jan 30, 2025
@github-project-automation github-project-automation bot moved this to Triage in SQL Queries Jan 30, 2025
@yuzefovich yuzefovich mentioned this issue Jan 31, 2025
Merged
@rafiss rafiss mentioned this issue Jan 31, 2025
Merged
craig bot pushed a commit that referenced this issue Jan 31, 2025
@rafiss
Merge #140187
1d4df57 
140187: logictest: disable autocommit_before_ddl under 3node-tenant r=rafiss a=rafiss

A recent change to the default value of this setting appears to have made many 3node-tenant tests flaky. This patch goes back to the old value for multitenant configs while we investigate.

informs #140162, #140163, #140170, #140172, #140183, #140182, #140173, #140171, #140168, #140164, 
Release note: None

Co-authored-by: Rafi Shamim <[email protected]>
craig bot pushed a commit that referenced this issue Jan 31, 2025
@rafiss
Merge #140187
51db1cb 
140187: logictest: disable autocommit_before_ddl under 3node-tenant r=yuzefovich a=rafiss

A recent change to the default value of this setting appears to have made many 3node-tenant tests flaky. This patch goes back to the old value for multitenant configs while we investigate.

informs #140162, #140163, #140170, #140172, #140183, #140182, #140173, #140171, #140168, #140164, 
Release note: None

Co-authored-by: Rafi Shamim <[email protected]>
@cockroach-teamcity
Copy link
Member Author

pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test.TestTenantLogicCCL_plpgsql_txn failed on master @ 5b931856d005214057f0703890ce3a77cdc21f30:

=== RUN   TestTenantLogicCCL_plpgsql_txn
    test_log_scope.go:165: test logs captured to: outputs.zip/logTestTenantLogicCCL_plpgsql_txn3770882071
    test_log_scope.go:76: use -show-logs to present logs inline
[15:09:06] setting distsql_workmem='64870B';
[15:09:06] setting distsql_workmem='64870B';
    logic.go:4425: -- test log scope end --
test logs left over in: outputs.zip/logTestTenantLogicCCL_plpgsql_txn3770882071
--- FAIL: TestTenantLogicCCL_plpgsql_txn (10.93s)
=== RUN   TestTenantLogicCCL_plpgsql_txn/set_isolation
    logic.go:2273: 
         
        /var/lib/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test_/3node-tenant_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/logictestccl/testdata/logic_test/plpgsql_txn:836: 
        expected success, but found
        (40001) restart transaction: TransactionRetryWithProtoRefreshError: TransactionAbortedError(ABORT_REASON_CLIENT_REJECT): "sql txn" meta={id=d52d5e38 key=/Tenant/10/Table/3/1/107/2/1 iso=Serializable pri=0.03859769 epo=0 ts=1738336148.912528580,0 min=1738336148.912528580,0 seq=24} lock=true stat=PENDING rts=1738336148.912528580,0 wto=false gul=1738336149.412528580,0
[15:09:14] --- done: /var/lib/engflow/worker/work/0/exec/bazel-out/k8-fastbuild/bin/pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test_/3node-tenant_test.runfiles/com_github_cockroachdb_cockroach/pkg/ccl/logictestccl/testdata/logic_test/plpgsql_txn with config 3node-tenant: 75 tests, 1 failures
[15:09:15] --- total progress: 75 statements
--- total: 75 tests, 1 failures
    --- FAIL: TestTenantLogicCCL_plpgsql_txn/set_isolation (6.02s)

Parameters:

  • attempt=1
  • run=1
  • shard=44
Help

See also: How To Investigate a Go Test Failure (internal)

This test on roachdash | Improve this report!

@rafiss rafiss self-assigned this Jan 31, 2025
@rafiss rafiss added P-1 Issues/test failures with a fix SLA of 1 month T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) and removed release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-queries SQL Queries Team labels Jan 31, 2025
@rafiss rafiss changed the title pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test: TestTenantLogicCCL_plpgsql_txn failed pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test: TestTenantLogicCCL_plpgsql_txn failed [3node-tenant tests are flaky with autocommit_before_ddl=true] Jan 31, 2025
@exalate-issue-sync exalate-issue-sync bot changed the title pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test: TestTenantLogicCCL_plpgsql_txn failed [3node-tenant tests are flaky with autocommit_before_ddl=true] pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test: TestTenantLogicCCL_plpgsql_txn failed Jan 31, 2025
rafiss added a commit to rafiss/cockroach that referenced this issue Jan 31, 2025
@rafiss
multiregionccl: deflake TestColdStartLatency
bbf71b2 
Similar to what we're seeing in
cockroachdb#140172 and the issues
linked to cockroachdb#140187, this
test is flaky due to running with autocommit_before_ddl=true in a
multitenant setup.

Release note: None
@rafiss rafiss mentioned this issue Jan 31, 2025
Merged
craig bot pushed a commit that referenced this issue Jan 31, 2025
@taroface @andrewbaptist @rafiss
Merge #139365 #140071 #140241
6a0a025 
139365: docgen: update COPY TO STDOUT portions of SQL diagram r=mgartner a=taroface

Updated `COPY` diagram so that it better reflects the `COPY TO` syntax that was added in 23.1 (this PR will be backported to every version up until 23.1). These changes accompany the docs updates in cockroachdb/docs#19310
 
Current diagram:

<img width="647" alt="image" src="https://github.com/user-attachments/assets/eb70b831-3473-40aa-823f-5432a8cf2698" />

New diagram:

<img width="829" alt="image" src="https://github.com/user-attachments/assets/314e1345-cec8-4a72-9be2-fb4b2f2885ee" />

Epic: none
Release note: none
Release justification: non-production code change

140071: storage: covert SizeSpec to protobuf r=RaduBerinde a=andrewbaptist

This commit moves SizeSpec to the storagepb directory and converts it toa protobuf. Additionally it pulls shared storage into the global StorageConfig file.

Epic: none

Release note: None

140241: multiregionccl: deflake TestColdStartLatency r=rafiss a=rafiss

Similar to what we're seeing in
#140172 and the issues linked to #140187, this test is flaky due to running with autocommit_before_ddl=true in a multitenant setup.

fixes #140174
Release note: None

Co-authored-by: Ryan Kuo <[email protected]>
Co-authored-by: Andrew Baptist <[email protected]>
Co-authored-by: Rafi Shamim <[email protected]>
@yuzefovich yuzefovich removed this from SQL Queries Jan 31, 2025
@rafiss rafiss mentioned this issue Feb 3, 2025
Merged
@exalate-issue-sync exalate-issue-sync bot changed the title pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test: TestTenantLogicCCL_plpgsql_txn failed pkg/ccl/logictestccl/tests/3node-tenant/3node-tenant_test: TestTenantLogicCCL_plpgsql_txn failed [3node-tenant tests are flaky with autocommit_before_ddl=true] Feb 4, 2025
rafiss added a commit that referenced this issue Feb 5, 2025
@rafiss
multiregionccl: deflake TestColdStartLatency
ca62d13 
Similar to what we're seeing in
#140172 and the issues
linked to #140187, this
test is flaky due to running with autocommit_before_ddl=true in a
multitenant setup.

Release note: None
@craig craig bot closed this as completed in d276833 Feb 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rafiss rafiss

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. P-1 Issues/test failures with a fix SLA of 1 month T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

spanconfigreconciler: use fixed timestamp when reading descriptors rafiss/cockroach
2 participants
@rafiss @cockroach-teamcity