Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

schema: TestRevertingJobsOnDatabasesAndSchemas/failed_due_to_injected_error/drop_database_cascade flaky #77048

Closed
RichardJCai opened this issue Feb 25, 2022 · 3 comments · Fixed by #85807
Closed

schema: TestRevertingJobsOnDatabasesAndSchemas/failed_due_to_injected_error/drop_database_cascade flaky #77048

RichardJCai opened this issue Feb 25, 2022 · 3 comments · Fixed by #85807
Assignees
@ajwerner
Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)

Comments

@RichardJCai
Copy link
Contributor

RichardJCai commented Feb 25, 2022 •
edited by cockroach-jira-scripts
Loading

TestRevertingJobsOnDatabasesAndSchemas/failed_due_to_injected_error/drop_database_cascade is flaky, see the test history: https://teamcity.cockroachdb.com/project.html?projectId=Cockroach_UnitTests&testNameId=-1013794384827983388&tab=testDetails

Going to skip for now.

Removing test in #77049

Jira issue: CRDB-13384
@RichardJCai RichardJCai added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-sql-schema-deprecated Use T-sql-foundations instead labels Feb 25, 2022
@postamar
Copy link 

 Error:        Received unexpected error:
                        query 'SELECT num_runs > 3 FROM crdb_internal.jobs WHERE status = 'reverting' AND description ~ '^DROP DATABASE db_drop CASCADE$'': expected:
                        (1) attached stack trace
                          -- stack trace:
                          | github.com/cockroachdb/cockroach/pkg/testutils/sqlutils.(*SQLRunner).CheckQueryResultsRetry.func1
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/testutils/sqlutils/sql_runner.go:238
                          | github.com/cockroachdb/cockroach/pkg/testutils.SucceedsWithinError.func1
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:71
                          | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:207
                          | github.com/cockroachdb/cockroach/pkg/testutils.SucceedsWithinError
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:77
                          | github.com/cockroachdb/cockroach/pkg/testutils/sqlutils.(*SQLRunner).succeedsWithin
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/testutils/sqlutils/sql_runner.go:74
                          | github.com/cockroachdb/cockroach/pkg/testutils/sqlutils.(*SQLRunner).CheckQueryResultsRetry
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/testutils/sqlutils/sql_runner.go:235
                          | github.com/cockroachdb/cockroach/pkg/sql_test.TestRevertingJobsOnDatabasesAndSchemas.func1.2
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/sql/schema_changer_test.go:6850
                          | testing.tRunner
                          |   /usr/local/go/src/testing/testing.go:1259
                          | runtime.goexit
                          |   /usr/local/go/src/runtime/asm_amd64.s:1581
                        Wraps: (2) query 'SELECT num_runs > 3 FROM crdb_internal.jobs WHERE status = 'reverting' AND description ~ '^DROP DATABASE db_drop CASCADE$'': expected:
                          | true
                          | got:
                          |
                        Error types: (1) *withstack.withStack (2) *errutil.leafError
          Test:         TestRevertingJobsOnDatabasesAndSchemas/failed_due_to_injected_error/drop_database_cascade

@ajwerner ajwerner self-assigned this Mar 1, 2022
@postamar
Copy link

postamar commented Mar 8, 2022

My understanding is that this test asserts on side effects from the old schema changer, which might not apply to the new. For a start in the new schema changer these jobs should be non-revertible.

@mgartner
Copy link
Collaborator

mgartner commented Aug 4, 2022

I'm reopening this issue because I witnessed this test fail on a recent CI build: https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_UnitTests/5966731?showRootCauses=true&expandBuildChangesSection=true&expandBuildProblemsSection=true&expandBuildTestsSection=true

------- Stdout: -------
=== RUN   TestRevertingJobsOnDatabasesAndSchemas/failed_due_to_injected_error/drop_schema
    sql_runner.go:101:
          Error Trace:  sql_runner.go:114
                              sql_runner.go:284
                              schema_changer_test.go:6931
          Error:        Received unexpected error:
                        query 'SELECT num_runs > 3 FROM crdb_internal.jobs WHERE status = 'reverting' AND description ~ '^DROP SCHEMA db_drop_schema.sc$'': expected:
                        (1) attached stack trace
                          -- stack trace:
                          | github.com/cockroachdb/cockroach/pkg/testutils/sqlutils.(*SQLRunner).CheckQueryResultsRetry.func1
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/testutils/sqlutils/sql_runner.go:287
                          | github.com/cockroachdb/cockroach/pkg/testutils.SucceedsWithinError.func1
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:69
                          | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:207
                          | github.com/cockroachdb/cockroach/pkg/testutils.SucceedsWithinError
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/testutils/soon.go:75
                          | github.com/cockroachdb/cockroach/pkg/testutils/sqlutils.(*SQLRunner).succeedsWithin
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/testutils/sqlutils/sql_runner.go:114
                          | github.com/cockroachdb/cockroach/pkg/testutils/sqlutils.(*SQLRunner).CheckQueryResultsRetry
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/testutils/sqlutils/sql_runner.go:284
                          | github.com/cockroachdb/cockroach/pkg/sql_test.TestRevertingJobsOnDatabasesAndSchemas.func1.2
                          |   /go/src/github.com/cockroachdb/cockroach/pkg/sql/schema_changer_test.go:6931
                          | testing.tRunner
                          |   /usr/local/go/src/testing/testing.go:1439
                          | runtime.goexit
                          |   /usr/local/go/src/runtime/asm_amd64.s:1571
                        Wraps: (2) query 'SELECT num_runs > 3 FROM crdb_internal.jobs WHERE status = 'reverting' AND description ~ '^DROP SCHEMA db_drop_schema.sc$'': expected:
                          | true
                          | got:
                          |
                        Error types: (1) *withstack.withStack (2) *errutil.leafError
        --- FAIL: TestRevertingJobsOnDatabasesAndSchemas/failed_due_to_injected_error/drop_schema (45.38s)

@mgartner mgartner reopened this Aug 4, 2022
@ajwerner ajwerner mentioned this issue Aug 9, 2022
Merged
craig bot pushed a commit that referenced this issue Aug 9, 2022
@ajwerner @rickystewart
Merge #85798 #85807 #85829
6e0b2f2 
85798: kvserver,roachpb,storage: add IsSpanEmptyRequest to check for any data r=erikgrinaker a=ajwerner

This commit adds a new request, `IsSpanEmptyRequest`, which checks to see if
there is any data in a key span of any kind. It ignore the GC threshold, and
operates across all versions.

Fixes #85726

Release note: None

85807: sql: deflake TestRevertingJobsOnDatabasesAndSchemas r=ajwerner a=ajwerner

The setting of the variable was not in the same session we needed it to be in.

Fixes #77048

Release note: None

85829: bazel: bump size of `serverccl_test` r=rail a=rickystewart

This has timed out in CI.

Release note: None

Co-authored-by: Andrew Werner <[email protected]>
Co-authored-by: Ricky Stewart <[email protected]>
@craig craig bot closed this as completed in 39256b7 Aug 9, 2022
@exalate-issue-sync exalate-issue-sync bot added T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) and removed T-sql-schema-deprecated Use T-sql-foundations instead labels May 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ajwerner ajwerner

Labels
C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

sql: deflake TestRevertingJobsOnDatabasesAndSchemas ajwerner/cockroach
4 participants
@mgartner @ajwerner @postamar @RichardJCai