Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

queue is in state Stopped flakyness #4374

Closed
koivunej opened this issue May 30, 2023 · 1 comment · Fixed by #4379
Closed

queue is in state Stopped flakyness #4374

koivunej opened this issue May 30, 2023 · 1 comment · Fixed by #4379
Labels
a/test/flaky Area: related to flaky tests c/storage/pageserver Component: storage: pageserver

Comments

@koivunej
Copy link
Member

koivunej commented May 30, 2023 •
edited
Loading

Noticed a new kind of flakyness:

ERROR test_runner/regress/test_timeline_delete.py::test_timeline_resurrection_on_attach[release-pg14-True-mock_s3] - AssertionError: assert not ['2023-05-30T12:54:31.842158Z ERROR compaction_loop{tenant_id=2c768f27e47546fad60025da63b54338}: Compaction failed, retrying in 2s: queue is in state Stopped\n']

Allure: https://neon-github-public-dev.s3.amazonaws.com/reports/pr-4372/5121613817/index.html#suites/f588e0a787c49e67b29490359c589fae/1ee34a64a74d46f9

Those nagging compaction failed ... but I wonder if we should just allow this globally? This believe this was added in is_deleted PR.

Originally posted by @koivunej in #4372 (comment)
@koivunej koivunej added a/test/flaky Area: related to flaky tests c/storage/pageserver Component: storage: pageserver labels May 30, 2023
@LizardWizzard
Copy link
Contributor

+1 to allow it globally, this is expected given our collaborative shutdown approach for the UploadQueue

@LizardWizzard LizardWizzard mentioned this issue May 30, 2023
Merged
5 tasks
LizardWizzard added a commit that referenced this issue May 30, 2023
@LizardWizzard
reduce flakiness by allowing Compaction failed, retrying in X queue i…
b190c3e 
…s in state Stopped. (#4379)

resolves #4374 by adding the error to allowed_errors
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
a/test/flaky Area: related to flaky tests c/storage/pageserver Component: storage: pageserver
Projects
None yet
Milestone
No milestone
2 participants
@koivunej @LizardWizzard