Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: just one way to shutdown a tenant #4407

Merged
merged 10 commits into from
Jun 6, 2023
Merged

refactor: just one way to shutdown a tenant #4407

merged 10 commits into from
Jun 6, 2023

Conversation

koivunej
Copy link
Member

@koivunej koivunej commented Jun 2, 2023

Problem

We have 2 ways of tenant shutdown, we should have just one.

Cc: #4327.

Summary of changes

Mostly mechanical simple refactorings.

Added warn! on the "shutdown all remaining tasks" should trigger test failures in the between time of not having solved the "tenant/timeline owns all spawned tasks" issue.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@github-actions
Copy link

github-actions bot commented Jun 2, 2023

1004 tests run: 964 passed, 0 failed, 40 skipped (full report)


Flaky tests (10)

Postgres 15

Postgres 14

The comment gets automatically updated with the latest test results
af801ba at 2023-06-06T12:29:45.678Z :recycle:

koivunej added 3 commits June 2, 2023 21:19
also make it non-pub, it shouldn't be called from anywhere except
shutdown.
good thing is that during the 10s most of the tenants and timelines
would had completed stopping.

one could consider if we should have a timeout here, but the shutdown
freeze_and_flush operations are best-effort. However, if a
timeline_delete is already ongoing, perhaps we should never attempt to
flush it.

this is however already tracked by flush loop being shut down, so there
is no need for more changes.
@koivunej
Copy link
Member Author

koivunej commented Jun 2, 2023

  • test_timeline_status[False]: release

Looks flaky.

@koivunej koivunej marked this pull request as ready for review June 5, 2023 06:39
@koivunej koivunej requested review from a team as code owners June 5, 2023 06:39
@koivunej koivunej requested review from lubennikovaav, bojanserafimov, problame and LizardWizzard and removed request for a team and lubennikovaav June 5, 2023 06:39
Copy link
Contributor

@LizardWizzard LizardWizzard left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. See one comment about potential to increase shutdown time

pageserver/src/tenant.rs Show resolved Hide resolved
pageserver/src/tenant.rs Outdated Show resolved Hide resolved
pageserver/src/tenant/mgr.rs Show resolved Hide resolved
@koivunej
Copy link
Member Author

koivunej commented Jun 6, 2023

  • test_threshold_based_eviction: release

Not sure about this...

  • test_delete_force[True]: debug
  • test_ondemand_download_timetravel[real_s3]: debug
  • test_tenant_physical_size[local_fs]: debug

Created #4426.

pageserver/src/tenant/mgr.rs Show resolved Hide resolved
@koivunej koivunej deleted the tenant_shutdown branch June 6, 2023 12:30
awestover pushed a commit that referenced this pull request Jun 14, 2023
We have 2 ways of tenant shutdown, we should have just one.

Changes are mostly mechanical simple refactorings.

Added `warn!` on the "shutdown all remaining tasks" should trigger test
failures in the between time of not having solved the "tenant/timeline
owns all spawned tasks" issue.

Cc: #4327.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants