Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

alloc_runner: stop sidecar tasks last #13055

Merged
merged 6 commits into from
Jun 7, 2022

Conversation

DerekStrickland
Copy link
Contributor

@DerekStrickland DerekStrickland commented May 18, 2022

Closes #10340

This PR addresses a defect where sidecar tasks are stopped prior to their associated main task being successfully stopped. This can create problems in scenarios where the main task can take a long time to shut down, but still need their sidecars to be operational while they finish their shutdown task (e.g. Consul Connect proxy).

This fix includes the following changes:

  • Adds the IsSidecarTask helper method.
  • Refactors other helpers to use IsSidecarTask.
  • Updates the doc string on the hasSidecarTasks method to correctly describe the behavior of the function.
  • Modifies the second kill loop in allocRunner.killTasks to skip shutting down sidecar tasks. This allows the existing logic in allocHandler.handleTaskStateUpdates to shutdown sidecars after the main task has exited as it was designed to do.
  • Renames a variable to avoid package import collision.

@DerekStrickland DerekStrickland force-pushed the f-stop-sidecar-after-parent-task branch from 86cf32f to be7adb9 Compare June 3, 2022 14:26
@DerekStrickland DerekStrickland changed the title [WIP] alloc_runner: stop sidecar tasks last alloc_runner: stop sidecar tasks last Jun 3, 2022
@DerekStrickland DerekStrickland marked this pull request as ready for review June 3, 2022 18:48
Copy link
Contributor

@lgfa29 lgfa29 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor suggestion about the log message.

I was also looking at the existing test suite, but it doesn't seem like we have any timing based tests. One thing that I thought was to check the task event order in TestAllocRunner_Lifecycle_Poststart but I don't know how reliable that would be.

@DerekStrickland
Copy link
Contributor Author

@lgfa29

Thanks for catching the test omission. I'll take a pass at adding a specific test for this.

@DerekStrickland DerekStrickland merged commit ec3b715 into main Jun 7, 2022
@DerekStrickland DerekStrickland deleted the f-stop-sidecar-after-parent-task branch June 7, 2022 15:35
@lgfa29 lgfa29 added backport/1.1.x backport to 1.1.x release line backport/1.2.x backport to 1.1.x release line backport/1.3.x backport to 1.3.x release line labels Jun 9, 2022
@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 120 days ⏳. This helps our maintainers find and focus on the active contributions.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 26, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backport/1.1.x backport to 1.1.x release line backport/1.2.x backport to 1.1.x release line backport/1.3.x backport to 1.3.x release line theme/task lifecycle
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Option for sidecar tasks to only be killed after main task
2 participants