Prevent failed post-stop script task from stopping Upstart respawning #765
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Overview
After one of our Docker processes ends, we configure Upstart to kill the container and delete it. However, because the process has ended, the command Upstart is configured to kill the container fails:
This failed
docker kill
command would cause Upstart to abort the process respawn, leaving the service stopped.It's not clear if
docker kill
is needed in thepost-stop script
section, but in order to prevent a failure from stopping Upstart from attempting to respawn the process this appends|| true
to each command so that they will be respawned even if one of thepost-stop
commands fails.Checklist
Testing Instructions
vagrant provision celery
sudo tail -f /var/log/upstart/driver-gradle.log
sudo service redis-server stop
celery
SSH terminal showing the Gradle process being repeatedly respawned and failingsudo service redis-server start
vagrant reload database
celery
SSH terminal showing the Gradle process being respawned, and recover when the database server has finished reloadingCloses #164140129