Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fail fast when contained dies in local-run. Resolves #213 #229

Merged
merged 1 commit into from
Feb 4, 2016
Merged

Fail fast when contained dies in local-run. Resolves #213 #229

merged 1 commit into from
Feb 4, 2016

Conversation

asottile
Copy link
Contributor

@asottile asottile commented Feb 4, 2016

No description provided.



@mock.patch('time.sleep', autospec=True)
@mock.patch('time.time', autospec=True)
@mock.patch('paasta_tools.cli.cmds.local_run.run_healthcheck_on_container',
autospec=True, return_value=[False, True])
autospec=True, side_effect=[False, False, True])
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this test was actually completely wrong! my changes ticked it so I fixed it

@asottile
Copy link
Contributor Author

asottile commented Feb 4, 2016

I also did some local testing with this branch:

Fast exiting

$ cli.py local-run -s www_pages
Local Makefile with 'cook-image' target detected. Assuming --build
Guesing cluster configuration for norcal-devb. To override, use the --cluster option.
Guessing instance configuration for main. To override, use the --instance option.
docker build -t paasta-local-run-www_pages-asottile .
Sending build context to Docker daemon 37.89 kB
Sending build context to Docker daemon 
Step 0 : FROM ubuntu:latest
 ---> 8693db7e8a00
Step 1 : CMD bash -exc echo RUNNING && exit 1
 ---> Using cache
 ---> 24dea624a3c6
Successfully built 24dea624a3c6
You're running a container in non-interactive mode.
This is how Mesos runs containers.
Note that some programs behave differently when running with no
tty attached (as your program is about to run).

Running docker command:
docker run <snip>
Found our container running with CID 93ffa70565dc494b0414e339c45e0a101b6f4de184647866a94a086397cedd52

Starting health check via http://dev25-devc.dev.yelpcorp.com:45028/status (waiting 60 seconds before considering failures due to grace period):
Healthcheck failed! (disregarded due to grace period)
Container exited with code 1
Your service failed to start, here is the stdout and stderr
+ echo RUNNING
+ exit 1
RUNNING

Stopping and removing the old container 93ffa70565dc494b0414e339c45e0a101b6f4de184647866a94a086397cedd52...
(Please wait or you may leave an orphaned container.)
...done

Slow exiting

$ cli.py local-run -s www_pages
Local Makefile with 'cook-image' target detected. Assuming --build
Guesing cluster configuration for norcal-devb. To override, use the --cluster option.
Guessing instance configuration for main. To override, use the --instance option.
docker build -t paasta-local-run-www_pages-asottile .
Sending build context to Docker daemon 37.89 kB
Sending build context to Docker daemon 
Step 0 : FROM ubuntu:latest
 ---> 8693db7e8a00
Step 1 : CMD bash -exc echo RUNNING && sleep 65 && exit 0
 ---> Using cache
 ---> 0b54b2d90e42
Successfully built 0b54b2d90e42
You're running a container in non-interactive mode.
This is how Mesos runs containers.
Note that some programs behave differently when running with no
tty attached (as your program is about to run).

Running docker command:
docker run <snip>
Found our container running with CID c7d0ced2c1d83219a7e17831b9ef22d59db67a4e503a70878f4917505988c91d

Starting health check via http://dev25-devc.dev.yelpcorp.com:53121/status (waiting 60 seconds before considering failures due to grace period):
Healthcheck failed! (disregarded due to grace period)
Healthcheck failed! (disregarded due to grace period)
Healthcheck failed! (disregarded due to grace period)
Healthcheck failed! (disregarded due to grace period)
Healthcheck failed! (disregarded due to grace period)
Healthcheck failed! (disregarded due to grace period)
Healthcheck failed! (Attempt 1 of 30) (via: http://dev25-devc.dev.yelpcorp.com:53121/status)
Container exited with code 0
Your service failed to start, here is the stdout and stderr
+ echo RUNNING
+ sleep 65
RUNNING
+ exit 0

Stopping and removing the old container c7d0ced2c1d83219a7e17831b9ef22d59db67a4e503a70878f4917505988c91d...
(Please wait or you may leave an orphaned container.)
...done

@solarkennedy
Copy link
Contributor

This looks great! Thanks so much!

solarkennedy added a commit that referenced this pull request Feb 4, 2016
Fail fast when contained dies in local-run.  Resolves #213
@solarkennedy solarkennedy merged commit f5d9ee4 into Yelp:master Feb 4, 2016
@asottile asottile deleted the faster_fail_dead_container_213 branch February 4, 2016 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants