-
Notifications
You must be signed in to change notification settings - Fork 14.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Retry K8s tests on failures #46064
Retry K8s tests on failures #46064
Conversation
1370ac3
to
d7dee31
Compare
Looks good. |
This reverts commit cffb3c8.
@@ -447,7 +447,7 @@ def check_async_run_results( | |||
from airflow_breeze.utils.docker_command_utils import fix_ownership_using_docker | |||
|
|||
fix_ownership_using_docker() | |||
sys.exit(1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@shubhamraj-git -> yeah here, the problem is that that we usually exited here.
@@ -458,6 +458,8 @@ def check_async_run_results( | |||
for output in outputs: | |||
Path(output.file_name).unlink(missing_ok=True) | |||
|
|||
return tests_failed |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The right solution here will be:
if tests_failed:
sys.exit(1)
The problem is that we do not check the result of check_async_run_results
- the method was supposed to "sys.exit(1)` here rather than return False.
My bad I have not noticed it :(
This reverts commit cffb3c8.
This reverts commit cffb3c8.
This reverts commit cffb3c8.
This reverts commit cffb3c8.
We face a lot of K8s failures on CI, Which usually gets pass after re-run.
This PR allows it to re-run for specific number of times, default being 2 without manual intervention.
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.