-
Notifications
You must be signed in to change notification settings - Fork 14.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fetch intermediate log async GKEStartPod #39348
Conversation
9dd6061
to
e594417
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Can you add the test case with last_log_time and last_log_time?
Any updates on this? |
I'll try address the pending review by end of this week |
Bump on this 🙏 |
@pankajastro Any ETA on this? Thanks |
786ba83
to
3c60132
Compare
CI is green now. Hopefully, we will merge soon. In the meantime, could you please test it once? |
* Fetch intermediate log in async GKEStartPod This PR introduces a parameter that enables the retrieval of intermediate logs for the GKEStartPod asynchronous operator. Add param last_log_time and logging_interval in GKEStartPodTrigger serialize Add optional param last_log_time in method invoke_defer_method Example DAG: start_pod = GKEStartPodOperator( task_id="start_pod", project_id=PROJECT_ID, location=LOCATION, cluster_name=GKE_CLUSTER_NAME, do_xcom_push=True, namespace=GKE_NAMESPACE, image="ubuntu:jammy", cmds=["sh", "-c", "timeout 300 bash -c 'while true; do echo \"meow\"; sleep 30; done'"], name="test-sleep", in_cluster=False, on_finish_action="delete_pod", deferrable=True, get_logs=True, logging_interval=5, gcp_conn_id=GCP_CONN_ID )
* Fetch intermediate log in async GKEStartPod This PR introduces a parameter that enables the retrieval of intermediate logs for the GKEStartPod asynchronous operator. Add param last_log_time and logging_interval in GKEStartPodTrigger serialize Add optional param last_log_time in method invoke_defer_method Example DAG: start_pod = GKEStartPodOperator( task_id="start_pod", project_id=PROJECT_ID, location=LOCATION, cluster_name=GKE_CLUSTER_NAME, do_xcom_push=True, namespace=GKE_NAMESPACE, image="ubuntu:jammy", cmds=["sh", "-c", "timeout 300 bash -c 'while true; do echo \"meow\"; sleep 30; done'"], name="test-sleep", in_cluster=False, on_finish_action="delete_pod", deferrable=True, get_logs=True, logging_interval=5, gcp_conn_id=GCP_CONN_ID )
This PR introduces a parameter that enables the retrieval of intermediate logs for the GKEStartPod asynchronous operator.
last_log_time
andlogging_interval
in GKEStartPodTrigger serializelast_log_time
in methodinvoke_defer_method
Example DAG:
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.