Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TriggerDagRunOperator to configure the run_id of the triggered dag #17438

Closed
waleedsamy opened this issue Aug 5, 2021 · 1 comment · Fixed by #18788
Closed

TriggerDagRunOperator to configure the run_id of the triggered dag #17438

waleedsamy opened this issue Aug 5, 2021 · 1 comment · Fixed by #18788
Labels
kind:feature Feature Requests

Comments

@waleedsamy
Copy link

Description

Airflow 1.10 support providing a run_id to TriggerDagRunOperator using DagRunOrder object that will be returned after calling TriggerDagRunOperator#python_callable. With #6317 (Airflow 2.0), this behavior changed and one could not provide run_id anymore to the triggered dag, which is very odd to say the least.

Use case / motivation

I want to be able to provide run_id for the TriggerDagRunOperator. In my case, I use the TriggerDagRunOperator to trigger 30K dag_run daily and will be so annoying to see them having unidentifiable run_id.

After discussing with @ashb
Suggested fixes can be

  • provide a templated run_id as a parameter to TriggerDagRunOperator.
  • restore the old behavior of Airflow 1.10, where DagRunOrder holds the run_id and dag_run_conf of the triggered dag
  • create a new sub-class of TriggerDagRunOperator to fully customize it

Are you willing to submit a PR?

Related Issues

@ashb
Copy link
Member

ashb commented Sep 15, 2021

/cc @BasPH

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:feature Feature Requests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants