clear-missing-dags failuring #136
Comments
|
I think i have the same issue. |
|
@waardd did you find a way to solve this issue? |
|
No i did not. |
|
I believe this is a dupe of this: #131 Try this as a solution: #131 (comment) |
|
@Robert-Zacchigna Thank you for your support! [2023-07-27, 19:02:04 UTC] {airflow-clear-missing-dags.py:110} INFO - Entry: <DAG: freshchat_source_agent_performance_v2> The above exception was the direct cause of the following exception: Traceback (most recent call last): [SQL: UPDATE dag SET dag_id = dag_id=%(param_1)s WHERE dag.dag_id = %(dag_id_1)s] [SQL: SELECT task_instance.task_id AS task_instance_task_id, task_instance.dag_id AS task_instance_dag_id, task_instance.run_id AS task_instance_run_id, task_instance.map_index AS task_instance_map_index, task_instance.start_date AS task_instance_start_date, task_instance.end_date AS task_instance_end_date, task_instance.duration AS task_instance_duration, task_instance.state AS task_instance_state, task_instance.try_number AS task_instance_try_number, task_instance.max_tries AS task_instance_max_tries, task_instance.hostname AS task_instance_hostname, task_instance.unixname AS task_instance_unixname, task_instance.job_id AS task_instance_job_id, task_instance.pool AS task_instance_pool, task_instance.pool_slots AS task_instance_pool_slots, task_instance.queue AS task_instance_queue, task_instance.priority_weight AS task_instance_priority_weight, task_instance.operator AS task_instance_operator, task_instance.queued_dttm AS task_instance_queued_dttm, task_instance.queued_by_job_id AS task_instance_queued_by_job_id, task_instance.pid AS task_instance_pid, task_instance.executor_config AS task_instance_executor_config, task_instance.updated_at AS task_instance_updated_at, task_instance.external_executor_id AS task_instance_external_executor_id, task_instance.trigger_id AS task_instance_trigger_id, task_instance.trigger_timeout AS task_instance_trigger_timeout, task_instance.next_method AS task_instance_next_method, task_instance.next_kwargs AS task_instance_next_kwargs |
|
hmm i think this is an issue with your airflow DB, its saying that some records in the "dag_id" table are null which is violating the not-null constraint: As a result, the process cant complete. My assumption is that the function is first performing some kind of integrity check on the data (in this case the table row) before checking if it needs to be cleared/removed.
Beyond the above, I'd need to see your script (if you modified it in anyway) and/or your DB (don't share the DB, that is not safe from a data protection perspective, I'm just saying where you need to look for problem resolution). You could also try checking the status/health of your DB using the airflow CLI, see here: https://airflow.apache.org/docs/apache-airflow/stable/howto/usage-cli.html#cli-db-clean MAKE A BACKUP OF THE DB FIRST BEFORE MODIFYING THE DB |
|
This solved it for me: I believe this is a dupe of this: #131 Try this as a solution: #131 (comment) tnx @Robert-Zacchigna Robert-Zacchigna |
AssertionError: Dependency rule tried to blank-out primary key column 'serialized_dag.dag_id' on instance '<SerializedDagModel at 0x7fc6bee71d90>'
[2023-06-12, 00:02:25 UTC] {standard_task_runner.py:100} ERROR - Failed to execute job 24386106 for task clear_missing_dags (This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: Dependency rule tried to blank-out primary key column 'serialized_dag.dag_id' on instance '<SerializedDagModel at 0x7fc6bee71d90>' (Background on this error at: https://sqlalche.me/e/14/7s2a); 7688)
[2023-06-12, 00:02:25 UTC] {local_task_job.py:208} INFO - Task exited with return code 1
[2023-06-12, 00:02:25 UTC] {taskinstance.py:2578} INFO - 0 downstream tasks scheduled from follow-on schedule check
The text was updated successfully, but these errors were encountered: