ERROR - pickle data was truncated on TaskInstance: airflow-db-cleanup.cleanup_BaseXCom

[artik00]

When Running %s on host %s <TaskInstance: airflow-db-cleanup.cleanup_BaseXCom 2020-09-09T00:00:00+00:00
It runs for 1 minute and then fails with :
[2020-09-10 00:04:11,000] {taskinstance.py:1150} ERROR - pickle data was truncated
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/airflow/models/taskinstance.py", line 984, in run_raw_task
result = task_copy.execute(context=context)
File "/usr/local/lib/python3.6/site-packages/airflow/operators/python_operator.py", line 113, in execute
return_value = self.execute_callable()
File "/usr/local/lib/python3.6/site-packages/airflow/operators/python_operator.py", line 118, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/usr/local/airflow/dags/maintenance/airflow-db-cleanup.py", line 289, in cleanup_function
entries_to_delete = query.all()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3319, in all
return list(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 101, in instances
cursor.close()
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 69, in exit
exc_value, with_traceback=exc_tb,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise
raise exception
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 81, in instances
rows = [proc(row) for row in fetch]
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 81, in
rows = [proc(row) for row in fetch]
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/loading.py", line 602, in _instance
state.manager.dispatch.load(state, context)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/event/attr.py", line 322, in call
fn(*args, **kw)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/mapper.py", line 3378, in _event_on_load
instrumenting_mapper._reconstructor(state.obj())
File "/usr/local/lib/python3.6/site-packages/airflow/models/xcom.py", line 72, in init_on_load
self.value = pickle.loads(self.value)
_pickle.UnpicklingError: pickle data was truncated
[2020-09-10 00:04:11,005] {taskinstance.py:1194} INFO - Marking task as FAILED. dag_id=airflow-db-cleanup, task_id=cleanup_BaseXCom, execution_date=20200909T000000, start_date=20200910T000320, end_date=20200910T000411
[2020-09-10 00:04:15,716] {local_task_job.py:102} INFO - Task exited with return code 1

[Kolpinetz]

Hello @artik00,

Have you found a solution to the problem?

[artik00]

@Kolpinetz I think I did back then, but now I don't remember what it was. I left my previous work so cannot check the code base. I vaguely remember implementing some parts of the cleanup myself, or something like that. Sorry I cannot help more.