Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error in matching columns #426

Closed
srggrs opened this issue Dec 2, 2020 · 0 comments · Fixed by #427
Closed

Error in matching columns #426

srggrs opened this issue Dec 2, 2020 · 0 comments · Fixed by #427
Assignees
Labels
bug Something isn't working

Comments

@srggrs
Copy link
Contributor

srggrs commented Dec 2, 2020

When running the pipeline with latest master, using parallel association, I get the following error, which breaks the run:

2020-12-02 03:44:30,116 association INFO Association iteration: #8 (sky region group 4)
2020-12-02 03:44:30,113 runpipeline ERROR Processing error:
The columns in the computed data do not match the columns in the provided metadata
  Extra:   ['flux_int_isl_ratio', 'flux_peak_isl_ratio']
  Missing: []
Traceback (most recent call last):
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast_pipeline/management/commands/runpipeline.py", line 134, in run_pipe
    pipeline.process_pipeline(p_run)
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast_pipeline/pipeline/main.py", line 344, in process_pipeline
    n_skyregion_groups,
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast_pipeline/pipeline/association.py", line 1276, in parallel_association
    ).compute(n_workers=n_cpu, scheduler='processes')
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/base.py", line 166, in compute
    (result,) = compute(self, traverse=False, **kwargs)
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/base.py", line 437, in compute
    results = schedule(dsk, keys, **kwargs)
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/multiprocessing.py", line 222, in get
    **kwargs
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/local.py", line 486, in get_async
    raise_exception(exc, tb)
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/local.py", line 316, in reraise
    raise exc
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/local.py", line 222, in execute_task
    result = _execute_task(task, data)
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/core.py", line 121, in _execute_task
    return func(*(_execute_task(a, cache) for a in args))
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/optimization.py", line 991, in __call__
    return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/core.py", line 151, in get
    result = _execute_task(task, cache)
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/core.py", line 121, in _execute_task
    return func(*(_execute_task(a, cache) for a in args))
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/utils.py", line 30, in apply
    return func(*args, **kwargs)
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/dataframe/core.py", line 5140, in apply_and_enforce
    check_matching_columns(meta, df)
  File "/home/srg/Documents/Projects/ASKAP/askap-pipe/vast-pipeline/vast-pipe-env/lib/python3.7/site-packages/dask/dataframe/utils.py", line 681, in check_matching_columns
    "The columns in the computed data do not match"
ValueError: The columns in the computed data do not match the columns in the provided metadata
  Extra:   ['flux_int_isl_ratio', 'flux_peak_isl_ratio']
  Missing: []
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants