Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't handle binary data in output streams #346

Closed
natefoo opened this issue Nov 22, 2023 · 0 comments · Fixed by #347
Closed

Can't handle binary data in output streams #346

natefoo opened this issue Nov 22, 2023 · 0 comments · Fixed by #347

Comments

@natefoo
Copy link
Member

natefoo commented Nov 22, 2023

e.g. someone ran a zip file through cutadapt, so the tool stderr contains:

Output will be written into the directory: /jetstream2/scratch/main/jobs/53509211/working/
File seems to be in SOLiD colorspace format which is not supported by Trim Galore (sequence is: '�Vn)}}%u`:�ᰱ��g/߶m
�)! Please use Cutadapt on colorspace files separately and check its documentation!                               o�<?v�yԩ(A��ФZ�5_hf/���/��η��mgz8;���6xv�M��=c7�z͛_Z�L������^���u]W���7|̍ߦ~��KF2-��b

This results in the following from Pulsar:

Exception in thread [manager=jetstream2]-[action=postprocess]-[job=53509211]:
  File "/usr/lib64/python3.9/threading.py", line 980, in _bootstrap_inner
    self.run()
  File "/srv/pulsar/main/venv/lib64/python3.9/site-packages/sentry_sdk/integrations/threading.py", line 72, in run
    reraise(*_capture_exception())
  File "/srv/pulsar/main/venv/lib64/python3.9/site-packages/sentry_sdk/_compat.py", line 115, in reraise
    raise value
  File "/srv/pulsar/main/venv/lib64/python3.9/site-packages/sentry_sdk/integrations/threading.py", line 70, in run
    return old_run_func(self, *a, **kw)
  File "/usr/lib64/python3.9/threading.py", line 917, in run
    self._target(*self._args, **self._kwargs)
  File "/srv/pulsar/main/venv/lib64/python3.9/site-packages/pulsar/managers/stateful.py", line 229, in do_postprocess
    self.__state_change_callback(final_status, job_id)
  File "/srv/pulsar/main/venv/lib64/python3.9/site-packages/pulsar/messaging/bind_amqp.py", line 73, in bind_on_status_change
    payload = manager_endpoint_util.full_status(manager, new_status, job_id)
  File "/srv/pulsar/main/venv/lib64/python3.9/site-packages/pulsar/manager_endpoint_util.py", line 26, in full_status
    full_status = __job_complete_dict(job_status, manager, job_id)
  File "/srv/pulsar/main/venv/lib64/python3.9/site-packages/pulsar/manager_endpoint_util.py", line 40, in __job_complete_dict
    stderr_contents = manager.stderr_contents(job_id).decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd1 in position 603: invalid continuation byte

This also results in a permanently non-terminal job.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant