Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

always write utf-8 documents #71

Closed
davemfish opened this issue Jan 30, 2025 · 0 comments · Fixed by #72
Closed

always write utf-8 documents #71

davemfish opened this issue Jan 30, 2025 · 0 comments · Fixed by #72

Comments

@davemfish
Copy link
Collaborator

davemfish commented Jan 30, 2025

Without specifying an encoding, it was possible to write a non-utf-8 encoded document, which then would fail to read back in using fsspec.open:

This was triggered by including a string such as "m²/person"

(C:\Users\dmf\projects\invest\env-py313) λ geometamaker validate admin_boundaries.gpkg.yml
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\dmf\projects\invest\env-py313\Scripts\geometamaker.exe\__main__.py", line 7, in <module>
    sys.exit(cli())
             ~~~^^
  File "C:\Users\dmf\projects\invest\env-py313\Lib\site-packages\click\core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "C:\Users\dmf\projects\invest\env-py313\Lib\site-packages\click\core.py", line 1082, in main
    rv = self.invoke(ctx)
  File "C:\Users\dmf\projects\invest\env-py313\Lib\site-packages\click\core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "C:\Users\dmf\projects\invest\env-py313\Lib\site-packages\click\core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\dmf\projects\invest\env-py313\Lib\site-packages\click\core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\dmf\projects\geometamaker\src\geometamaker\cli.py", line 81, in validate
    error = geometamaker.validate(filepath)
  File "C:\Users\dmf\projects\geometamaker\src\geometamaker\geometamaker.py", line 431, in validate
    yaml_string = file.read()
  File "<frozen codecs>", line 325, in decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb2 in position 971: invalid start byte
davemfish added a commit to davemfish/geometamaker that referenced this issue Jan 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant