Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix unicode/str inconsistency in Python 2 #177

Merged
merged 1 commit into from
May 12, 2019

Conversation

bbc2
Copy link
Collaborator

@bbc2 bbc2 commented May 6, 2019

This encodes environment keys and values for os.environ in Python 2
and fixes the following issues:

  • Values loaded from .env being unicode whereas regular values being
    str (i.e. bytes) in Python 2.
  • Error when loading .env with UTF-8 characters in Python 2.
  • # type: ignore hiding the issue from the type checker.

sys.getfilesystemencoding() is used because it seems to match the
encoding used for environment variables with Python 2. Since it can
apparently return None, we fall back to a default encoding (UTF-8
here) if that's the case.

I tested this on Windows where "mbcs" is the file system encoding.
Hopefully, this will also work on other systems.

Closes #121. Closes #176.

This encodes environment keys and values for `os.environ` in Python 2
and fixes the following issues:

* Values loaded from `.env` being `unicode` whereas regular values being
  `str` (i.e. `bytes`) in Python 2.
* Error when loading `.env` with UTF-8 characters in Python 2.
* `# type: ignore` hiding the issue from the type checker.

`sys.getfilesystemencoding()` is used because it seems to match the
encoding used for environment variables with Python 2.  Since it can
apparently return `None`, we fall back to a default encoding (UTF-8
here) if that's the case.

I tested this on Windows where `"mbcs"` is the file system encoding.
Hopefully, this will also work on other systems.
@bbc2 bbc2 requested a review from theskumar May 6, 2019 17:19
@coveralls
Copy link

Coverage Status

Coverage increased (+0.9%) to 88.737% when pulling 45f697a on bbc2:fix-str-unicode-env-python-2 into c50c6f7 on theskumar:master.

4 similar comments
@coveralls
Copy link

Coverage Status

Coverage increased (+0.9%) to 88.737% when pulling 45f697a on bbc2:fix-str-unicode-env-python-2 into c50c6f7 on theskumar:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.9%) to 88.737% when pulling 45f697a on bbc2:fix-str-unicode-env-python-2 into c50c6f7 on theskumar:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.9%) to 88.737% when pulling 45f697a on bbc2:fix-str-unicode-env-python-2 into c50c6f7 on theskumar:master.

@coveralls
Copy link

Coverage Status

Coverage increased (+0.9%) to 88.737% when pulling 45f697a on bbc2:fix-str-unicode-env-python-2 into c50c6f7 on theskumar:master.

@theskumar theskumar merged commit 62df65a into theskumar:master May 12, 2019
@theskumar
Copy link
Owner

thanks this looks great!

theskumar pushed a commit that referenced this pull request Jun 2, 2019
* master: (49 commits)
  Fix typo in README.md (#181)
  Refactor move run_command in cli
  Refractor: move 'to_env' to compat.py
  Refactor parser to fix inconsistencies (#180)
  chore(pypi): switch to markdown and twine for pypi upload + update supported version
  Bump version: 0.10.1 → 0.10.2
  readme: Add new release stub
  Fix unicode/str inconsistency in Python 2 (#177)
  Add argument to choose .env file encoding, defaults to None
  Fix links in readme
  Restrict typing dependency to Python < 3.5
  Add type hints and expose them to users
  Updated `.gitignore` with results from https://www.gitignore.io/api/python
  Added special case for `__file__` in Python 2.  Fixes #130 Updated tests to show that this works.
  Moved `dotenv` package into `src` directory so that tests run against installed version. Added tox to run test suite against multiple versions of Python (with coverage) and run flake8. Updated Travis CI to use tox as well. Updated documentation about running tests.
  Fix ResourceWarning: unclosed file in setup.py and test_cli.py
  Clarify the usuages of export
  Isolate test files (#160)
  Deleted some temporary files that CLI tests were creating.
  Add python 3.7 to testsuite + udpate pypi creds
  ...
@bbc2 bbc2 deleted the fix-str-unicode-env-python-2 branch February 16, 2020 10:13
johnbergvall pushed a commit to johnbergvall/python-dotenv that referenced this pull request Aug 13, 2021
This encodes environment keys and values for `os.environ` in Python 2
and fixes the following issues:

* Values loaded from `.env` being `unicode` whereas regular values being
  `str` (i.e. `bytes`) in Python 2.
* Error when loading `.env` with UTF-8 characters in Python 2.
* `# type: ignore` hiding the issue from the type checker.

`sys.getfilesystemencoding()` is used because it seems to match the
encoding used for environment variables with Python 2.  Since it can
apparently return `None`, we fall back to a default encoding (UTF-8
here) if that's the case.

I tested this on Windows where `"mbcs"` is the file system encoding.
Hopefully, this will also work on other systems.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants