Decode python_eval template resource as utf-8. #6379
Conversation
Without this, the template was rendered with literal `\n`'s under python3. I did not linger to figure out how the rendered string could possibly load in a python interpreter with no errors! Fixes pantsbuild#6354
|
/cc @Eric-Arellano |
|
NB: The error described in #6354 reproduced 100% before this fix and now does not occur. |
There was a problem hiding this comment.
Great find. I've been bitten by pkgutil a couple of other times.
In general, it looks like pkgutil.get_data leads to extremely silent errors, i.e. it doesn't raise errors but results in undesirable behavior.
I have no idea how you debugged this, but thank you so much for finding the issue!
This means contrib CI will be 100% py3 compliant :)
|
The Python 3 CI failures look quite similar to #6384, in that the actual output has more values than expected and in both instances the tests are called |
Got this sussed. Fix forthcoming. I'll probably lump the fix into this RB for expediency but push back if you want it broken out. |
Previously it wrote and read keys asymmetrically; now the key is uniformly represented (and compared!) as a unicode utf-8 string in memory.
| @@ -223,7 +223,7 @@ def _read_sha(self, cache_key): | |||
| def _read_sha_by_id(self, id): | |||
| try: | |||
| with open(self._sha_file_by_id(id), 'rb') as fd: | |||
| return fd.read().strip() | |||
| return fd.read().strip().decode('utf-8') | |||
There was a problem hiding this comment.
Preferred to open in mode 'r', and then you can avoid the decode. Don't worry about fixing and repushing though - https://github.com/pantsbuild/pants/pull/6372/files#diff-0bfcef172d3faad2ba6f85c7330adae2R225 will fix it.
Thanks again John for your help here!
Without this, the template was rendered with literal
\n's underpython3. I did not linger to figure out how the rendered string could
possibly load in a python interpreter with no errors!
Fixes #6354
Fixes #6384