gh-125660: Python implementation of json.loads() accepts invalid unicode escapes

[nineteendo]

This condition in json.decoder is insufficient:

cpython/Lib/json/decoder.py

Line 61 in f203d1c

if len(esc) == 4 and esc[1] not in 'xX':

>>> import sys
>>> sys.modules["_json"] = None
>>> import json
>>> json.loads(r'["\u 000", "\u-000", "\u+000", "\u0_00"]')
['\x00', '\x00', '\x00', '\x00']

• Issue: Python implementation of json.loads() accepts invalid unicode escapes #125660

[ZeroIntensity]

For other reviewers, see also: #125687

I think this is fine, but per @taleinat, this shouldn't get backported--it's technically a very minor breaking change.

[taleinat]

I think this is fine, but per @taleinat, this shouldn't get backported--it's technically a very minor breaking change.

After further consideration, I do consider this to be a clear bug fix, which should be backported to versions in "bugfix" status (i.e. 3.12).

[ZeroIntensity]

Are you sure? I'm not certain what policy PyPy, IronPython, Jython, etc. have for changes that we backport, but point of pure-Python modules was almost exclusively for other Python implementations--I don't want to accidentally cause any surprises.

[taleinat]

Are you sure?

No, I am not sure, I've been away from Python development for several years. It would be good to get another core dev's perspective.

[taleinat]

Upon further consideration, yes, the chance of breaking something with a backport is large relative to the likely pain caused by the current, buggy, state. Let's avoid backporting.

[serhiy-storchaka]

LGTM. 👍

[serhiy-storchaka]

This is clearly a bug. json.loads() can be a part of validation. If it accepts an invalid JSON, it can fail later, in other program with more strict JSON parser. I do not think that anybody depends on such buggy behavior.

[miss-islington-app]

Thanks @nineteendo for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

[bedevere-app]

GH-125694 is a backport of this pull request to the 3.13 branch.

[bedevere-app]

GH-125695 is a backport of this pull request to the 3.12 branch.