Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

anyio.Path.iterdir blocks in Python 3.13 #873

Closed
2 tasks done
cbornet opened this issue Feb 20, 2025 · 1 comment · Fixed by #874
Closed
2 tasks done

anyio.Path.iterdir blocks in Python 3.13 #873

cbornet opened this issue Feb 20, 2025 · 1 comment · Fixed by #874
Labels
bug Something isn't working

Comments

@cbornet
Copy link
Contributor

cbornet commented Feb 20, 2025

Things to check first

  • I have searched the existing issues and didn't find my bug already reported there

  • I have checked that my bug is still present in the latest release

AnyIO version

HEAD

Python version

3.13

What happened?

Python 3.13 introduced a breaking change on how pathlib.Path.iterdir behaves.
iterdir now materializes the list of files eagerly. See python/cpython#129871 .
The reason for this change was to raise OS errors directly and not at the first iteration : python/cpython#78722

anyio.Path.iterdir is calling pathlib.Path.iterdir directly (not in a thread)

gen = self._path.iterdir()

and probably users would expect it not to block.

Some possibilities to fix would be:

  • Reimplement anyio.Path.iterdir to not use pathlib.Path.iterdir and use os.scandir directly without materializing the iterator. In this case, there will be a small difference with pathlib.Path.iterdir that OS errors will only happen when we start iterating.
  • Make anyio.Path.iterdir async which is a breaking change.

LMK what you think. I can do a PR.

How can we reproduce the bug?

N/A

@cbornet cbornet added the bug Something isn't working label Feb 20, 2025
@agronholm
Copy link
Owner

I would prefer not to introduce any breaking changes, but instead reimplement the async iterator in a way that defers any I/O until the first iteration, like how it used to work.

cbornet added a commit to cbornet/anyio that referenced this issue Feb 20, 2025
cbornet added a commit to cbornet/anyio that referenced this issue Feb 20, 2025
cbornet added a commit to cbornet/anyio that referenced this issue Feb 20, 2025
cbornet added a commit to cbornet/anyio that referenced this issue Feb 20, 2025
agronholm added a commit that referenced this issue Feb 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants