gh-109534: switch from sock_call to sock_call_ex in sock_send #114311
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
While writing webcrawlers with https://github.com/sonic182/aiosonic I came across the issues best described in #109534
I had the great idea of explicitly deleting the event loop my crawler was running on, and hey - I got a traceback of the leaks, repeated ad nauseam:
Notice this traceback happens with Python <= 3.10 only. Python >= 3.11 leaks forever regardless of whether the loop is deleted or not.
That got me thinking, there must be something special about the socket.send() function that isn't raising the exception... so I went looking in socketmodules.c, and lo and behold, there are multiple instances in that file where we use the sock_call() thin wrapper around sock_call_ex().
The problem is that wrapper does not set an err pointer to pass to sock_call_ex(). As a result, the exception may never be raised.
As I said, multiple instances here, this one-liner won't solve #109534 by itself but I believe it's a step forward. I have a crawler running much more stabler now with this fix, particularly when I have to abort() the underlying SSL transport.