-
-
Notifications
You must be signed in to change notification settings - Fork 31.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deadlock when calling PyGILState_Ensure() from a fresh C thread #96071
Comments
Do you have a self-contained example that demonstrates the problem? I quickly checked using PyObjC on macOS (which basically does this when starting threads using the Cocoa interface), and that code did not deadlock. |
Ok, I investigated a little deeper and admittedly I have a bit more going on in the environment where I found this. Tried duplicating until I found the difference: |
I can reproduce using PyObjC by using @pablogsal is this a release blocker? |
Indeed it is. Thanks for pinging me. |
@ronaldoussoren Can you share the reproducer? |
Or alternatively, could you bisect the issue? |
In previous versions the memory for the new |
The reproducer needs PyObjC (from v8.5-branch in https://github.com/ronaldoussoren/pyobjc because the latest release does not work with Python 3.11 yet). The reproducer script:
This script hangs when using "python3.11 -X tracemalloc script.py It is probably easier to reproduce this using a small C extension that uses I can look into this during the weekend at the earliest, but also need to get a release of PyObjC out to fix Python 3.11 support (which is needed due to documented API changes, nothing to worry about there). |
I will prepare a patch tomorrow. I have enough information to produce a fix. |
Minimal reproducer: #include "Python.h"
#include "pthread.h"
static struct PyModuleDef deadlockmodule = {
PyModuleDef_HEAD_INIT,
"deadlock",
NULL,
-1,
NULL,
NULL,
NULL,
NULL,
NULL
};
static void *func(void *arg)
{
printf("in func\n");
PyGILState_STATE state = PyGILState_Ensure();
printf("called PyGILState_Ensure\n");
PyGILState_Release(state);
printf("called PyGILState_Release\n");
return NULL;
}
PyMODINIT_FUNC
PyInit_deadlock(void)
{
Py_BEGIN_ALLOW_THREADS
pthread_t thread;
pthread_create(&thread, NULL, func, NULL);
pthread_join(thread, NULL);
Py_END_ALLOW_THREADS
return PyModule_Create(&deadlockmodule);
} |
@pablogsal I have a fix for this #96107 |
Thanks for the eyes on this, everyone, and for the fixes. |
Alternative of pythonGH-96107 (cherry picked from commit e0d54a4) Co-authored-by: Kumar Aditya <[email protected]>
I've merged the fix into main. Now I'm wondering if it would make sense to also add a test that would have caught the bug. That wouldn't be too hard, would it @kumaraditya303? |
Alternative of GH-96107 (cherry picked from commit e0d54a4) Co-authored-by: Kumar Aditya <[email protected]> Co-authored-by: Kumar Aditya <[email protected]>
PR #96137 adds a test for this. |
Automerge-Triggered-By: GH:ericsnowcurrently (cherry picked from commit 079baee) Co-authored-by: Kumar Aditya <[email protected]>
Automerge-Triggered-By: GH:ericsnowcurrently (cherry picked from commit 079baee) Co-authored-by: Kumar Aditya <[email protected]>
…GH-96137) Automerge-Triggered-By: GH:ericsnowcurrently (cherry picked from commit 079baee) Co-authored-by: Kumar Aditya <[email protected]>
…GH-96137) Automerge-Triggered-By: GH:ericsnowcurrently (cherry picked from commit 079baee) Co-authored-by: Kumar Aditya <[email protected]>
Automerge-Triggered-By: GH:ericsnowcurrently (cherry picked from commit 079baee) Co-authored-by: Kumar Aditya <[email protected]>
Closing as I have fixed both the issue and added tests to ensure no further regression. Thanks! |
Thanks, @kumaraditya303! |
Automerge-Triggered-By: GH:ericsnowcurrently
Does deadlock. Seems this could be related to python/cpython#96071
…yInterpreterState Field (pythongh-126989) This approach eliminates the originally reported race. It also gets rid of the deadlock reported in pythongh-96071, so we can remove the workaround added then.
…PyInterpreterState Field (gh-127114) This approach eliminates the originally reported race. It also gets rid of the deadlock reported in gh-96071, so we can remove the workaround added then. This is mostly a cherry-pick of 1c0a104 (AKA gh-126989). The difference is we add PyInterpreterState.threads_preallocated at the end of PyInterpreterState, instead of adding PyInterpreterState.threads.preallocated. That avoids ABI disruption.
…yInterpreterState Field (pythongh-126989) This approach eliminates the originally reported race. It also gets rid of the deadlock reported in pythongh-96071, so we can remove the workaround added then.
Bug report
This used to work with py 3.8, 3.9 and 3.10 but now fails. The failure is due to a deadlock condition due to the move of an alloc from before a HEAD_LOCK() to after causing a recursive call to PyGILState_Ensure() which eventually deadlocks on the HEAD_LOCK().
The following is a trace of calls from the initial call to PyGILState_Ensure() to the eventual deadlock:
Steps to reproduce: In a Python C extension module, create a C thread then try to call PyGILState_Ensure() in that thread with no previous existing Python threadstate.
Your environment
Python 3.11.0rc1
Linux tom-VirtualBox 5.15.0-46-generic #49~20.04.1-Ubuntu SMP Thu Aug 4 19:15:44 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
The text was updated successfully, but these errors were encountered: