[BUG]: 2.11.0 regression on PyPy3: pybind11::handle::dec_ref() is being called while the GIL is either not held or invalid. while running tests
#4748
Comments
|
bisect blames the following commit:
|
|
FWICS you don't have to actually run any tests to trigger it — it's sufficient to import Apparently the problem occurs when destructors are called while the interpreter is exiting: |
mgorny
changed the title
pybind11::handle::dec_ref() is being called while the GIL is either not held or invalid. while running testspybind11::handle::dec_ref() is being called while the GIL is either not held or invalid. while running tests
|
I have also reproduced this with PyPy3.9 (7.3.12 also). |
|
This bug also exists in cpython 3.8 with pybind 2.11.1, with following bind functions: void BoxOps::transform_pc(pybind11::array_t<float> out_points, pybind11::array_t<float> points, pybind11::array_t<float> mat) {
using DType = float;
auto N = points.shape(0);
auto out_points_data = out_points.mutable_data();
auto points_data = points.mutable_data();
auto stride = points.shape(1);
DType* out_ptr, *ptr;
DType x, y, z;
DType m00 = mat.at(0, 0);
DType m01 = mat.at(0, 1);
DType m02 = mat.at(0, 2);
DType m10 = mat.at(1, 0);
DType m11 = mat.at(1, 1);
DType m12 = mat.at(1, 2);
DType m20 = mat.at(2, 0);
DType m21 = mat.at(2, 1);
DType m22 = mat.at(2, 2);
if (mat.shape(0) == 3 && mat.shape(1) == 3){
for (size_t i = 0; i < N; ++i){
out_ptr = out_points_data + i * stride;
ptr = points_data + i * stride;
x = ptr[0];
y = ptr[1];
z = ptr[2];
out_ptr[0] = m00 * x + m01 * y + m02 * z;
out_ptr[1] = m10 * x + m11 * y + m12 * z;
out_ptr[2] = m20 * x + m21 * y + m22 * z;
}
}else{
DType m03 = mat.at(0, 3);
DType m13 = mat.at(1, 3);
DType m23 = mat.at(2, 3);
for (size_t i = 0; i < N; ++i){
out_ptr = out_points_data + i * stride;
ptr = points_data + i * stride;
x = ptr[0];
y = ptr[1];
z = ptr[2];
out_ptr[0] = m00 * x + m01 * y + m02 * z + m03;
out_ptr[1] = m10 * x + m11 * y + m12 * z + m13;
out_ptr[2] = m20 * x + m21 * y + m22 * z + m23;
}
}
}
module_BoxOps.def_static("transform_pc", &csrc::boxops::BoxOps::transform_pc, pybind11::arg("out_points"), pybind11::arg("points"), pybind11::arg("mat"), pybind11::return_value_policy::automatic, pybind11::call_guard<pybind11::gil_scoped_release>());
@rwgk |
Yes, almost certainly.
Not like your doing it right now (assuming you don't want undefined behavior). You have to be more careful: you have to ensure you're holding the GIL again before the destructors run. This is a classical trap that is very easy to fall into (I had a couple myself before I added the guards). The fix is probably subtle. Maybe (!), don't release the GIL with Inside the subscope you cannot have any code calling back into the Python C API. The simple guards we have just cover the most common oversights. |
Required prerequisites
What version (or hash if on master) of pybind11 are you using?
2.11.0
Problem description
At the very end of pybind11's test suite I'm getting the following exception:
This is with PyPy3.10 7.3.12. In pybind11 2.10.4, the test suite passes without any exceptions.
Full build/test log: dev-python:pybind11-2.11.0:20230715-024418.log
I have to leave now,, I can investigate further later.
Reproducible example code
Is this a regression? Put the last known working version here if it is.
2.10.4
The text was updated successfully, but these errors were encountered: