Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running pytest on a file importing pandas and torch.onnx leads to access violation #12728

Closed
4 tasks done
anates opened this issue Aug 21, 2024 · 5 comments
Closed
4 tasks done

Running pytest on a file importing pandas and torch.onnx leads to access violation #12728

anates opened this issue Aug 21, 2024 · 5 comments
Labels
status: needs information reporter needs to provide more information; can be closed after 2 or more weeks of inactivity

Comments

@anates
Copy link

anates commented Aug 21, 2024

  • a detailed description of the bug or problem you are having

    While running pytest on a file containing import of both torch.onnx and pandas, pytest crashes with an access violation.

  • output of pip list from the virtual environment you are using

absl-py                       2.1.0
alabaster                     0.7.16
anyio                         4.4.0
argon2-cffi                   23.1.0
argon2-cffi-bindings          21.2.0
arrow                         1.3.0
astroid                       3.2.4
asttokens                     2.4.1
async-lru                     2.0.4
attrs                         24.2.0
audioread                     3.0.1
babel                         2.16.0
beautifulsoup4                4.12.3
black                         24.8.0
bleach                        6.1.0
certifi                       2024.7.4
cffi                          1.17.0
cfgv                          3.4.0
charset-normalizer            3.3.2
click                         8.1.7
cloudpickle                   3.0.0
colorama                      0.4.6
coloredlogs                   15.0.1
comm                          0.2.2
contourpy                     1.2.1
coverage                      7.6.1
cycler                        0.12.1
dask                          2024.8.0
dask-expr                     1.1.10
debugpy                       1.8.5
decorator                     5.1.1
defusedxml                    0.7.1
dill                          0.3.8
distlib                       0.3.8
docutils                      0.21.2
entrypoints                   0.4
executing                     2.0.1
fastjsonschema                2.20.0
filelock                      3.15.4
flatbuffers                   24.3.25
fonttools                     4.53.1
fqdn                          1.5.1
fsspec                        2024.6.1
gitdb                         4.0.11
GitPython                     3.1.43
grpcio                        1.65.5
h11                           0.14.0
h5py                          3.11.0
httpcore                      1.0.5
httpx                         0.27.0
humanfriendly                 10.0
identify                      2.6.0
idna                          3.7
imagesize                     1.4.1
imbalanced-learn              0.11.0
importchecker                 2.0
importlib_metadata            8.3.0
iniconfig                     2.0.0
interchange                   2021.0.4
ipykernel                     6.29.5
ipython                       8.18.1
ipython-genutils              0.2.0
ipywidgets                    8.1.3
isoduration                   20.11.0
isort                         5.13.2
jedi                          0.19.1
Jinja2                        3.1.4
joblib                        1.4.2
json5                         0.9.25
jsonpointer                   3.0.0
jsonschema                    4.23.0
jsonschema-specifications     2023.12.1
jupyter                       1.0.0
jupyter_client                8.6.2
jupyter-console               6.6.3
jupyter_core                  5.7.2
jupyter-events                0.10.0
jupyter-lsp                   2.2.5
jupyter_server                2.14.2
jupyter_server_terminals      0.5.3
jupyterlab                    4.2.4
jupyterlab_pygments           0.3.0
jupyterlab_server             2.27.3
jupyterlab_widgets            3.0.11
kiwisolver                    1.4.5
lazy_loader                   0.4
librosa                       0.10.2.post1
llvmlite                      0.41.1
locket                        1.0.0
Markdown                      3.7
MarkupSafe                    2.1.5
matplotlib                    3.9.2
matplotlib-inline             0.1.7
mccabe                        0.7.0
mistune                       3.0.2
monotonic                     1.6
mpmath                        1.3.0
msgpack                       1.0.8
mypy                          1.11.1
mypy-extensions               1.0.0
nbclassic                     1.1.0
nbclient                      0.10.0
nbconvert                     7.16.4
nbformat                      5.10.4
nest-asyncio                  1.6.0
networkx                      3.2.1
nodeenv                       1.9.1
nodejs                        0.1.1
notebook                      7.2.1
notebook_shim                 0.2.4
npTDMS                        1.9.0
Nuitka                        2.4.7
numba                         0.58.1
numpy                         1.26.4
onnx                          1.16.2
onnxconverter-common          1.13.0
onnxmltools                   1.12.0
onnxruntime                   1.19.0
optional-django               0.1.0
ordered-set                   4.1.0
overrides                     7.7.0
packaging                     24.1
pandarallel                   1.6.5
pandas                        2.2.2
pandocfilters                 1.5.1
pansi                         2020.7.3
parso                         0.8.4
partd                         1.4.2
pathspec                      0.12.1
patsy                         0.5.6
pickleshare                   0.7.5
pillow                        10.4.0
pip                           23.3.1
platformdirs                  4.2.2
pluggy                        1.5.0
pooch                         1.8.2
pre-commit                    3.8.0
prometheus_client             0.20.0
prompt_toolkit                3.0.47
protobuf                      3.20.3
psutil                        5.9.8
pure_eval                     0.2.3
py2neo                        2021.2.4
pyarrow                       14.0.2
pycparser                     2.22
pydmd                         1.0.0
pyflakes                      2.5.0
Pygments                      2.18.0
pylint                        3.2.6
pyoxidizer                    0.22.0
pyparsing                     3.1.2
pyqtgraph                     0.13.7
pyreadline3                   3.4.1
PySide6                       6.7.2
PySide6_Addons                6.7.2
PySide6_Essentials            6.7.2
pytest                        7.4.4
pytest-cov                    5.0.0
pytest-regtest                2.1.1
python-dateutil               2.9.0.post0
python-json-logger            2.0.7
pytz                          2024.1
PyWavelets                    1.6.0
pywin32                       306
pywinpty                      2.0.13
PyYAML                        6.0.2
pyzmq                         26.1.1
qtconsole                     5.5.2
QtPy                          2.4.1
referencing                   0.35.1
requests                      2.32.3
rfc3339-validator             0.1.4
rfc3986-validator             0.1.1
rpds-py                       0.20.0
scikit-base                   0.8.2
scikit-learn                  1.5.1
scipy                         1.13.1
seaborn                       0.13.2
Send2Trash                    1.8.3
setuptools                    69.5.1
shap                          0.46.0
shiboken6                     6.7.2
six                           1.16.0
skl2onnx                      1.17.0
sktime                        0.32.1
slicer                        0.0.8
smmap                         5.0.1
snakeviz                      2.2.0
sniffio                       1.3.1
snowballstemmer               2.2.0
soundfile                     0.12.1
soupsieve                     2.6
soxr                          0.4.0
Sphinx                        7.4.7
sphinxcontrib-applehelp       2.0.0
sphinxcontrib-devhelp         2.0.0
sphinxcontrib-htmlhelp        2.1.0
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          2.0.0
sphinxcontrib-serializinghtml 2.0.0
stack-data                    0.6.3
statistics                    1.0.3.5
statsmodels                   0.14.2
stumpy                        1.13.0
sympy                         1.13.2
tensorboard                   2.17.1
tensorboard-data-server       0.7.2
terminado                     0.18.1
threadpoolctl                 3.5.0
tikzplotlib                   0.10.1
tinycss2                      1.3.0
tomlkit                       0.13.2
toolz                         0.12.1
torch                         2.4.0+cu121
torchaudio                    2.4.0+cu121
torchvision                   0.19.0+cu121
tornado                       6.4.1
tqdm                          4.66.5
traitlets                     5.14.3
tsfresh                       0.20.3
types-python-dateutil         2.9.0.20240316
typing_extensions             4.12.2
tzdata                        2024.1
uri-template                  1.3.0
urllib3                       2.2.2
virtualenv                    20.26.3
wcwidth                       0.2.13
webcolors                     24.8.0
webencodings                  0.5.1
websocket-client              1.8.0
Werkzeug                      3.0.3
widgetsnbextension            4.0.11
xgboost                       1.7.6
zipp                          3.20.0
zstandard                     0.23.0

  • pytest and operating system versions

  • pytest: Version 7.4.4

  • Windows: Win 11 x64

  • minimal example if possible

Test code:

#
# Created on Wed Aug 21 2024
#
"""_summary_
"""

import pandas as pd

import torch.onnx


def test_demo_case() -> None:
    """_summary_"""
    print("Hello from test case")


if __name__ == "__main__":
    test_demo_case()
    print("Hello World")

Execution with pytest demo_case.py results in

============================================================================================ test session starts ============================================================================================
platform win32 -- Python 3.11.9, pytest-7.4.4, pluggy-1.5.0
rootdir: C:\Users\user_name\Documents\Python Scripts\TARGET_project
configfile: pyproject.toml
plugins: anyio-4.4.0, cov-5.0.0, regtest-2.1.1
collecting ... Windows fatal exception: access violation

Current thread 0x00009614 (most recent call first):
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1233 in create_module
  File "<frozen importlib._bootstrap>", line 573 in module_from_spec
  File "<frozen importlib._bootstrap>", line 676 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\onnx\__init__.py", line 77 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\torch\onnx\_internal\onnxruntime.py", line 36 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\torch\onnx\__init__.py", line 62 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "C:\Users\user_name\Documents\Python Scripts\\tests\demo_cases\demo_testcase_onnx.py", line 14 in <module>
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\assertion\rewrite.py", line 186 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1204 in _gcd_import
  File "C:\Program Files\Python311\Lib\importlib\__init__.py", line 126 in import_module
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\pathlib.py", line 567 in import_path
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\python.py", line 617 in _importtestmodule
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\python.py", line 528 in _getobj
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\python.py", line 310 in obj
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\python.py", line 545 in _inject_setup_module_fixture
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\python.py", line 531 in collect
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\runner.py", line 372 in <lambda>
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\runner.py", line 341 in from_call
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\runner.py", line 372 in pytest_make_collect_report
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\pluggy\_callers.py", line 103 in _multicall
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\pluggy\_manager.py", line 120 in _hookexec
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\pluggy\_hooks.py", line 513 in __call__
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\runner.py", line 547 in collect_one_node
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\main.py", line 842 in genitems
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\main.py", line 675 in perform_collect
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\main.py", line 335 in pytest_collection
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\pluggy\_callers.py", line 103 in _multicall
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\pluggy\_manager.py", line 120 in _hookexec
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\pluggy\_hooks.py", line 513 in __call__
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\main.py", line 324 in _main
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\main.py", line 271 in wrap_session
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\main.py", line 318 in pytest_cmdline_main
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\pluggy\_callers.py", line 103 in _multicall
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\pluggy\_manager.py", line 120 in _hookexec
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\pluggy\_hooks.py", line 513 in __call__
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\config\__init__.py", line 169 in main
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Lib\site-packages\_pytest\config\__init__.py", line 192 in console_main
  File "C:\Users\user_name\AppData\Local\pypoetry\Cache\virtualenvs\target_project-MV-Np-p2-py3.11\Scripts\pytest.exe\__main__.py", line 7 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main
@RonnyPfannschmidt
Copy link
Member

More details needed

Pytest running most likely causes a different memory layout and triggers the bug in the other libraries

@anates
Copy link
Author

anates commented Aug 21, 2024

Which details would you need in addition? Thanks!

@RonnyPfannschmidt
Copy link
Member

I don't have windows available, please try different versions of panda and torch as starting point

@anates
Copy link
Author

anates commented Aug 21, 2024

Will do as soon as I have time for that, thanks!

@Zac-HD Zac-HD added the status: needs information reporter needs to provide more information; can be closed after 2 or more weeks of inactivity label Oct 30, 2024
@Zac-HD
Copy link
Member

Zac-HD commented Oct 30, 2024

Closing this because without any native code in pytest this is certain to be a bug in one (or several) other libraries.

@Zac-HD Zac-HD closed this as completed Oct 30, 2024
@DanielYang59 DanielYang59 mentioned this issue Feb 14, 2025
Draft
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
status: needs information reporter needs to provide more information; can be closed after 2 or more weeks of inactivity
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@RonnyPfannschmidt @Zac-HD @anates