Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Access violation with pytest >= 5.0.0 and windows 10 #7634

Closed
ghost opened this issue Aug 10, 2020 · 13 comments
Closed

Access violation with pytest >= 5.0.0 and windows 10 #7634

ghost opened this issue Aug 10, 2020 · 13 comments
Labels
platform: windows windows platform-specific problem status: help wanted developers would like help from experts on this topic type: regression indicates a problem that was introduced in a release which was working previously

Comments

@ghost
Copy link

ghost commented Aug 10, 2020

A detailed description of the bug or suggestion:

So, I used to run tests on windows 10 using pytest 3.x.x and custom plugins but I recently upgraded to pytest 5.x.x.
Since then, each time I run a test featuring a call to a DLL using pythonnet 2.5.1, I get a generic windows access violation error.
Even though I found similar cases online, each time accepted fix seemed to be to downgrade pytest - which I do not want to do, as I rather like the new functionalities.

Issue was reproduced with every pytest version above 5.0.0, including 6.0.0+.
Versions under 4.6.11 did not allow me to reproduce the error.

Output of pip list from the virtual environment you are using:

my privatevenv pip list:
Package Version


atomicwrites 1.4.0
attrs 19.3.0
colorama 0.4.3
comtypes 1.1.7
importlib-metadata 1.7.0
iniconfig 1.0.1
more-itertools 8.4.0
packaging 20.4
pip 19.0.3
pluggy 0.13.1
py 1.9.0
pycparser 2.20
pyparsing 2.4.7
pytest 5.4.3
pythonnet 2.5.1
pywin32 228
pywinauto 0.6.8
setuptools 40.8.0
six 1.15.0
toml 0.10.1
wcwidth 0.2.5
zipp 3.1.0

pytest and operating system versions:

platform win10 x64 -- Python 3.7.4 32 bits, pytest-5.3.4, py-1.9.0, pluggy-0.13.1

Minimal example if possible:

here is my code for test.py:

import clr
import pytest


class TestClass:
    def test_function(self):
        clr.AddReference(r"C:\path\to\my\privateDll.dll")
        from privateDll import privateDllClass
        myDllObject = privateDllClass()
        myDllObject.LoadFile(r"C:\path\to\my\file.xml")
        myDllObject.function(12, False)

and the output I get when executing "pytest test.py":

C:\my\private\path\mytmptestDir>pytest test.py
============================================================================ test session starts ============================================================================
platform win32 -- Python 3.7.4, pytest-5.3.4, py-1.9.0, pluggy-0.13.1
rootdir: C:\my\private\path\mytmptestDir
collected 1 item                                                                                                                                                             

test.py Windows fatal exception: access violation

Current thread 0x000c69bc (most recent call first):
  File "C:\my\private\path\mytmptestDir\test.py", line 15 in test_function
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\python.py", line 167 in pytest_pyfunc_call
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\python.py", line 1445 in runtest
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\runner.py", line 134 in pytest_runtest_call
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\runner.py", line 210 in <lambda>
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\runner.py", line 237 in from_call
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\runner.py", line 210 in call_runtest_hook
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\runner.py", line 185 in call_and_report
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\runner.py", line 99 in runtestprotocol
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\runner.py", line 84 in pytest_runtest_protocol
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\main.py", line 271 in pytest_runtestloop
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\main.py", line 247 in _main
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\main.py", line 197 in wrap_session
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\main.py", line 240 in pytest_cmdline_main
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\my\private\path\privatevenv\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\my\private\path\privatevenv\lib\site-packages\_pytest\config\__init__.py", line 93 in main
  File "C:\my\private\path\privatevenv\Scripts\pytest.exe\__main__.py", line 9 in <module>
  File "C:\Python37-32\lib\runpy.py", line 85 in _run_code
  File "C:\Python37-32\lib\runpy.py", line 193 in _run_module_as_main
.                                                                                                                                                               [100%]

============================================================================= warnings summary ==============================================================================
test.py:14
  C:\my\private\path\mytmptestDir\test.py:14: DeprecationWarning: invalid escape sequence \S
    mobj.LoadATMLFile("C:\path\to\my\file.xml")

-- Docs: https://docs.pytest.org/en/latest/warnings.html
======================================================================= 1 passed, 1 warning in 0.50s ========================================================================

As an additional question, is there a better way or methodology to troubleshoot this one ?

Thanks.

@Zac-HD Zac-HD added platform: windows windows platform-specific problem status: help wanted developers would like help from experts on this topic type: regression indicates a problem that was introduced in a release which was working previously labels Aug 10, 2020
@The-Compiler
Copy link
Member

The-Compiler commented Aug 11, 2020

Not much we can do without having a way to reproduce this... However, something that might help to find the issue (whether that's in pytest or in your DLL) is a git bisect. This tells us which commit in pytest exactly introduces the issue, by letting git always test the middle between two commits.

To do so, run something like this (untested):

  • git clone https://github.com/pytest-dev/pytest.git
  • C:\my\private\path\privatevenv\Scripts\pip install -e pytest
  • cd pytest

Now for the following commands, use two different terminals, one where you do the bisect in the pytest repository, and one where you run pytest in your project directory:

  • git checkout 4.6.11
  • Verify that you can run pytest test.py in your project directory and that the issue does not occur (also check if the header shows pytest-4.6.11 like expected)
  • git checkout 5.0.0
  • Verify that the issue does occur (and double check the header)
  • git bisect start
  • git bisect good 4.6.11
  • git bisect bad 5.0.0

Then, git will check out the middle between the two. Test it, and do git bisect good or git bisect bad. If something is broken for unrelated reasons, do git bisect skip. After testing ~10 times, git should present you with the broken commit.

@ghost
Copy link
Author

ghost commented Aug 12, 2020

Hello, and thanks a lot for your help.

Git bisect was able to pinpoint the "broken" commit, which in the end was the following:

 /c/Bailly/github_pytestdev_7634/pytest ((a37b902af...)|BISECTING)
$ git bisect bad
a37b902afea21621639b114f087e84f70fb057ba is the first bad commit
commit a37b902afea21621639b114f087e84f70fb057ba
Author: Bruno Oliveira <nicoddemus@gmail.com>
Date:   Wed Jun 12 18:49:51 2019 -0300

    Integrate pytest-faulthandler into the core

    * Add pytest-faulthandler files unchanged
    * Adapt imports and tests
    * Add code to skip registration of the external `pytest_faulthandler`
      to avoid conflicts

    Fix #5440

 changelog/5440.feature.rst     |   8 ++++
 doc/en/usage.rst               |  19 +++++++-
 src/_pytest/config/__init__.py |   3 +-
 src/_pytest/deprecated.py      |   8 ++++
 src/_pytest/faulthandler.py    | 102 +++++++++++++++++++++++++++++++++++++++++
 testing/deprecated_test.py     |  23 ++++------
 testing/test_faulthandler.py   |  99 +++++++++++++++++++++++++++++++++++++++
 7 files changed, 245 insertions(+), 17 deletions(-)
 create mode 100644 changelog/5440.feature.rst
 create mode 100644 src/_pytest/faulthandler.py
 create mode 100644 testing/test_faulthandler.py

which makes me think it could be related to the issue #7059 and/or #7022.
The nice thing about this is that I now have a workaround, but as stated in the issue #7059 "it would be nice to understand why commit a37b902 causes such a problem".

Thanks in advance for any additional information.

@The-Compiler
Copy link
Member

The faulthandler integration is just what's printing the "Windows fatal exception: access violation" output. If I had to guess, I'd say the access violation was already happening earlier (In a thread, hence why it's not aborting pytest itself?), it's just that nothing ever told you about it?

What happens if you run with pytest 4.6.11 but run python -X faulthandler -m pytest ... instead?

@ghost
Copy link
Author

ghost commented Aug 12, 2020

When running that command, I do not get any unexpected output.


[...]
Successfully installed pytest-4.6.11

(privatevenv) C:\my\private\path>python -X faulthandler -m pytest tmp.py
Test session starts (platform: win32, Python 3.7.4, pytest 4.6.11, pytest-sugar 0.9.3)
rootdir: C:\my\private\path
plugins: cov-2.6.1, html-2.1.1, metadata-1.8.0, modal-3.0, steps-1.7.3, sugar-0.9.3
collecting ... 
 tmp.py ✓                                                                                                                                                      100% ██████████


Results (0.49s):
       1 passed

@RonnyPfannschmidt
Copy link
Member

To me this looks like the memory layout may be part of the trigger for the fault

I suspect an issue in the c library causes the issue, on Linux one would use a memory debugger now, I'm unfamiliar with the tools to use on Windows for such a issue

@The-Compiler
Copy link
Member

Some more things that might be interesting:

  • 4.6.11 and python -X faulthandler -m pytest -s tmp.py (I forgot that pytest would capture the default faulthandler output...)
  • 5.x and pytest -p no:faulthandler tmp.py (I assume that works fine?)
  • 5.x and python -X faulthandler -m pytest -p no:faulthandler -s tmp.py (newer pytest, Python faulthandler instead of pytest's integration)

What about pytest 5.x and -p no:faulthandler? I assume that works? If so, what about python -X faulthandler -m pytest -p no:faulthandler tmp.py?

@ghost
Copy link
Author

ghost commented Aug 12, 2020

outputs :

pytest 4.6.11
python -X faulthandler -m pytest -s tmp.py

python -X faulthandler -m pytest -s tmp.py
Test session starts (platform: win32, Python 3.7.4, pytest 4.6.11, pytest-sugar 0.9.3)
rootdir: C:\Snaps\SnapDev\swtools\soft\mytmptestDir
plugins: cov-2.6.1, html-2.1.1, metadata-1.8.0, modal-3.0, steps-1.7.3, sugar-0.9.3
collecting ... Windows fatal exception: access violation

Current thread 0x00017410 (most recent call first):
  File "C:\Snaps\SnapDev\swtools\soft\mytmptestDir\tmp.py", line 13 in test_function
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\python.py", line 174 in pytest_pyfunc_call
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\python.py", line 1464 in runtest
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 123 in pytest_runtest_call
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 198 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 226 in from_call
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 198 in call_runtest_hook
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 173 in call_and_report
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 93 in runtestprotocol
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 78 in pytest_runtest_protocol
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\main.py", line 271 in pytest_runtestloop
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\main.py", line 250 in _main
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\main.py", line 206 in wrap_session
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\main.py", line 243 in pytest_cmdline_main
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\config\__init__.py", line 84 in main
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pytest.py", line 102 in <module>
  File "C:\Python37-32\Lib\runpy.py", line 85 in _run_code
  File "C:\Python37-32\Lib\runpy.py", line 193 in _run_module_as_main

 tmp.py ✓                                                                                                                                                      100% ██████████


Results (0.61s):
       1 passed

pytest 5.4.3
python -X faulthandler -m pytest -p no:faulthandler -s tmp.py

python -X faulthandler -m pytest -p no:faulthandler -s tmp.py
Test session starts (platform: win32, Python 3.7.4, pytest 5.4.3, pytest-sugar 0.9.3)
rootdir: C:\Snaps\SnapDev\swtools\soft\mytmptestDir
plugins: cov-2.6.1, html-2.1.1, metadata-1.8.0, modal-3.0, steps-1.7.3, sugar-0.9.3
collecting ... Windows fatal exception: access violation

Current thread 0x00018740 (most recent call first):
  File "C:\Snaps\SnapDev\swtools\soft\mytmptestDir\tmp.py", line 13 in test_function
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\python.py", line 182 in pytest_pyfunc_call
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\python.py", line 1477 in runtest
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 135 in pytest_runtest_call
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 217 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 244 in from_call
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 217 in call_runtest_hook
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 186 in call_and_report
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 100 in runtestprotocol
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\runner.py", line 85 in pytest_runtest_protocol
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\main.py", line 272 in pytest_runtestloop
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\main.py", line 247 in _main
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\main.py", line 191 in wrap_session
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\main.py", line 240 in pytest_cmdline_main
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 87 in <lambda>
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\manager.py", line 93 in _hookexec
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pluggy\hooks.py", line 286 in __call__
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\_pytest\config\__init__.py", line 125 in main
  File "C:\Users\Public\Envs\PyTATenv_v2.2.0\lib\site-packages\pytest\__main__.py", line 7 in <module>
  File "C:\Python37-32\Lib\runpy.py", line 85 in _run_code
  File "C:\Python37-32\Lib\runpy.py", line 193 in _run_module_as_main

 tmp.py ✓                                                                                                                                                      100% ██████████


Results (0.57s):
       1 passed

pytest 5.4.3
pytest -p no:faulthandler tmp.py

pytest -p no:faulthandler tmp.py
Test session starts (platform: win32, Python 3.7.4, pytest 5.4.3, pytest-sugar 0.9.3)
rootdir: C:\Snaps\SnapDev\swtools\soft\mytmptestDir
plugins: cov-2.6.1, html-2.1.1, metadata-1.8.0, modal-3.0, steps-1.7.3, sugar-0.9.3
collecting ... 
 tmp.py ✓                                                                                                                                                      100% ██████████


Results (0.55s):
       1 passed

So yeah... that seems conclusive, I will happily settle with saying the issue is in the DLL, and that pytest actually suffered no regression (on the contrary).

@The-Compiler
Copy link
Member

Yep, definitely looks like this isn't pytest fault, only that the faulthandler introduction made you aware of an issue in your DLL. Closing this then!

@fossum
Copy link

fossum commented Jan 14, 2022

As Google sent me here and I think I have a similar issue, my problem was with python.net package and my build system. If I set the PROCESSOR_ARCHITEW6432 environment variable to None, I get this issue. Leave it as is ("AMD64") and the problem goes away.

@RonnyPfannschmidt
Copy link
Member

@W2Q3Q1 it's strongly discouraged to chdir a lot, in particular the way used in your example it's very fragile due to order dependency

The recommended mechanism is to determine module paths and/or using importlib.resources

@W2Q3Q1
Copy link

W2Q3Q1 commented Sep 29, 2023

@RonnyPfannschmidt It is a very useful comment. But I still cannot explain why Access Violation happened, and I am not sure about the solution I proposed. It seems like an occasional bug.

@RonnyPfannschmidt
Copy link
Member

@W2Q3Q1 the observed bug/issue seems to be a condition that strongly depends on memory layout and external circumstance

that in part is the reason why actions like deferring imports to as late as possible have an effect on it
if the memory layout is different the issue is not triggered

loading modules at vastly different times to ensure memory allocations obscure the memory layout are one technique to deter the issue from being triggered when one has no reliable way to detect and resolve the root cause

in particular memory layout related triggers for bugs are at times spooky
to debug because the part being broken and the part where you see the breaks may be very different

however your code you had pasted bedore also chdirs to the locations of test/code files at seemingly "random" times during both import and function calls, this type of directory switching can lead to quite some confusion depending on which order the imports/code executions happen

for usages in tests i strongly recommend either pytests monkeypatch.chdir or contextlib.chdir

@Marco-Sulla
Copy link

Marco-Sulla commented Nov 29, 2023

In my case, the fault was in pytest. I changed in my pipeline the command from pytest to

python -X faulthandler -m pytest -p no:faulthandler -s

and now the pipeline have no error. The error happened every time with python 3.10 on Windows.

Before the cure:
https://github.com/Marco-Sulla/python-frozendict/actions/runs/7023832830/job/19154138512

The error:

  ============================= test session starts =============================
  platform win32 -- Python 3.10.10, pytest-7.4.3, pluggy-1.3.0
  rootdir: D:\a\python-frozendict\python-frozendict
  collected 242 items
  
  D:\a\python-frozendict\python-frozendict\test\test_frozendict.py ....... [  2%]
  ........................................................................ [ 32%]
  ...........................................                              [ 50%]
  D:\a\python-frozendict\python-frozendict\test\test_frozendict_subclass.py . [ 50%]
  Windows fatal exception: access violation
  
  Current thread 0x00001570 (most recent call first):
    Garbage-collecting
    File "D:\a\python-frozendict\python-frozendict\test\test_frozendict_subclass.py", line 12 in __new__
    File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.10.10\tools\lib\copy.py", line 84 in copy
    File "D:\a\python-frozendict\python-frozendict\test\subclass_only.py", line 43 in test_copycopy_sub
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\python.py", line 194 in pytest_pyfunc_call
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_callers.py", line 77 in _multicall
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_manager.py", line 115 in _hookexec
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_hooks.py", line 493 in __call__
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\python.py", line 1792 in runtest
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\runner.py", line 169 in pytest_runtest_call
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_callers.py", line 77 in _multicall
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_manager.py", line 115 in _hookexec
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_hooks.py", line 493 in __call__
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\runner.py", line 262 in <lambda>
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\runner.py", line 341 in from_call
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\runner.py", line 261 in call_runtest_hook
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\runner.py", line 222 in call_and_report
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\runner.py", line 133 in runtestprotocol
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\runner.py", line 114 in pytest_runtest_protocol
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_callers.py", line 77 in _multicall
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_manager.py", line 115 in _hookexec
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_hooks.py", line 493 in __call__
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\main.py", line 350 in pytest_runtestloop
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_callers.py", line 77 in _multicall
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_manager.py", line 115 in _hookexec
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_hooks.py", line 493 in __call__
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\main.py", line 325 in _main
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\main.py", line 271 in wrap_session
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\main.py", line 318 in pytest_cmdline_main
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_callers.py", line 77 in _multicall
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_manager.py", line 115 in _hookexec
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\pluggy\_hooks.py", line 493 in __call__
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\config\__init__.py", line 169 in main
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\lib\site-packages\_pytest\config\__init__.py", line 192 in console_main
    File "C:\Users\runneradmin\AppData\Local\Temp\cibw-run-t4qbgm8m\cp310-win_amd64\venv-test\Scripts\pytest.exe\__main__.py", line 7 in <module>
    File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.10.10\tools\lib\runpy.py", line 86 in _run_code
    File "C:\Users\runneradmin\AppData\Local\pypa\cibuildwheel\Cache\nuget-cpython\python.3.10.10\tools\lib\runpy.py", line 196 in _run_module_as_main
  Error: Command python D:\a\python-frozendict\python-frozendict/test/debug.py && pytest D:\a\python-frozendict\python-frozendict failed with code 3221225477. None

The cure:
Marco-Sulla/python-frozendict@c1aad3c

After the cure:
https://github.com/Marco-Sulla/python-frozendict/actions/runs/7038238457/job/19154728949

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform: windows windows platform-specific problem status: help wanted developers would like help from experts on this topic type: regression indicates a problem that was introduced in a release which was working previously
Projects
None yet
Development

No branches or pull requests

6 participants