Flaky mark breaks when using along with pytest-depends plugin

[alejomejia11]

When using in the same environment the pytest plugin pytest-depends, the system is unable to re-run a failed test. A general and catastrophic error is obtained. The traceback can be seen below.

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\_pytest\main.py", line 269, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\_pytest\main.py", line 323, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\_pytest\main.py", line 348, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pytest_rerunfailures.py", line 497, in pytest_runtest_protocol
INTERNALERROR>     reports = runtestprotocol(item, nextitem=nextitem, log=False)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\_pytest\runner.py", line 120, in runtestprotocol
INTERNALERROR>     rep = call_and_report(item, "setup", log)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\_pytest\runner.py", line 217, in call_and_report
INTERNALERROR>     report: TestReport = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\manager.py", line 84, in <lambda>
INTERNALERROR>     self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pluggy\callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pytest_depends\__init__.py", line 139, in pytest_runtest_makereport
INTERNALERROR>     manager.register_result(item, outcome.get_result())
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pytest_depends\main.py", line 209, in register_result
INTERNALERROR>     self.results[nodeid].register_result(result)
INTERNALERROR>   File "c:\programdata\anaconda3\envs\pyintegrationtest\lib\site-packages\pytest_depends\main.py", line 40, in register_result
INTERNALERROR>     raise AttributeError(f'Received multiple results for step {result.when} of test {self.nodeid}')
INTERNALERROR> AttributeError: Received multiple results for step setup of test v3_controllers_test/v3_offline_controller_test/temp_test.py::test_temp_mine

This error can be replicated with a simple set of test like follows.

import pytest


def test_temp_mine3():
    # logging.info("trying...")
    assert 1 == 1


@pytest.mark.flaky(reruns=2)
def test_temp_mine():
    # logging.info("trying...")
    assert 1 == 0


def test_temp_mine2():
    # logging.info("trying...")
    assert 1 == 1

[sallner]

I edited the issue for clearer formatting.

@alejomejia11 looking at the traceback you provided and at the code of pytest-depends it looks like pytest-depends cannot deal with a test the runs the setup (or any other step during the run) more than once. GitLab:pytest-depends:main.py

But this is precisely, what pytest-rerunfailures is supposed to do: Run every step again when re-running a failing test.
What do you expect from pytest-rerunfailures to be done in this case? I would recommend to go over to pytest-depends and try to solve the problem there.

[alejomejia11]

As a user or consumer of pytest and their plugins, I just want to notify that pytest-rerunfailures is not compatible with another existent plugin. I'm not responsible for any of both so I don't think I would be able to do modifications in pytest-depends. If you think that doesn't make sense to have both libraries coexisting then that should be, at least, stated in the documentation file. What I would expect as a consumer should be a coexistence between different plugins. Also, it might be important to note that pytest-rerunfailures seems to be compatible with pytest-dependency which's another library to handle tests' dependencies