When a snapshot test is marked with skip or skipif syrupy exits with a failure exit code

[rodrigogiraoserrao]

Describe the bug

If a snapshot test is skipped because it was marked with pytest.mark.skip or with pytest.mark.skipif, syrupy will:

• correctly report that the related snapshots were unused; but
• incorrectly report a failed test run.

To reproduce

Mark a test with pytest.mark.skip.

Expected behavior

The test run shouldn't terminate with an error.
(It's ok that the snapshots that weren't used because the tests were skipped are reported as unused, but that currently happens with a red [error] message.
It would potentially make sense to count skipped tests separately from truly unused ones...)

Screenshots

Environment (please complete the following information):

• OS: MacOS Sonoma 14.0 (23A344)
• Syrupy Version: tested with 3.0.9 & 4.6.0
• Python Version: 3.12

Additional context

We have some snapshot tests that can only run on Python versions below 3.12, so we'd like to skip some snapshot tests if the Python version is 3.12.

[zzzachzzz]

You can use the --snapshot-warn-unused option to warn instead of error, resulting in an exit code of 0 (assuming all tests pass). You can make this option an automatic default when running pytest by specifying it in a pytest.ini file:

[pytest]
addopts = --snapshot-warn-unused

I do have some gripes with how snapshots interact with skipped tests. I have a test relying on an external api request whose results I snapshot. I mark this test to be skipped because I only wish to run this test occasionally to test my integration with said external api. There's a couple problems:

1. The problem outlined in the original issue (okay whatever, we can workaround it with --snapshot-warn-unused)
2. When I perform a snapshot update with --snapshot-update, because I have a snapshot corresponding to a skipped test, syrupy deletes the snapshot, since it has been identified as "unused". Any suggested workarounds for this issue would be appreciated.

Edit Just found issue #787 which is my problem 2 ^

[ddorian]

I mark this test to be skipped because I only wish to run this test occasionally to test my integration with said external API.

Maybe you can use pytest-recording to cache the response?

[noahnu]

🎉 This issue has been resolved in version 4.6.3 🎉

The release is available on:

• v4.6.3
• GitHub release

Your semantic-release bot 📦🚀