Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix flaky tests involving scripttest's result.files_updated #6235

Closed
cjerdonek opened this issue Feb 3, 2019 · 4 comments
Closed

Fix flaky tests involving scripttest's result.files_updated #6235

cjerdonek opened this issue Feb 3, 2019 · 4 comments
Labels
C: tests Testing and related things

Comments

@cjerdonek
Copy link
Member

Looks like it's failed a few times on test_force_reinstall_with_same_version_specifier(), and only on the Python 2.7 job.

This is what the intermittent failure looks like:

assert result2.files_updated, 'force-reinstall failed'
AssertionError: force-reinstall failed
assert {}
 +  where {} = <tests.lib.TestPipResult object at 0x10913cb90>.files_updated

My guess is that this is an issue with the scripttest library when running on Mac, specifically with how equality is computed with the files_updated dict:

https://github.com/pypa/scripttest/blob/01c7db96ff01de034bd7781a4ed28d74eb716d32/scripttest.py#L391-L394

You can even see that someone filed a similar issue four years ago in pip's tracker: #2580

Someone should probably file an issue in scripttest's tracker.

Originally posted by @cjerdonek in #6043 (comment)

@pradyunsg
Copy link
Member

pradyunsg commented Jun 12, 2019

@cjerdonek Since you'd raised concerns about scripttest during our conversation at PyCon, I just realized that there's something kinda-sorta similar at: https://github.com/kvas-it/pytest-console-scripts

@ichard26
Copy link
Member

Have we faced flaky failures involving scripttest's result.files_updated lately? If not, let's close this.

@uranusjr
Copy link
Member

I don’t recall this being a problem recently.

@ichard26
Copy link
Member

Let's close this then.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 26, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
C: tests Testing and related things
Projects
None yet
Development

No branches or pull requests

4 participants