Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Possible regression in 20.2b1: AssertionError: a direct req shouldn't have a parent... #8637

Closed
JulienPalard opened this issue Jul 27, 2020 · 6 comments
Closed

Possible regression in 20.2b1: AssertionError: a direct req shouldn't have a parent... #8637

JulienPalard opened this issue Jul 27, 2020 · 6 comments

Comments

@JulienPalard
Copy link

Environment

  • pip version: 20.2b1
  • Python version: 3.8.4
  • OS: Debian testing

Description

Given those two files:

$ cat setup.cfg
[metadata]
name = test
version = 0.0.1
description = test

[options]
install_requires =
    oeis

$ cat setup.py
from setuptools import setup

setup()

when I run pip-compile setup.py (from pip-tools), I get:

Traceback (most recent call last):
  File "/tmp/pto/.venv/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
  File "/tmp/pto/.venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/scripts/compile.py", line 458, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/resolver.py", line 169, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/resolver.py", line 274, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/resolver.py", line 384, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/repositories/pypi.py", line 231, in get_dependencies
    self._dependencies_cache[ireq] = self.resolve_reqs(
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/repositories/pypi.py", line 171, in resolve_reqs
    reqset.add_requirement(ireq)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/pip/_internal/req/req_set.py", line 113, in add_requirement
    assert install_req.is_direct == (parent_req_name is None), (
AssertionError: a direct req shouldn't have a parent and also, a non direct req should have a parent

But I don't get this with pip 20.1.1.

I did not tried to reproduce with pip only yet, so my reproducer is still bound to pip-tools, which is not ideal, I know.
@triage-new-issues triage-new-issues bot added the S: needs triage Issues/PRs that need to be triaged label Jul 27, 2020
@McSinyx
Copy link
Contributor

McSinyx commented Jul 27, 2020

Hello there, is this still reproducible with pip-tools 5.3.0, which was released yesterday and included jazzband/pip-tools#1168?

@AndydeCleyre
Copy link
Contributor

FWIW with the above example files, and the current release (5.3.0, pip 20.1.1, python 3.8.4), tested on linux, it works fine:

#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile setup.py
#
cycler==0.10.0            # via matplotlib
kiwisolver==1.2.0         # via matplotlib
matplotlib==3.3.0         # via oeis
mpmath==1.1.0             # via sympy
numpy==1.19.1             # via matplotlib
oeis==2020.1.20           # via test (setup.py)
pillow==7.2.0             # via matplotlib
pyparsing==2.4.7          # via matplotlib
python-dateutil==2.8.1    # via matplotlib
six==1.15.0               # via cycler, python-dateutil
sympy==1.6.1              # via oeis

@JulienPalard
Copy link
Author

JulienPalard commented Jul 27, 2020 •
edited
Loading

I tested again, in the exact same environment and can't reproduce it. It's funny because I still have the terminal where I made it fail:

The initial failure, leading me to open this issue, that I still had in a terminal history. 
mdk@seraph:~
$ cd /tmp
mdk@seraph:/tmp
$ mkdir pto
mdk@seraph:/tmp
$ cd pto
mdk@seraph:/tmp/pto
$ cp ~/clones/JulienPalard/oeis/setup.py .
mdk@seraph:/tmp/pto
$ cp ~/clones/JulienPalard/oeis/setup.cfg .
mdk@seraph:/tmp/pto
$ emacs setup.*  # Simplifying the files to the minimum
mdk@seraph:/tmp/pto
$ venv  # a bash function of mine to create a venv with pip (in beta  because you called for testers :p) and some other usefull packages
Collecting black
  Using cached black-19.10b0-py36-none-any.whl (97 kB)
Collecting jedi
  Using cached jedi-0.17.2-py2.py3-none-any.whl (1.4 MB)
Collecting wheel
  Using cached wheel-0.34.2-py2.py3-none-any.whl (26 kB)
Collecting pip
  Using cached pip-20.2b1-py2.py3-none-any.whl (1.5 MB)
Collecting appdirs
  Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting pathspec<1,>=0.6
  Using cached pathspec-0.8.0-py2.py3-none-any.whl (28 kB)
Collecting click>=6.5
  Using cached click-7.1.2-py2.py3-none-any.whl (82 kB)
Collecting regex
  Using cached regex-2020.7.14-cp38-cp38-manylinux2010_x86_64.whl (672 kB)
Collecting toml>=0.9.4
  Using cached toml-0.10.1-py2.py3-none-any.whl (19 kB)
Collecting attrs>=18.1.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting typed-ast>=1.4.0
  Using cached typed_ast-1.4.1-cp38-cp38-manylinux1_x86_64.whl (768 kB)
Collecting parso<0.8.0,>=0.7.0
  Using cached parso-0.7.1-py2.py3-none-any.whl (109 kB)
Installing collected packages: appdirs, pathspec, click, regex, toml, attrs, typed-ast, black, parso, jedi, wheel, pip
  Attempting uninstall: pip
    Found existing installation: pip 20.1.1
    Uninstalling pip-20.1.1:
      Successfully uninstalled pip-20.1.1
Successfully installed appdirs-1.4.4 attrs-19.3.0 black-19.10b0 click-7.1.2 jedi-0.17.2 parso-0.7.1 pathspec-0.8.0 pip-20.2b1 regex-2020.7.14 toml-0.10.1 typed-ast-1.4.1 wheel-0.34.2
(pto) mdk@seraph:/tmp/pto
$ pip install pip-tools
Collecting pip-tools
  Using cached pip_tools-5.3.0-py2.py3-none-any.whl (45 kB)
Collecting six
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Requirement already satisfied: pip>=20.0 in ./.venv/lib/python3.8/site-packages (from pip-tools) (20.2b1)
Requirement already satisfied: click>=7 in ./.venv/lib/python3.8/site-packages (from pip-tools) (7.1.2)
Installing collected packages: six, pip-tools
Successfully installed pip-tools-5.3.0 six-1.15.0
(pto) mdk@seraph:/tmp/pto
$ pip-compile setup.py
Traceback (most recent call last):
  File "/tmp/pto/.venv/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
  File "/tmp/pto/.venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/click/decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/scripts/compile.py", line 458, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/resolver.py", line 169, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/resolver.py", line 274, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/resolver.py", line 384, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/repositories/pypi.py", line 231, in get_dependencies
    self._dependencies_cache[ireq] = self.resolve_reqs(
  File "/tmp/pto/.venv/lib/python3.8/site-packages/piptools/repositories/pypi.py", line 171, in resolve_reqs
    reqset.add_requirement(ireq)
  File "/tmp/pto/.venv/lib/python3.8/site-packages/pip/_internal/req/req_set.py", line 113, in add_requirement
    assert install_req.is_direct == (parent_req_name is None), (
AssertionError: a direct req shouldn't have a parent and also, a non direct req should have a parent
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpauzbybpfsource'>
  _warnings.warn(warn_message, ResourceWarning)
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpa6a84oijbuild'>
  _warnings.warn(warn_message, ResourceWarning)
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.0.0.13', 58334), raddr=('151.101.192.223', 443)>

Having this in my terminal history is great because I have the exact versions of each packets so I can try an exact reproducer:

Me trying to reproduce it again, failing. 
$ python3.8 -m venv .venv
mdk@seraph:/tmp/pt1
$ source .venv/bin/activate
(.venv) mdk@seraph:/tmp/pt1
$ mdk@seraph:~
$ pip install appdirs==1.4.4 attrs==19.3.0 black==19.10b0 click==7.1.2 jedi==0.17.2 parso==0.7.1 pathspec==0.8.0 pip==20.2b1 regex==2020.7.14 toml==0.10.1 typed-ast==1.4.1 wheel==0.34.2 pip-tools==5.3.0 six==1.15.0
Collecting appdirs==1.4.4
  Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB)
Collecting attrs==19.3.0
  Using cached attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting black==19.10b0
  Using cached black-19.10b0-py36-none-any.whl (97 kB)
Collecting click==7.1.2
  Using cached click-7.1.2-py2.py3-none-any.whl (82 kB)
Collecting jedi==0.17.2
  Using cached jedi-0.17.2-py2.py3-none-any.whl (1.4 MB)
Collecting parso==0.7.1
  Using cached parso-0.7.1-py2.py3-none-any.whl (109 kB)
Collecting pathspec==0.8.0
  Using cached pathspec-0.8.0-py2.py3-none-any.whl (28 kB)
Collecting pip==20.2b1
  Using cached pip-20.2b1-py2.py3-none-any.whl (1.5 MB)
Collecting regex==2020.7.14
  Using cached regex-2020.7.14-cp38-cp38-manylinux2010_x86_64.whl (672 kB)
Collecting toml==0.10.1
  Using cached toml-0.10.1-py2.py3-none-any.whl (19 kB)
Collecting typed-ast==1.4.1
  Using cached typed_ast-1.4.1-cp38-cp38-manylinux1_x86_64.whl (768 kB)
Collecting wheel==0.34.2
  Using cached wheel-0.34.2-py2.py3-none-any.whl (26 kB)
Collecting pip-tools==5.3.0
  Using cached pip_tools-5.3.0-py2.py3-none-any.whl (45 kB)
Collecting six==1.15.0
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Installing collected packages: appdirs, attrs, toml, click, pathspec, regex, typed-ast, black, parso, jedi, pip, wheel, six, pip-tools
  Attempting uninstall: pip
    Found existing installation: pip 20.1.1
    Uninstalling pip-20.1.1:
      Successfully uninstalled pip-20.1.1
Successfully installed appdirs-1.4.4 attrs-19.3.0 black-19.10b0 click-7.1.2 jedi-0.17.2 parso-0.7.1 pathspec-0.8.0 pip-20.2b1 pip-tools-5.3.0 regex-2020.7.14 six-1.15.0 toml-0.10.1 typed-ast-1.4.1 wheel-0.34.2
(.venv) mdk@seraph:/tmp/pt1
$ diff /tmp/pto/setup.py setup.py 
(.venv) mdk@seraph:/tmp/pt1
$ diff /tmp/pto/setup.cfg setup.cfg
(.venv) mdk@seraph:/tmp/pt1
$ pip-compile setup.py 
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmp5zdzfdvnbuild'>
  _warnings.warn(warn_message, ResourceWarning)
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpwvwuk76ysource'>
  _warnings.warn(warn_message, ResourceWarning)
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpqutjlnv4build'>
  _warnings.warn(warn_message, ResourceWarning)
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpb_0sfl4_source'>
  _warnings.warn(warn_message, ResourceWarning)
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpl9lf7s5ubuild'>
  _warnings.warn(warn_message, ResourceWarning)
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpnmtxnsj_source'>
  _warnings.warn(warn_message, ResourceWarning)
#
# This file is autogenerated by pip-compile
# To update, run:
#
#    pip-compile setup.py
#
cycler==0.10.0            # via matplotlib
kiwisolver==1.2.0         # via matplotlib
matplotlib==3.3.0         # via oeis
mpmath==1.1.0             # via sympy
numpy==1.19.1             # via matplotlib
oeis==2020.1.20           # via test (setup.py)
pillow==7.2.0             # via matplotlib
pyparsing==2.4.7          # via matplotlib
python-dateutil==2.8.1    # via matplotlib
six==1.15.0               # via cycler, python-dateutil
sympy==1.6.1              # via oeis
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmp8uxcysx7build'>
  _warnings.warn(warn_message, ResourceWarning)
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmp046fe90xsource'>
  _warnings.warn(warn_message, ResourceWarning)
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpz8b921rysource'>
  _warnings.warn(warn_message, ResourceWarning)
/home/mdk/.local/lib/python3.8/tempfile.py:819: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmphsa83je_build'>
  _warnings.warn(warn_message, ResourceWarning)

OK so this is harder to reproduce than initially though, I don't understand what's happening here, either a cache issue, or a Schrödinger bug?

@pfmoore
Copy link
Member

pfmoore commented Jul 27, 2020

pip-tools calls pip internals directly, which we don't support (and pip-tools are fine with that, as far as I know). So this should be tracked via the pip-tools issue, not here. In particular, pip-tools is modifying is_direct/user_supplied on an ireq, which seems very brave of them if you ask me 🙂

If it's possible to reproduce with just pip, then we can take a look.

@fatalcoder fatalcoder mentioned this issue Jul 27, 2020
Closed
@JulienPalard
Copy link
Author

OK to track it pip-tools side :)

@atugushev
Copy link
Contributor

FWIW it's fixed in #8026.

@pradyunsg pradyunsg added S: needs triage Issues/PRs that need to be triaged and removed S: needs triage Issues/PRs that need to be triaged labels Feb 12, 2021
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 1, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@JulienPalard @pfmoore @AndydeCleyre @pradyunsg @atugushev @McSinyx