Error with packages where their `setup.py egg_info` call fails #343

StephenHesperus · 2016-04-15T10:11:38Z

This is what I got when I have psycopg2 in .in file:

Traceback (most recent call last):
  File "/home/ste/virtualenvs/favproglang/bin/pip-compile", line 11, in <module>
    sys.exit(cli())
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/piptools/scripts/compile.py", line 163, in cli
    results = resolver.resolve()
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/piptools/resolver.py", line 78, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/piptools/resolver.py", line 160, in _resolve_one_round
    for best_match in best_matches
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/piptools/resolver.py", line 161, in <genexpr>
    for dep in self._iter_dependencies(best_match))
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/piptools/resolver.py", line 230, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/piptools/repositories/local.py", line 48, in get_dependencies
    return self.repository.get_dependencies(ireq)
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/piptools/repositories/pypi.py", line 135, in get_dependencies
    dependencies = reqset._prepare_file(self.finder, ireq)
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/pip/req/req_set.py", line 591, in _prepare_file
    abstract_dist.prep_for_dist()
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/pip/req/req_set.py", line 127, in prep_for_dist
    self.req_to_install.run_egg_info()
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/pip/req/req_install.py", line 430, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/home/ste/virtualenvs/favproglang/lib/python3.5/site-packages/pip/utils/__init__.py", line 718, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/tmpijrpvg0fbuild/psycopg2/

The text was updated successfully, but these errors were encountered:

LuisAlejandro · 2016-04-25T22:23:03Z

I see this error repeating itself lately, its also reported here #279 and #228. Do you have any info on how to resolve this? @nvie @flavianh @veegee

Thanks in advance.

LuisAlejandro · 2016-04-25T23:08:20Z

Well, for more info, if one changes this line to show_stdout=True on pip, then you can see the output of the run_egg_info method. For example, for psycopg2:

pip-compile -r -v x.txt 
Using indexes:
  https://pypi.python.org/simple

                          ROUND 1                           
Current constraints:
  psycopg2

Finding the best candidates:
  found candidate psycopg2==2.6.1 (constraint was <any>)

Finding secondary dependencies:
  psycopg2==2.6.1 not in cache, need to check index
['/usr/bin/python', '-c', "import setuptools, tokenize;__file__='/tmp/tmpOeliZVbuild/psycopg2/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))", 'egg_info', '--egg-base', 'pip-egg-info']
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found

Error: You need to install postgresql-server-dev-X.Y for building a server-side extension or libpq-dev for building a client-side application.

Traceback (most recent call last):
  File "/usr/local/bin/pip-compile", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/dist-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/piptools/scripts/compile.py", line 163, in cli
    results = resolver.resolve()
  File "/usr/local/lib/python2.7/dist-packages/piptools/resolver.py", line 78, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/usr/local/lib/python2.7/dist-packages/piptools/resolver.py", line 160, in _resolve_one_round
    for best_match in best_matches
  File "/usr/local/lib/python2.7/dist-packages/piptools/resolver.py", line 161, in <genexpr>
    for dep in self._iter_dependencies(best_match))
  File "/usr/local/lib/python2.7/dist-packages/piptools/resolver.py", line 230, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/usr/local/lib/python2.7/dist-packages/piptools/repositories/local.py", line 48, in get_dependencies
    return self.repository.get_dependencies(ireq)
  File "/usr/local/lib/python2.7/dist-packages/piptools/repositories/pypi.py", line 135, in get_dependencies
    dependencies = reqset._prepare_file(self.finder, ireq)
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 591, in _prepare_file
    abstract_dist.prep_for_dist()
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 127, in prep_for_dist
    self.req_to_install.run_egg_info()
  File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 431, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/usr/local/lib/python2.7/dist-packages/pip/utils/__init__.py", line 718, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/tmpOeliZVbuild/psycopg2/

Which seems right as i don't have the build dependencies satisfied. But is ilogical that i have to met dependencies on an information command as egg_info. I will go on with my research and report here.

LuisAlejandro · 2016-04-25T23:59:31Z

Well, i've tracked this particular problem to this ticket psycopg/psycopg2#128, which is not a pip-tools problem.

However, there are several other packages that have a bad written setup.py that causes an informative command as egg_info to fail if the build dependencies are not met or if the egg_info fails because of another unknown reason.

Generally it seems to me that pip-tools should find another way around to figure dependencies out. Because this discovery means that every package that pip-tools processes, needs to have its build dependencies satisfied.

@nvie what do you think?

LuisAlejandro · 2016-04-26T00:01:57Z

More on this pypa/pip#661

nvie · 2016-04-26T07:05:21Z

Generally it seems to me that pip-tools should find another way around to figure dependencies out. Because this discovery means that every package that pip-tools processes, needs to have its build dependencies satisfied.

Yeah, that would be ideal. This is notoriously hard with Python packaging though. As the thread you linked points out, Python packages don't carry static metadata, so they typically need to execute setup.py to acquire that information. If you have ideas for a better solution, I'm obviously interested!

pablote · 2017-10-21T18:03:11Z

This just started happening to me, is there any workaround for it? This is my requirements.in:

assertpy
flask
gunicorn
joblib
keras
matplotlib
numpy
pandas
scikit-learn
scipy
pillow
scikit-image
tensorflow

asmaps · 2017-10-25T12:41:35Z

Same here. When I comment out matplotlib it works.

Traceback (most recent call last):
  File "/usr/local/bin/pip-compile", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/piptools/scripts/compile.py", line 184, in cli
    results = resolver.resolve(max_rounds=max_rounds)
  File "/usr/local/lib/python3.6/site-packages/piptools/resolver.py", line 101, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/usr/local/lib/python3.6/site-packages/piptools/resolver.py", line 198, in _resolve_one_round
    for dep in self._iter_dependencies(best_match):
  File "/usr/local/lib/python3.6/site-packages/piptools/resolver.py", line 284, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/usr/local/lib/python3.6/site-packages/piptools/repositories/local.py", line 62, in get_dependencies
    return self.repository.get_dependencies(ireq)
  File "/usr/local/lib/python3.6/site-packages/piptools/repositories/pypi.py", line 170, in get_dependencies
    self._dependencies_cache[ireq] = reqset._prepare_file(self.finder, ireq)
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 634, in _prepare_file
    abstract_dist.prep_for_dist()
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_set.py", line 129, in prep_for_dist
    self.req_to_install.run_egg_info()
  File "/usr/local/lib/python3.6/site-packages/pip/req/req_install.py", line 439, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/usr/local/lib/python3.6/site-packages/pip/utils/__init__.py", line 707, in call_subprocess
    % (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command "python setup.py egg_info" failed with error code 1 in /tmp/tmp_888bvtbbuild/subprocess32/

pablote · 2017-10-25T14:22:12Z

still not working without matplotlib

vphilippon · 2017-10-25T19:16:12Z

@asmaps This is an issue that is fixed in master (cannot make release, check #531) where pip-compile fetches python 2 dependencies, even in a python 3 environment.
For the record, this issue was not in 1.9.0, so you can always downpin to that version for the time being (sigh)

@pablote could you provide your pip-compile --verbose output. We'll confirm if its the same (or similar) issue.

Note: If you decide to try with the code in master, be sure to run pip-compile --rebuild to clear the dependency cache, which could be corrupted due to the said bug.

vphilippon · 2017-11-22T22:12:10Z

This is now fixed in 1.10.2
Make sure to clear you dependency cache: pip-compile --rebuild

pablote · 2017-11-23T13:14:58Z

I recreated the virtualenv and now is working fine, thanks :)

kronion · 2017-12-28T21:46:12Z

I was still seeing this issue with pip-tools 1.11.0 when trying to compile a bare-bones requirements.in containing only the word Django.

The issue was that the latest version of Django supports only Python 3.x, but my virtualenv was for Python 2.7. The solution was to pin Django in requirements.in: Django==1.11.

I don't know if pip-tools can do anything to clarify the cause of the error (it already shows the same error message that you'll see if you pip install Django manually), but people should know to watch out for this.

StephenHesperus closed this as completed Apr 17, 2016

nvie added the PR wanted Feature is discussed or bug is confirmed, PR needed label Apr 26, 2016

nvie reopened this Apr 26, 2016

This was referenced Apr 26, 2016

pip-compile fails when scipy==0.9.0 is in requirements.in #279

Closed

pip-compile fails when enum34 is listed in requirements.in #228

Closed

nvie changed the title ~~Error when pip-compile with psycopg2~~ Error with packages where their setup.py egg_info call fails Apr 26, 2016

vphilippon closed this as completed Nov 22, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error with packages where their `setup.py egg_info` call fails #343

Error with packages where their `setup.py egg_info` call fails #343

StephenHesperus commented Apr 15, 2016

LuisAlejandro commented Apr 25, 2016

LuisAlejandro commented Apr 25, 2016 •

edited

Loading

LuisAlejandro commented Apr 25, 2016 •

edited

Loading

LuisAlejandro commented Apr 26, 2016

nvie commented Apr 26, 2016 •

edited

Loading

pablote commented Oct 21, 2017

asmaps commented Oct 25, 2017

pablote commented Oct 25, 2017

vphilippon commented Oct 25, 2017

vphilippon commented Nov 22, 2017

pablote commented Nov 23, 2017

kronion commented Dec 28, 2017

Error with packages where their setup.py egg_info call fails #343

Error with packages where their setup.py egg_info call fails #343

Comments

StephenHesperus commented Apr 15, 2016

LuisAlejandro commented Apr 25, 2016

LuisAlejandro commented Apr 25, 2016 • edited Loading

LuisAlejandro commented Apr 25, 2016 • edited Loading

LuisAlejandro commented Apr 26, 2016

nvie commented Apr 26, 2016 • edited Loading

pablote commented Oct 21, 2017

asmaps commented Oct 25, 2017

pablote commented Oct 25, 2017

vphilippon commented Oct 25, 2017

vphilippon commented Nov 22, 2017

pablote commented Nov 23, 2017

kronion commented Dec 28, 2017

Error with packages where their `setup.py egg_info` call fails #343

Error with packages where their `setup.py egg_info` call fails #343

LuisAlejandro commented Apr 25, 2016 •

edited

Loading

LuisAlejandro commented Apr 25, 2016 •

edited

Loading

nvie commented Apr 26, 2016 •

edited

Loading