Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Snorkel fails to install for Python 3.8 due to scikit-learn version #1627

Closed
rjurney opened this issue Feb 12, 2021 · 6 comments · Fixed by #1628
Closed

Snorkel fails to install for Python 3.8 due to scikit-learn version #1627

rjurney opened this issue Feb 12, 2021 · 6 comments · Fixed by #1628
Assignees
@brahmaneya

Comments

@rjurney
Copy link

rjurney commented Feb 12, 2021

Issue description

When you install Snorkel, it tries to install scikit-learn - which does not build. Thus Snorkel does not build. Can we please update the version of scikit-learn so people can build and use Snorkel?

Code example/repro steps

pip install snorkel

It tries to install scikit-learn<0.22.0,>=0.20.2 and fails :(

Building wheels for collected packages: scikit-learn
  Building wheel for scikit-learn (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/rjurney/opt/anaconda3/envs/deep_discovery/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py'"'"'; __file__='"'"'/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-wheel-1u1ts_5o
       cwd: /private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/
  Complete output (28 lines):
  Partial import of sklearn during the build process.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py", line 290, in <module>
      setup_package()
    File "/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py", line 286, in setup_package
      setup(**metadata)
    File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/core.py", line 135, in setup
      config = configuration()
    File "/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py", line 174, in configuration
      config.add_subpackage('sklearn')
    File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1019, in add_subpackage
      config_list = self.get_subpackage(subpackage_name, subpackage_path,
    File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 985, in get_subpackage
      config = self._get_configuration_from_setup_py(
    File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 927, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
    File "sklearn/setup.py", line 62, in configuration
      config.add_subpackage('utils')
    File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1019, in add_subpackage
      config_list = self.get_subpackage(subpackage_name, subpackage_path,
    File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 985, in get_subpackage
      config = self._get_configuration_from_setup_py(
    File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 927, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
    File "sklearn/utils/setup.py", line 8, in configuration
      from Cython import Tempita
  ModuleNotFoundError: No module named 'Cython'
  ----------------------------------------
  ERROR: Failed building wheel for scikit-learn
  Running setup.py clean for scikit-learn
Failed to build scikit-learn
Installing collected packages: grpcio, absl-py, tensorboard, scikit-learn, networkx, munkres, snorkel
  Attempting uninstall: scikit-learn
    Found existing installation: scikit-learn 0.24.1
    Uninstalling scikit-learn-0.24.1:
      Successfully uninstalled scikit-learn-0.24.1
    Running setup.py install for scikit-learn ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/rjurney/opt/anaconda3/envs/deep_discovery/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py'"'"'; __file__='"'"'/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-record-kias8bzm/install-record.txt --single-version-externally-managed --compile --install-headers /Users/rjurney/opt/anaconda3/envs/deep_discovery/include/python3.8/scikit-learn
         cwd: /private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/
    Complete output (28 lines):
    Partial import of sklearn during the build process.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py", line 290, in <module>
        setup_package()
      File "/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py", line 286, in setup_package
        setup(**metadata)
      File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/core.py", line 135, in setup
        config = configuration()
      File "/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py", line 174, in configuration
        config.add_subpackage('sklearn')
      File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1019, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 985, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 927, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/setup.py", line 62, in configuration
        config.add_subpackage('utils')
      File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1019, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 985, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 927, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/utils/setup.py", line 8, in configuration
        from Cython import Tempita
    ModuleNotFoundError: No module named 'Cython'
    ----------------------------------------
  Rolling back uninstall of scikit-learn
  Moving to /Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/scikit_learn-0.24.1.dist-info/
   from /Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/~cikit_learn-0.24.1.dist-info
  Moving to /Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/sklearn/
   from /Users/rjurney/opt/anaconda3/envs/deep_discovery/lib/python3.8/site-packages/~klearn
ERROR: Command errored out with exit status 1: /Users/rjurney/opt/anaconda3/envs/deep_discovery/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py'"'"'; __file__='"'"'/private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-install-oo_m8xg5/scikit-learn_d710130497834af2bd9b9f9514867b07/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/3h/vnn5rkyx3nx7yyrfyynrhrlw0000gn/T/pip-record-kias8bzm/install-record.txt --single-version-externally-managed --compile --install-headers /Users/rjurney/opt/anaconda3/envs/deep_discovery/include/python3.8/scikit-learn Check the logs for full command output.

Expected Behavior

I expect the install to succeed. It fails.

System Info

  • How you installed Snorkel (conda, pip, source): pip
  • Build command you used (if compiling from source):
  • OS: Mac OS X 10.15.7 (19H524) - bash
  • Python version: Python 3.8.5
  • Snorkel version: snorkel-0.9.6-py3-none-any.whl
  • Versions of any other relevant libraries:

Additional context

Add any other context about the problem here.
@bhancock8
Copy link
Member

bhancock8 commented Feb 12, 2021 •
edited
Loading

Hi @rjurney, we're taking a look. I haven't been able to repro your issue yet. As a minimal example, on Mac OS X 11.1 with Python 3.7.3, the following commands succeed for me:

python -m venv env
source env/bin/activate
pip install snorkel
python [enter python shell]
import snorkel

@bhancock8
Copy link
Member

It looks like Python 3.7 works fine, Python 3.8 hits the Cython import issue.

@ScottIsWorking
Copy link

scikit-learn versioning is definitely hampering my ability to use snorkel either locally with pipenv and either python 3.7 or 3.8, or remotely google colab. I'll explain:

Locally

mkdir snorkel_test; cd snorkel_test
pyenv local 3.7.7
pipenv install snorkel

output:

Creating a Pipfile for this project…
Error:  An error occurred while installing snorkel!
Error text: Collecting snorkel
  Using cached snorkel-0.9.6-py3-none-any.whl (144 kB)
Requirement already satisfied, skipping upgrade: pandas<2.0.0,>=0.25.0 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (1.2.2)
Requirement already satisfied, skipping upgrade: munkres>=1.0.6 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (1.1.4)
Requirement already satisfied, skipping upgrade: networkx<2.4,>=2.2 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (2.3)
Requirement already satisfied, skipping upgrade: tqdm<5.0.0,>=4.33.0 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (4.56.2)
Collecting scikit-learn<0.22.0,>=0.20.2
  Using cached scikit-learn-0.21.3.tar.gz (12.2 MB)
Requirement already satisfied, skipping upgrade: torch<2.0.0,>=1.2.0 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (1.7.1)
Requirement already satisfied, skipping upgrade: numpy<2.0.0,>=1.16.0 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (1.20.1)
Requirement already satisfied, skipping upgrade: scipy<2.0.0,>=1.2.0 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (1.6.0)
Requirement already satisfied, skipping upgrade: tensorboard<2.0.0,>=1.14.0 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (1.15.0)
Requirement already satisfied, skipping upgrade: pytz>=2017.3 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from pandas<2.0.0,>=0.25.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (2021.1)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.7.3 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from pandas<2.0.0,>=0.25.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (2.8.1)
Requirement already satisfied, skipping upgrade: decorator>=4.3.0 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from networkx<2.4,>=2.2->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (4.4.2)
Requirement already satisfied, skipping upgrade: joblib>=0.11 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from scikit-learn<0.22.0,>=0.20.2->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (1.0.1)
Requirement already satisfied, skipping upgrade: typing-extensions in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from torch<2.0.0,>=1.2.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (3.7.4.3)
Requirement already satisfied, skipping upgrade: setuptools>=41.0.0 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from tensorboard<2.0.0,>=1.14.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (51.1.2)
Requirement already satisfied, skipping upgrade: six>=1.10.0 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from tensorboard<2.0.0,>=1.14.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (1.15.0)
Requirement already satisfied, skipping upgrade: wheel>=0.26; python_version >= "3" in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from tensorboard<2.0.0,>=1.14.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (0.36.2)
Requirement already satisfied, skipping upgrade: protobuf>=3.6.0 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from tensorboard<2.0.0,>=1.14.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (3.14.0)
Requirement already satisfied, skipping upgrade: werkzeug>=0.11.15 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from tensorboard<2.0.0,>=1.14.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (1.0.1)
Requirement already satisfied, skipping upgrade: absl-py>=0.4 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from tensorboard<2.0.0,>=1.14.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (0.11.0)
Requirement already satisfied, skipping upgrade: markdown>=2.6.8 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from tensorboard<2.0.0,>=1.14.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (3.3.3)
Requirement already satisfied, skipping upgrade: grpcio>=1.6.3 in /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages (from tensorboard<2.0.0,>=1.14.0->snorkel->-r /var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pipenv-mn50gqr6-requirements/pipenv-f03uyyos-requirement.txt (line 1)) (1.35.0)
Building wheels for collected packages: scikit-learn
  Building wheel for scikit-learn (setup.py): started
  Building wheel for scikit-learn (setup.py): finished with status 'error'
  Running setup.py clean for scikit-learn
Failed to build scikit-learn
Installing collected packages: scikit-learn, snorkel
    Running setup.py install for scikit-learn: started
    Running setup.py install for scikit-learn: finished with status 'error'

  ERROR: Command errored out with exit status 1:
   command: /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py'"'"'; __file__='"'"'/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-wheel-hhb3lksb
       cwd: /private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/
  Complete output (28 lines):
  Partial import of sklearn during the build process.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py", line 290, in <module>
      setup_package()
    File "/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py", line 286, in setup_package
      setup(**metadata)
    File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/core.py", line 135, in setup
      config = configuration()
    File "/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py", line 174, in configuration
      config.add_subpackage('sklearn')
    File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1019, in add_subpackage
      config_list = self.get_subpackage(subpackage_name, subpackage_path,
    File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 985, in get_subpackage
      config = self._get_configuration_from_setup_py(
    File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 927, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
    File "sklearn/setup.py", line 62, in configuration
      config.add_subpackage('utils')
    File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1019, in add_subpackage
      config_list = self.get_subpackage(subpackage_name, subpackage_path,
    File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 985, in get_subpackage
      config = self._get_configuration_from_setup_py(
    File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 927, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
    File "sklearn/utils/setup.py", line 8, in configuration
      from Cython import Tempita
  ModuleNotFoundError: No module named 'Cython'
  ----------------------------------------
  ERROR: Failed building wheel for scikit-learn
    ERROR: Command errored out with exit status 1:
     command: /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py'"'"'; __file__='"'"'/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-record-vxkh22ce/install-record.txt --single-version-externally-managed --compile --install-headers /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/include/site/python3.8/scikit-learn
         cwd: /private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/
    Complete output (28 lines):
    Partial import of sklearn during the build process.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py", line 290, in <module>
        setup_package()
      File "/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py", line 286, in setup_package
        setup(**metadata)
      File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/core.py", line 135, in setup
        config = configuration()
      File "/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py", line 174, in configuration
        config.add_subpackage('sklearn')
      File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1019, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 985, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 927, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/setup.py", line 62, in configuration
        config.add_subpackage('utils')
      File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 1019, in add_subpackage
        config_list = self.get_subpackage(subpackage_name, subpackage_path,
      File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 985, in get_subpackage
        config = self._get_configuration_from_setup_py(
      File "/Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/lib/python3.8/site-packages/numpy/distutils/misc_util.py", line 927, in _get_configuration_from_setup_py
        config = setup_module.configuration(*args)
      File "sklearn/utils/setup.py", line 8, in configuration
        from Cython import Tempita
    ModuleNotFoundError: No module named 'Cython'
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py'"'"'; __file__='"'"'/private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-install-9qhcv9yj/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/g6/z_v9cfkj2w74ns3cfz8dn2b40000gn/T/pip-record-vxkh22ce/install-record.txt --single-version-externally-managed --compile --install-headers /Users/cambo/.local/share/virtualenvs/snorkel_test-mdj0W--R/include/site/python3.8/scikit-learn Check the logs for full command output.

I can confirm that the virtualenv approach laid out by @bhancock8 above works for 3.7 and not 3.8, but it seems using pipenv does not work in either case.

Google Colab

Google Colab seems to run Python 3.6.9. Here the issue seems to be that snorkel wants to install UMAP which requires scikit-learn>=0.22 while colab installs 0.21.3 by default. At the same time, snorkel 0.9.6 requires scikit-learn<0.22.0.

Here's a Colab notebook that runs through this problem

@rth
Copy link
Contributor

rth commented Feb 12, 2021 •
edited
Loading

Yes, this is due to the fact that the scikit-learn dependency is outdated and does not have pre-built wheels for Python 3.8. The solution is to relax the dependency to 0.24. Related #1613

Should be resolved in #1628

@henryre henryre mentioned this issue Feb 13, 2021
Merged
5 tasks
@henryre henryre changed the title Snorkel does not build! Please bump scikit-learn version or the library is unusable. Snorkel fails to install for Python 3.8 due to scikit-learn version Feb 14, 2021
@yongzx
Copy link

yongzx commented Feb 19, 2021 •
edited
Loading

When I use pip3 install snorkel I still run into the same error "ModuleNotFoundError: No module named 'Cython'"

I have to use the command pip3 install git+https://github.com/snorkel-team/snorkel.git#egg=snorkel to install snorkel.

@rjurney
Copy link
Author

rjurney commented Feb 19, 2021 •
edited
Loading

@yongzx this is fixed by bumping the version of scikit-learn to scikit-learn>=0.24.0 which you can do in a fork. There is a PR right now for this that was just merged into master branch, so try using master or at least monkey patching that commit into v0.9.6: #1628

You may need to create a fresh conda, pipenv or virtualenv and then do the install as old components screw things up. I had your issue, bumped the version myself, still had it, nuked my conda env, rebuilt it from scratch with the new snorkel and it works.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brahmaneya brahmaneya

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update build to support Python 3.8 snorkel-team/snorkel
6 participants
@rjurney @rth @brahmaneya @bhancock8 @yongzx @ScottIsWorking