undefined symbol: nvrtcGetProgramLogSize with pytorch=1.0.1

[lesteve]

Actual Behavior

python -c 'import torch' causes the following error:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/anaconda/lib/python3.7/site-packages/torch/__init__.py", line 102, in <module>
    from torch._C import *
ImportError: /opt/anaconda/lib/python3.7/site-packages/torch/lib/libtorch.so.1: undefined symbol: nvrtcGetProgramLogSize

According to pytorch/pytorch#2091 (comment)

I can reproduce the problem with docker see below. Note that I tried on various computers (outside of docker) and it sometime works. It's not clear why, but maybe having a NVIDIA graphics card or GPU is enough for 'import torch' to work without error.

If I understand correctly conda-forge/pytorch-cpu-feedstock#7 (comment) pytorch from defaults channel should work without a GPU so I guess this feels like an unexpected error.

Expected Behavior

python -c 'import torch' succeeds without error.

Steps to Reproduce

I can reproduce it with docker, the Dockerfile is based on Ubuntu 16.04, downloads the miniconda installer and installs pytorch with conda from the defaults channel.

Dockerfile

FROM ubuntu:16.04

RUN apt-get update
RUN apt-get install curl -y
RUN curl -o miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
    bash miniconda.sh -f -b -p /opt/anaconda && \
    /opt/anaconda/bin/conda clean -tipy && \
    rm -f miniconda.sh
ENV PATH /opt/anaconda/bin:$PATH
RUN conda install --yes python=3.7 pytorch=1.0.1

docker build . -t test && docker run -it test python -c 'import torch'

Output:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/opt/anaconda/lib/python3.7/site-packages/torch/__init__.py", line 102, in <module>
    from torch._C import *
ImportError: /opt/anaconda/lib/python3.7/site-packages/torch/lib/libtorch.so.1: undefined symbol: nvrtcGetProgramLogSize

Anaconda or Miniconda version:

PyTorch 1.0.1

Operating System:

Ubuntu 16.04

conda info

     active environment : None
       user config file : /root/.condarc
 populated config files : 
          conda version : 4.7.10
    conda-build version : not installed
         python version : 3.7.3.final.0
       virtual packages : 
       base environment : /opt/anaconda  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/anaconda/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/anaconda/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.7.10 requests/2.22.0 CPython/3.7.3 Linux/4.15.0-54-generic ubuntu/16.04.6 glibc/2.23
                UID:GID : 0:0
             netrc file : None
           offline mode : False

conda list --show-channel-urls

# packages in environment at /opt/anaconda:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main    defaults
asn1crypto                0.24.0                   py37_0    defaults
blas                      1.0                         mkl    defaults
bzip2                     1.0.8                h7b6447c_0    defaults
ca-certificates           2019.5.15                     0    defaults
certifi                   2019.6.16                py37_1    defaults
cffi                      1.12.3           py37h2e261b9_0    defaults
chardet                   3.0.4                    py37_1    defaults
conda                     4.7.10                   py37_0    defaults
conda-package-handling    1.3.11                   py37_0    defaults
cryptography              2.7              py37h1ba5d50_0    defaults
cudatoolkit               10.0.130                      0    defaults
cudnn                     7.6.0                cuda10.0_0    defaults
idna                      2.8                      py37_0    defaults
intel-openmp              2019.4                      243    defaults
libarchive                3.3.3                h5d8350f_5    defaults
libedit                   3.1.20181209         hc058e9b_0    defaults
libffi                    3.2.1                hd88cf55_4    defaults
libgcc-ng                 9.1.0                hdf63c60_0    defaults
libgfortran-ng            7.3.0                hdf63c60_0    defaults
libstdcxx-ng              9.1.0                hdf63c60_0    defaults
libxml2                   2.9.9                hea5a465_1    defaults
lz4-c                     1.8.1.2              h14c3975_0    defaults
lzo                       2.10                 h49e0be7_2    defaults
mkl                       2019.4                      243    defaults
mkl_fft                   1.0.12           py37ha843d7b_0    defaults
mkl_random                1.0.2            py37hd81dba3_0    defaults
ncurses                   6.1                  he6710b0_1    defaults
ninja                     1.9.0            py37hfd86e86_0    defaults
numpy                     1.16.4           py37h7e9f1db_0    defaults
numpy-base                1.16.4           py37hde5b4d6_0    defaults
openssl                   1.1.1c               h7b6447c_1    defaults
pip                       19.1.1                   py37_0    defaults
pycosat                   0.6.3            py37h14c3975_0    defaults
pycparser                 2.19                     py37_0    defaults
pyopenssl                 19.0.0                   py37_0    defaults
pysocks                   1.7.0                    py37_0    defaults
python                    3.7.3                h0371630_0    defaults
python-libarchive-c       2.8                     py37_11    defaults
pytorch                   1.0.1           cuda100py37he554f03_0    defaults
readline                  7.0                  h7b6447c_5    defaults
requests                  2.22.0                   py37_0    defaults
ruamel_yaml               0.15.46          py37h14c3975_0    defaults
setuptools                41.0.1                   py37_0    defaults
six                       1.12.0                   py37_0    defaults
sqlite                    3.29.0               h7b6447c_0    defaults
tk                        8.6.8                hbc83047_0    defaults
tqdm                      4.32.1                     py_0    defaults
urllib3                   1.24.2                   py37_0    defaults
wheel                     0.33.4                   py37_0    defaults
xz                        5.2.4                h14c3975_4    defaults
yaml                      0.1.7                had09818_2    defaults
zlib                      1.2.11               h7b6447c_3    defaults
zstd                      1.3.7                h0b5b093_0    defaults

Work-arounds:

• one work-around is to use pytorch-cpu from conda-forge
• another work around is to use cudatoolkit=8 (you need to reenable the free channel which is probably not recommended.

conda config --set restore_free_channel true
conda create -n test pytorch=1.0.1 cudatoolkit=8 -y

• Installing from the pytorch channel works too which would indicate this is not a problem in PyTorch per se but more a packaging problem

[lesteve]

Note that the pytorch 1.1.0 uploaded recently (7 August 2019) does not seem to have this problem.

[jjhelmus]

I updated PyTorch to 1.1.0 yesterday. These new packages should work on system without a GPU, although slower than if you have a GPU available. PyTorch has had this ability in previous releases as well but some of the flags that are used to build the packages in the anaconda package repository can interfere with this feature.