dyld: Library not loaded: @rpath/libfftw3.3.dylib with conda FFTW

[pavlin-policar]

I'm putting this here because it may be helpful to someone in the future (probably me), experiencing the following error on Macs:

dyld: Library not loaded: @rpath/libfftw3.3.dylib
  Referenced from: /.../FIt-SNE/bin/fast_tsne
  Reason: image not found

I installed FFTW by running conda install -c conda-forge fftw. Next, I cloned this repo and ran the following command

g++ -I"${CONDA_PREFIX}"/include -L"${CONDA_PREFIX}"/lib \
    -std=c++11 -O3  src/sptree.cpp src/tsne.cpp src/nbodyfft.cpp \
    -o bin/fast_tsne -pthread -lfftw3 -lm -Wno-address-of-packed-member

so the compiler and linker knew what to do. Trying to do the (I guess) standard import for fast_tsne

import sys; sys.path.append("/.../FIt-SNE")
from fast_tsne import fast_tsne

I got the error above

dyld: Library not loaded: @rpath/libfftw3.3.dylib
  Referenced from: /.../FIt-SNE/bin/fast_tsne
  Reason: image not found

My current fix is setting the environmental variable

DYLD_LIBRARY_PATH="$CONDA_PREFIX/lib" python run.py

Not very satisfying. If somebody comes up with something better, that would be fantastic.

[pavlin-policar]

I have indeed found a better way: we can statically link libfftw3 instead of dynamically linking it, as in the command above. Then we can compile with

g++ -I"${CONDA_PREFIX}"/include -std=c++11 -O3 \
    src/sptree.cpp src/tsne.cpp src/nbodyfft.cpp "${CONDA_PREFIX}"/lib/libfftw3.a \
    -o bin/fast_tsne -pthread -lm -Wno-address-of-packed-member 

Then I don't need to bother with DYLD_LIBRARY_PATH anymore. True, the executable file size jumps from 170KB to 2.1MB, but that's a small price to pay for convenience.

[dkobak]

I thought @linqiaozhi is working on a Mac laptop (?). Have you not run into these problems, George?

[pavlin-policar]

This is not a problem if FFTW is installed system-wide, which I suppose is the recommended procedure. Sometimes you don't have root access, or in my case, I just wanted the package to be self-contained and not pollute my system directories.

[dkobak]

Hmm, I did have to install FIt-SNE on our linux server where I don't have sudo rights, and I followed the instructions here #35 and used

g++ -std=c++11 -O3 src/sptree.cpp src/tsne.cpp src/nbodyfft.cpp -I<path_to_include_directory> -L<path_to_lib_directory> -o bin/fast_tsne -pthread -lfftw3 -lm

which is what you did first. And I don't think I had any problems with it afterwards. Anyway, static linking looks like it might be a more robust option. We should maybe add these instructions it to the README.