Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Installation issue on apple M2 #490

Closed
Machadum opened this issue Jul 4, 2024 · 10 comments
Closed

Installation issue on apple M2 #490

Machadum opened this issue Jul 4, 2024 · 10 comments
Labels
question

Comments

@Machadum
Copy link

Machadum commented Jul 4, 2024 •
edited
Loading

Hi

I am trying to install sniffles on my new Mac without success. Can you help me please?

I created a conda environment for sniffles using:

conda create --name sniffles
conda activate sniffles

but then none of the command lines below worked
pip install sniffles
or
conda install sniffles=2.3

Please see the below the different outputs
pip install sniffles

Collecting sniffles
  Using cached sniffles-2.3.3-py3-none-any.whl.metadata (5.1 kB)
Requirement already satisfied: pysam>=0.21.0 in ./sniffles/lib/python3.12/site-packages (from sniffles) (0.22.1)
Collecting edlib>=1.3.9 (from sniffles)
  Using cached edlib-1.3.9.tar.gz (91 kB)
  Preparing metadata (setup.py) ... done
Requirement already satisfied: psutil>=5.9.4 in ./sniffles/lib/python3.12/site-packages (from sniffles) (6.0.0)
Using cached sniffles-2.3.3-py3-none-any.whl (57 kB)
Building wheels for collected packages: edlib
  Building wheel for edlib (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [14 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'edlib' extension
      creating build
      creating build/temp.macosx-11.0-arm64-cpython-312
      creating build/temp.macosx-11.0-arm64-cpython-312/edlib
      creating build/temp.macosx-11.0-arm64-cpython-312/edlib/src
      clang -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/miniconda3/envs/sniffles/include -arch arm64 -fPIC -O2 -isystem /opt/miniconda3/envs/sniffles/include -arch arm64 -Iedlib/include -I/opt/miniconda3/envs/sniffles/include/python3.12 -c edlib.bycython.cpp -o build/temp.macosx-11.0-arm64-cpython-312/edlib.bycython.o -O3 -std=c++11
      edlib.bycython.cpp:198:12: fatal error: 'longintrepr.h' file not found
        #include "longintrepr.h"
                 ^~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for edlib
  Running setup.py clean for edlib
Failed to build edlib
ERROR: Could not build wheels for edlib, which is required to install pyproject.toml-based projects

Then I did
pip install edlib

Collecting edlib
  Using cached edlib-1.3.9.tar.gz (91 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: edlib
  Building wheel for edlib (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [14 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'edlib' extension
      creating build
      creating build/temp.macosx-11.0-arm64-cpython-312
      creating build/temp.macosx-11.0-arm64-cpython-312/edlib
      creating build/temp.macosx-11.0-arm64-cpython-312/edlib/src
      clang -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/miniconda3/envs/sniffles/include -arch arm64 -fPIC -O2 -isystem /opt/miniconda3/envs/sniffles/include -arch arm64 -Iedlib/include -I/opt/miniconda3/envs/sniffles/include/python3.12 -c edlib.bycython.cpp -o build/temp.macosx-11.0-arm64-cpython-312/edlib.bycython.o -O3 -std=c++11
      edlib.bycython.cpp:198:12: fatal error: 'longintrepr.h' file not found
        #include "longintrepr.h"
                 ^~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for edlib
  Running setup.py clean for edlib
Failed to build edlib
ERROR: Could not build wheels for edlib, which is required to install pyproject.toml-based projects

so I tried instead
conda install sniffles=2.3

Channels:
 - bioconda
 - conda-forge
 - defaults
Platform: osx-arm64
Collecting package metadata (repodata.json): done
Solving environment: failed

LibMambaUnsatisfiableError: Encountered problems while solving:
  - nothing provides pysam >=0.21.0 needed by sniffles-2.3.2-pyhdfd78af_0

Could not solve for environment specs
The following packages are incompatible
└─ sniffles 2.3**  is not installable because there are no viable options
   ├─ sniffles 2.3.2 would require
   │  └─ pysam >=0.21.0 , which does not exist (perhaps a missing channel);
   └─ sniffles [2.3.2|2.3.3] would require
      └─ edlib >=1.3.9 , which does not exist (perhaps a missing channel).

pip show pysam

Name: pysam
Version: 0.22.1
Summary: Package for reading, manipulating, and writing genomic data
Home-page: https://github.com/pysam-developers/pysam
Author: Andreas Heger
Author-email: Andreas Heger <[email protected]>
License: MIT License
Location: /opt/miniconda3/envs/sniffles/lib/python3.12/site-packages
Requires: 
Required-by:

pip show edlib
WARNING: Package(s) not found: edlib

pip install edlib

Collecting edlib
  Using cached edlib-1.3.9.tar.gz (91 kB)
  Preparing metadata (setup.py) ... done
Building wheels for collected packages: edlib
  Building wheel for edlib (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [14 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'edlib' extension
      creating build
      creating build/temp.macosx-11.0-arm64-cpython-312
      creating build/temp.macosx-11.0-arm64-cpython-312/edlib
      creating build/temp.macosx-11.0-arm64-cpython-312/edlib/src
      clang -fno-strict-overflow -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /opt/miniconda3/envs/sniffles/include -arch arm64 -fPIC -O2 -isystem /opt/miniconda3/envs/sniffles/include -arch arm64 -Iedlib/include -I/opt/miniconda3/envs/sniffles/include/python3.12 -c edlib.bycython.cpp -o build/temp.macosx-11.0-arm64-cpython-312/edlib.bycython.o -O3 -std=c++11
      edlib.bycython.cpp:198:12: fatal error: 'longintrepr.h' file not found
        #include "longintrepr.h"
                 ^~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for edlib
  Running setup.py clean for edlib
Failed to build edlib
ERROR: Could not build wheels for edlib, which is required to install pyproject.toml-based projects

pip show wheel

Name: wheel
Version: 0.43.0
Summary: A built-package format for Python
Home-page: 
Author: 
Author-email: Daniel Holth <[email protected]>
License: 
Location: /opt/miniconda3/envs/sniffles/lib/python3.12/site-packages
Requires: 
Required-by:
@hermannromanek
Copy link
Collaborator

Hi @Machadum

While both pysam and psutil installed successfully, it looks like there are no prebuilt binaries for your environment for edlib available on pip, and building failed due to missing header files (which seems to be caused by the move in python/cpython#28968).

While we added edlib as a requirement to improve population calling, it isnt strictly neccessary - you should be able to run sniffles without it being installed as it is not used for single sample calling.

HTH,
Hermann

@Machadum
Copy link
Author

Machadum commented Jul 5, 2024 •
edited
Loading

Hi thank you for your quick answer. Unfortunately I have been having a heck of a time trying to get installation work. Could you please help me with this?

@hermannromanek
Copy link
Collaborator

Sure...you can try installing sniffles without dependencies:

pip install --no-deps sniffles

This will install sniffles without installing dependencies. Since you already have pysam and psutil installed, this will skip edlib. Once installed, you should be able to run sniffles normally.

Multisample merging also works without edlib, although it may overmerge a bit, alignments and the --combine-pctseq parameter define one more measurement to split otherwise similar variants.

HTH,
Hermann

@Machadum
Copy link
Author

Machadum commented Jul 8, 2024

Thank you very much for your valuable help Hermann! I managed to install it but it still does not work, please see below:

(sniffles) RB07:Desktop ipmc$ sniffles --input /Volumes/Rita/Project/Pilot/Pilot_bam_pass_merged_sorted.bam --vcf SL_7-1.vcf --threads 24
2024-07-08 10:00:36,362 INFO sniffles.main (47403): Running Sniffles2, build 2.3.3
2024-07-08 10:00:36,362 INFO sniffles.main (47403):   Run Mode: call_sample
2024-07-08 10:00:36,362 INFO sniffles.main (47403):   Start on: 2024/07/08 10:00:36
2024-07-08 10:00:36,362 INFO sniffles.main (47403):   Working dir: /Users/ipmc/Desktop
2024-07-08 10:00:36,362 INFO sniffles.main (47403):   Used command: /opt/miniconda3/envs/sniffles/bin/sniffles --input /Volumes/Rita/Project/Pilot/Pilot_bam_pass_merged_sorted.bam --vcf SL_7-1.vcf --threads 24
2024-07-08 10:00:36,362 INFO sniffles.main (47403): ==============================
2024-07-08 10:00:36,382 INFO sniffles.main (47403): Opening for reading: /Volumes/Rita/Project/Pilot/Pilot_bam_pass_merged_sorted.bam
2024-07-08 10:00:36,616 INFO sniffles.main (47403): Opening for writing: SL_7-1.vcf (single-sample, sorted)
2024-07-08 10:00:36,620 INFO sniffles.main (47403): 
2024-07-08 10:00:36,620 INFO sniffles.main (47403): Analyzing 23324136 alignments total...
2024-07-08 10:00:36,620 INFO sniffles.main (47403): 
2024-07-08 10:00:36,620 INFO sniffles.worker (47403): Starting worker 0
2024-07-08 10:00:36,630 ERROR sniffles.main (47403): Unhandled error while running sniffles.
Traceback (most recent call last):
  File "/opt/miniconda3/envs/sniffles/bin/sniffles", line 516, in <module>
    Sniffles2_Main(processes)
  File "/opt/miniconda3/envs/sniffles/bin/sniffles", line 435, in Sniffles2_Main
    p.start()
  File "/opt/miniconda3/envs/sniffles/lib/python3.12/site-packages/sniffles/parallel.py", line 490, in start
    self.process.start()
  File "/opt/miniconda3/envs/sniffles/lib/python3.12/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
                  ^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/sniffles/lib/python3.12/multiprocessing/context.py", line 224, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/sniffles/lib/python3.12/multiprocessing/context.py", line 289, in _Popen
    return Popen(process_obj)
           ^^^^^^^^^^^^^^^^^^
  File "/opt/miniconda3/envs/sniffles/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/opt/miniconda3/envs/sniffles/lib/python3.12/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/opt/miniconda3/envs/sniffles/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 47, in _launch
    reduction.dump(process_obj, fp)
  File "/opt/miniconda3/envs/sniffles/lib/python3.12/multiprocessing/reduction.py", line 60, in dump
    ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle '_thread.RLock' object

@hermannromanek
Copy link
Collaborator

So this seems to finally be a man specific issue :(

See also #480 - we don't "officially" support or test on Mac. Nevertheless this issue is probably not too hard to fix, I would need you to test it though - I don't have any Macs available for testing.

I'll try to create a branch for this tonight, you'll have to do a git pull from this branch. Is this possible?

Thanks, Hermann

@Machadum
Copy link
Author

Machadum commented Jul 8, 2024 •
edited
Loading

I do not know how to do a git pull but I will do my best and follow your instructions. Thank you again for your help!
Otherwise I have an other mac on which I installed sniffles 1.0.7 by mistake. Since then, I also installed the last version but the old one is running by default and but I do not know how to remove it.

pip install --no-deps sniffles 

Collecting sniffles
Using cached sniffles-2.3.3-py3-none-any.whl.metadata (5.1 kB)
Using cached sniffles-2.3.3-py3-none-any.whl (57 kB)
Installing collected packages: sniffles
Successfully installed sniffles-2.3.3


sniffles --version
sniffles  version: 1.0.7

@hermannromanek
Copy link
Collaborator

Seems like sniffles was installed on your path. You can determine which program is run by doing which sniffles, which will point you to where it is installed.

Uninstall probably depends on how you installed it, and may be as simple as removing the executable.

hermannromanek referenced this issue Jul 8, 2024
Set start method to fork
df523e5
@hermannromanek
Copy link
Collaborator

I also checked out the other issue - with Python 3.8, on Mac the default start method for multiprocessing was changed from fork to spawn. I am not sure if forking still works on Mac, but feel free to give it a try:

In the main sniffles file, you'll need to insert the following two lines after line 491 to this:

Sniffles/src/sniffles/sniffles

Lines 491 to 494 in df523e5

if __name__ == "__main__":
import multiprocessing as mp
mp.set_start_method('fork')
processes = []

This is probably easier if you are not familiar with git. It changes pythons behaviour for sniffles to the same as on Unix.

Hermann

@Machadum
Copy link
Author

Machadum commented Jul 9, 2024

It works! Awesome! Thank you so much for having the patience to help me through this process.

@hermannromanek
Copy link
Collaborator

Glad to hear it :)

I'll take over the commit to our main branch - thank you for testing it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Machadum @hermannromanek