Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update install scripts using the Conda env manager #53

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
143 changes: 105 additions & 38 deletions doc/install-linux.md
Original file line number Diff line number Diff line change
@@ -1,63 +1,130 @@
# Linux Installation
# Linux Installation Instructions

To install the fluxon-mhd modeling framework, there are three main requirements - a perl installation with libraries, the perl data language toolset, and finally compiling the fluxon-mhd software itself.
These instructions were created by installing FLUX on two different linux servers running AlmaLinux. Please let us know if anything breaks! Extra troubleshooting steps are listed at the bottom.

Here we'll outline steps for installing on a Linux system - notably tested on an Ubuntu system. The broad steps can be applied for other flavors of Linux.
## 1. Get Conda Environment Manager

## Perl installation

First, use apt-get to install cpanminus, an easier tool to use for installing perl packages.
```shell
sudo apt-get install cpanminus
```sh
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh
```

Install PDL using cpanminus, forcing to bypass any dependency warnings.
```shell
sudo cpanm --force PDL
```
Follow the prompts:
- Hold down the arrow key and then accept terms.
- Accept Conda activation at startup.
- Exit and reconnect to the SSH session/ shell session.

## 2. Create Conda Environment and Get Dependencies

If problems are encountered with that PDL installation route, one can install an older version of PDL using apt-get. This may run into code compilation issues down the road.
```shell
sudo apt-get install pdl
```sh
conda create -n fluxenv
conda activate fluxenv
conda install -c conda-forge gcc gxx binutils c-compiler gsl perl perl-app-cpanminus perl-extutils-makemaker make cmake automake autoconf libtool m4 patch libxcrypt gnuplot cairo pango qt pfsspy sqlite
```

Write the following to ~/.perldlrc, making sure to reference the appropriate location local perl libraries will be installed to.
```shell
push(@INC, "/usr/local/lib/aarch64-linux-gnu/perl/5.38.2");
Confirm installation by typing `y` when prompted.

require('PDL/default.perldlrc');
## 3. Install the Perl Data Language (PDL) and Dependencies

use PDL::AutoLoader;
$PDL::AutoLoader::Rescan=1;
```sh
cpanm Alien::Build::Plugin::Gather::Dino Capture::Tiny Chart::Gnuplot Config::IniFiles Devel::CheckLib File::HomeDir File::Map File::ShareDir File::ShareDir::Install File::Which Inline Inline::C Inline::Python List::MoreUtils Math::GSL Math::GSL::Alien Math::Interpolate Math::Interpolator Math::RungeKutta Moo::Role Net::SSLeay PDL PDL::GSL PDL::GSL::INTEG PDL::Graphics::Gnuplot PDL::Graphics::Simple Parallel::ForkManager Term::ReadKey Test::Builder Text::CSV local::lib
```

1;
If the system struggles with `File::Map`, try:

```sh
export LDFLAGS=""
export CFLAGS="-O2"
export PERL_LDFLAGS=""
export PERL_CFLAGS="-O2"
export LD=x86_64-conda-linux-gnu-gcc
export CC=x86_64-conda-linux-gnu-gcc
```

A few other packages of note will be needed for the later fluxon-mhd install.
```shell
sudo cpanm File::ShareDir
sudo cpanm Math::RungeKutta
sudo cpanm Term::ReadKey
```
If `Inline::C` isn’t found, run:

Optionally, if you need graphics, install gnuplot
```shell
sudo apt-get install gnuplot
sudo cpanm PDL::Graphics::Gnuplot
```sh
cpanm Inline::C
```

## FLUX installation
## 4. Get FLUX

To build some of the C code, make sure you have a compiler installed.
```shell
sudo apt-get install gcc
Set the prefixes for a conda installation:

```sh
export PL_PREFIX="$CONDA_PREFIX/lib/perl5/site_perl"
export FL_PREFIX="$CONDA_PREFIX"
mkdir -p "$FL_PREFIX"
```

If you haven't already, clone the fluxon-mhd repository and then make everything from the base directory.
```shell
Clone and configure Flux:

```sh
git clone https://github.com/lowderchris/fluxon-mhd.git
cd fluxon-mhd
make everything
```

If Flux.pm isn't found at the end of the install, try:

```sh
export PERL5LIB=$(dirname $(find $CONDA_PREFIX -name Flux.pm)):$PERL5LIB
make everything
```
To set up the necessary environment variables and update the appropriate configuration files, please download or navigate to the provided script located at [fluxon-mhd/doc/set_paths.sh](https://github.com/lowderchris/fluxon-mhd/blob/documentation/doc/set_paths.sh). Run the script, which will automatically add the Perl autoloader and Flux installation paths to your conda environment configuration files.

```sh
# Download the script
wget https://github.com/lowderchris/fluxon-mhd/raw/documentation/doc/set_paths.sh

# Run the script
bash set_paths.sh
```

Confirm installation success:

```sh
perl -e "use Flux;"
```

## 5. Get Fluxpype Wrapper (To run FLUX with)

```sh
cd ..
git clone https://github.com/GillySpace27/fluxpype.git
cd fluxpype
git checkout pythonify
pip install -e .
```

If the GCC version is too old, fix it with:

```sh
export CC=$(which gcc)
export CXX=$(which g++)
```

## 6. Configure the `config.ini` File

```sh
nano fluxpype/config.ini
```

Set the code to do your desired run using the documentation (TBD)

Run the configuration run script:

```sh
python fluxpype/config_runner.py
```
- For the paths to work out, the config runner must be called from one directory above the file.


## FAQs & Troubleshooting
- If gnuplot is not detected correctly in PDL, try the following, and if it works, put this line into either the conda activation script or the bashrc file:
- ```export GNUPLOT_BINARY=$(realpath $(which gnuplot))```

- If GSL is not found, use
- ```conda install gsl```
- ```cpanm --notest PDL::GSL```

186 changes: 129 additions & 57 deletions doc/install-macos.md
Original file line number Diff line number Diff line change
@@ -1,92 +1,164 @@
# macOS Installation
# macOS Installation Instructions

To install the fluxon-mhd modeling framework, there are three main requirements - a perl installation with libraries, the perl data language toolset, and finally compiling the fluxon-mhd software itself.
These instructions were created by adapting the linux conda install to macOS. Please let us know if anything breaks!
Extra troubleshooting steps are listed at the bottom.

Homebrew and the base perl installation will be installed as a user that has full system access, with further packages and flux itself installed locally. This provides both a buffer if the system perl is updated, and also allows regular access for users who do not have regular administrative privileges. Note that if you have an admin user account, you can also install everything within the homebrew perl library directory, updating paths below accordingly.
## 1. Get Conda Environment Manager
Depending on your mac's architecture, either get
```sh
curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh
bash ~/Miniconda3-latest-MacOSX-arm64.sh
```

## Perl installation
or

First, install the homebrew package, which will require being logged in as an admin user. Follow through the installation process.
```shell
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
``` sh
curl -O https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
bash ~/Miniconda3-latest-MacOSX-x86_64.sh
```

Make sure at the end of this to follow the instructions for adding homebrew to your path
```shell
(echo; echo 'eval "$(/opt/homebrew/bin/brew shellenv)"') >> /Users/<USERNAME>/.zprofile
eval "$(/opt/homebrew)
Follow the prompts:
- Hold down the arrow key and then accept terms.
- Accept Conda activation at startup.
- Exit and reconnect to the SSH session/ shell session.

## 2. Create Conda Environment and Get Dependencies

```sh
conda create -n fluxenv
conda activate fluxenv
conda install -c conda-forge c-compiler gsl perl perl-app-cpanminus perl-extutils-makemaker make cmake automake autoconf libtool m4 patch libxcrypt gnuplot cairo pango qt pfsspy sqlite
```

Next, use homebrew as an admin user to install perl.
```shell
brew install perl
Confirm installation by typing `y` when prompted.

## 3. Install the Perl Data Language (PDL) and Dependencies

```sh
cpanm Alien::Build::Plugin::Gather::Dino Capture::Tiny Chart::Gnuplot Config::IniFiles Devel::CheckLib File::HomeDir File::Map File::ShareDir File::ShareDir::Install File::Which Inline Inline::C Inline::Python List::MoreUtils Math::GSL Math::GSL::Alien Math::Interpolate Math::Interpolator Math::RungeKutta Moo::Role Net::SSLeay PDL PDL::GSL PDL::GSL::INTEG PDL::Graphics::Gnuplot PDL::Graphics::Simple Parallel::ForkManager Term::ReadKey Test::Builder Text::CSV local::lib
```

If you plan on using homebrew for other packages, and would like to keep perl from updating alongside other packages, the installation can be frozen.
```shell
brew pin perl
If the system struggles with `File::Map`, try:

```sh
export LDFLAGS=""
export CFLAGS="-O2"
export PERL_LDFLAGS=""
export PERL_CFLAGS="-O2"
export LD=x86_64-conda-linux-gnu-gcc
export CC=x86_64-conda-linux-gnu-gcc
```

Two more homebrew packages will be needed later, which can also be installed via an admin user.
```shell
brew install gnuplot
brew install fftw
If `Inline::C` isn’t found, run:

```sh
cpanm Inline::C
```

The final suggested step as an admin user is to setup the cpanminus tool for installing perl packages.
```shell
brew install cpanminus
## 4. Get FLUX

Set the prefixes for a conda installation:

```sh
export PL_PREFIX="$CONDA_PREFIX/lib/perl5/site_perl"
export FL_PREFIX="$CONDA_PREFIX"
mkdir -p "$FL_PREFIX"
```

Set perl to install packages to a local library directory. Here we've chosen to use a perl5 directory inside the ~/Library folder, but this could be anywhere your user has permissions. This is then stored to automatically update in a .zprofile startup script for Zsh - modify if using a different shell.
```shell
PERL_MM_OPT="INSTALL_BASE=~/Library/perl5" cpanm --local-lib=~/Library/perl5 local::lib
echo 'eval `perl -I ~/Library/perl5/lib/perl5 -Mlocal::lib=~/Library/perl5`' >> ~/.zprofile
source ~/.zprofile
Clone and configure Flux:

```sh
git clone https://github.com/lowderchris/fluxon-mhd.git
cd fluxon-mhd
make everything
```

Write the following to ~/.perldlrc, making sure to reference the appropriate location local perl libraries will be installed to.
```shell
push(@INC, "/Users/<USERNAME>/Library/perl5/lib/perl5/5.38.2");
If Flux.pm isn't found at the end of the install, try:

```sh
export PERL5LIB=$(dirname $(find $CONDA_PREFIX -name Flux.pm)):$PERL5LIB
make everything
```
To set up the necessary environment variables and update the appropriate configuration files, please download or navigate to the provided script located at [fluxon-mhd/doc/set_paths.sh](https://github.com/lowderchris/fluxon-mhd/blob/documentation/doc/set_paths.sh). Run the script, which will automatically add the Perl autoloader and Flux installation paths to your conda environment configuration files.

require('PDL/default.perldlrc');
```sh
# Download the script
wget https://github.com/lowderchris/fluxon-mhd/raw/documentation/doc/set_paths.sh

use PDL::AutoLoader;
$PDL::AutoLoader::Rescan=1;
# Run the script
bash set_paths.sh
```

1;
Confirm installation success:

```sh
perl -e "use Flux;"
```

## Perl Data Language (PDL) installation
## 5. Get Fluxpype Wrapper (To run FLUX with)

Using cpanm, install PDL and required depedencies.
```shell
cpanm PDL
```sh
cd ..
git clone https://github.com/GillySpace27/fluxpype.git
cd fluxpype
git checkout pythonify
pip install -e .
```

A few other packages of note will be needed for the later fluxon-mhd install.
```shell
cpanm File::ShareDir
cpanm PDL::Graphics::Gnuplot
cpanm Math::RungeKutta
cpanm Term::ReadKey
If the GCC version is too old, fix it with:

```sh
export CC=$(which gcc)
export CXX=$(which g++)
```

## FLUX installation
## 6. Configure the `config.ini` File

Finally, for the first install, and for future recompilations, set the perl and flux prefix paths.
```shell
export PL_PREFIX='/Users/<USERNAME>/Library/perl5'
export FL_PREFIX='/Users/<USERNAME>/Library/flux'
```sh
nano fluxpype/config.ini
```

If you haven't already, clone the fluxon-mhd repository and then make everything from the base directory.
```shell
git clone https://github.com/lowderchris/fluxon-mhd.git
cd fluxon-mhd
Set the code to do your desired run using the documentation (TBD)

make everything
Run the configuration run script:

```sh
python fluxpype/config_runner.py
```
- For the paths to work out, the config runner must be called from one directory above the file.


## FAQs & Troubleshooting
- If an error says that a perl file cannot be found, make sure there isn't a syntax error somewhere in that file, because it will say it cannot find it when it means it cannot compile it.

- If gnuplot is not detected correctly in PDL, try the following, and if it works, put this line into either the conda activation script or the bashrc file:
- ```export GNUPLOT_BINARY=$(realpath $(which gnuplot))```

- If GSL is not found, use
- ```conda install gsl```
- ```cpanm --notest PDL::GSL```

- If net-SSLeay fails to install, try this
- ```conda install openssl=1.1.1```
- ```cpanm PDL::Graphics::Gnuplot```

- If Math::GSL::Alien fails to install, try
- ```conda install gsl```
- ```cpanm --notest PDL::GSL```


- if the wrong cpanm is getting in the way, delete it using the wrong path as follows
- ```export PATH=$(echo "$PATH" | sed 's|/Users/cgilbert/perl5/perlbrew/bin:||')```
- then try to use the huge cpanm installation command above
- then run the set_paths.sh file
- activate the environment
- make everything

- if the paths seem completely wrong, try these in the fluxon-mhd directory
- ```make uninstall```
- ```make clean```

- if the error is: Can't locate Alien/Gnuplot.pm in @INC
- ```cpanm Alien::Gnuplot```

After this, make sure to follow the printed instructions for appending your PDLLIB path, and .perldlrc file.
- if fluxpype cannot find the run file, check the paths at the top of ```config.ini``` to make sure that the path is correct.
Loading