Skip to content

Commit

Permalink
Update documentation for spack-stack-1.4.0 release part 3 (#607)
Browse files Browse the repository at this point in the history
* Remove external [email protected] from jet/orion/s4 site configs

* Second round of updates in doc/source/KnownIssues.rst and doc/source/PreConfiguredSites.rst for spack-stack 1.4.0 installations

* Update doc/source/NewSiteConfigs.rst for Ubuntu and Red Hat, and update doc/source/PreConfiguredSites.rst for Orion

* Add linker error info to doc/source/NewSiteConfigs.rst

* Remove TODO from doc/source/PreConfiguredSites.rst
  • Loading branch information
climbfuji authored May 26, 2023
1 parent 84e0f1d commit 02095bc
Show file tree
Hide file tree
Showing 2 changed files with 46 additions and 45 deletions.
34 changes: 24 additions & 10 deletions doc/source/NewSiteConfigs.rst
Original file line number Diff line number Diff line change
Expand Up @@ -235,11 +235,11 @@ Remember to activate the ``lua`` module environment and have MacTeX in your sear
7. Set default compiler and MPI library (make sure to use the correct ``apple-clang`` version for your system and the desired ``openmpi`` version)

.. code-block:: console
# Check your clang version then add it to your site compiler config.
clang --version
spack config add "packages:all:compiler:[apple-clang@YOUR-VERSION]"
spack config add "packages:all:providers:mpi:[[email protected]]"
8. If applicable (depends on the environment), edit the main config file for the environment and adjust the compiler matrix to match the compilers for macOS, as above:

Expand Down Expand Up @@ -454,30 +454,44 @@ It is recommended to increase the stacksize limit by using ``ulimit -S -s unlimi
7. Set default compiler and MPI library (make sure to use the correct ``gcc`` version for your system and the desired ``openmpi`` version)

.. code-block:: console
# Check your gcc version then add it to your site compiler config.
gcc --version
spack config add "packages:all:compiler:[gcc@YOUR-VERSION]"
# Example for Red Hat 8 following the above instructions
spack config add "packages:all:providers:mpi:[[email protected]]"
# Example for Ubuntu 20.04 or 22.04 following the above instructions
spack config add "packages:all:providers:mpi:[[email protected]]"
8. If you have manually installed lmod, you will need to update the site module configuration to use lmod instead of tcl. Skip this step if you followed the Ubuntu or Red Hat instructions above.
.. warning::
On some systems, the default compiler (e.g., ``gcc`` on Ubuntu 20) may not get used by spack if a newer version is found. Compare your entry to the output of the concretization step later and adjust the entry, if necessary.

8. Set a few more package variants and versions to avoid linker errors and duplicate packages being built (for both Red Hat and Ubuntu):

.. code-block:: console
spack config add "packages:fontconfig:variants:+pic"
spack config add "packages:pixman:variants:+pic"
spack config add "packages:cairo:variants:+pic"
spack config add "packages:libffi:version:[3.3]"
spack config add "packages:flex:version:[2.6.4]"
9. If you have manually installed lmod, you will need to update the site module configuration to use lmod instead of tcl. Skip this step if you followed the Ubuntu or Red Hat instructions above.

.. code-block:: console
sed -i 's/tcl/lmod/g' envs/jedi-ufs.mylinux/site/modules.yaml
9. If applicable (depends on the environment), edit the main config file for the environment and adjust the compiler matrix to match the compilers for Linux, as above:
10. If applicable (depends on the environment), edit the main config file for the environment and adjust the compiler matrix to match the compilers for Linux, as above:

.. code-block:: console
definitions:
- compilers: ['%gcc']
10. Edit site config files and common config files, for example to remove duplicate versions of external packages that are unwanted, add specs in ``envs/jedi-ufs.mylinux/spack.yaml``, etc.
11. Edit site config files and common config files, for example to remove duplicate versions of external packages that are unwanted, add specs in ``envs/jedi-ufs.mylinux/spack.yaml``, etc.

.. warning::
**Important:** Remove any external ``[email protected]`` package from ``envs/jedi-ufs.mylinux/site/packages.yaml``. It is in fact recommended to remove all versions of ``cmake`` up to ``3.20``. Further, on Red Hat/CentOS, remove any external curl that might have been found.
Expand All @@ -488,23 +502,23 @@ It is recommended to increase the stacksize limit by using ``ulimit -S -s unlimi
vi envs/jedi-ufs.mylinux/common/*.yaml
vi envs/jedi-ufs.mylinux/site/*.yaml
11. Process the specs and install
12. Process the specs and install

.. code-block:: console
spack concretize
spack install [--verbose] [--fail-fast]
12. Create tcl module files
13. Create tcl module files (replace ``tcl`` with ``lmod`` if you have manually installed lmod)

.. code-block:: console
spack module tcl refresh
13. Create meta-modules for compiler, mpi, python
14. Create meta-modules for compiler, mpi, python

.. code-block:: console
spack stack setup-meta-modules
14. You now have a spack-stack environment that can be accessed by running ``module use ./envs/jedi-ufs.mymacos/install/modulefiles/Core``. The modules defined here can be loaded to build and run code as described in :numref:`Section %s <UsingSpackEnvironments>`.
15. You now have a spack-stack environment that can be accessed by running ``module use ./envs/jedi-ufs.mymacos/install/modulefiles/Core``. The modules defined here can be loaded to build and run code as described in :numref:`Section %s <UsingSpackEnvironments>`.
57 changes: 22 additions & 35 deletions doc/source/PreConfiguredSites.rst
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Ready-to-use spack-stack 1.4.0 installations are available on the following, ful
+------------------------------------------------------------+-------------------------------+--------------------------------------------------------------------------------------------------------------+
| System | Maintainers | Location |
+============================================================+===============================+==============================================================================================================+
| MSU Orion Intel/GNU | Cam Book / Dom Heinzeller | ``/work/noaa/epic-ps/role-epic-ps/spack-stack/spack-stack-1.3.0/envs/unified-env`` |
| MSU Orion Intel/GNU | Cam Book / Dom Heinzeller | ``/work/noaa/epic-ps/role-epic-ps/spack-stack/spack-stack-1.4.0/envs/unified-env`` |
+------------------------------------------------------------+-------------------------------+--------------------------------------------------------------------------------------------------------------+
| MSU Hercules Intel/GNU^* | Cam Book / Dom Heinzeller | ``/work/noaa/epic-ps/role-epic-ps/spack-stack/spack-stack-1.4.0-hercules/envs/unified-env`` |
+------------------------------------------------------------+-------------------------------+--------------------------------------------------------------------------------------------------------------+
Expand Down Expand Up @@ -50,7 +50,9 @@ Ready-to-use spack-stack 1.4.0 installations are available on the following, ful
+------------------------------------------------------------+-------------------------------+--------------------------------------------------------------------------------------------------------------+
| Amazon Web Services Parallelcluster Ubuntu 20.04 Intel/GNU | Dom Heinzeller / ??? | ``/mnt/experiments-efs/skylab-v5/spack-stack-1.4.0/envs/unified-env`` |
+------------------------------------------------------------+-------------------------------+--------------------------------------------------------------------------------------------------------------+
| Amazon Web Services AMI Red Hat 8 GNU | Dom Heinzeller / ??? | ``/home/ec2-user/spack-stack/spack-stack-1.3.0/envs/unified-env`` |
| Amazon Web Services AMI Red Hat 8 GNU | Dom Heinzeller / ??? | ``/home/ec2-user/spack-stack/spack-stack-1.4.0/envs/unified-env`` |
+------------------------------------------------------------+-------------------------------+--------------------------------------------------------------------------------------------------------------+
| Amazon Web Services AMI Ubuntu 20 GNU | Dom Heinzeller / ??? | ``/home/ubuntu/spack-stack/spack-stack-1.4.0/envs/unified-env`` |
+------------------------------------------------------------+-------------------------------+--------------------------------------------------------------------------------------------------------------+

^* This system uses a different wgrib2 version 3.1.1 than the default 2.0.8.
Expand Down Expand Up @@ -79,41 +81,21 @@ The following is required for building new spack environments and for using spac
module load ecflow/5.8.4
module load mysql/8.0.31
For ``spack-stack-1.3.0`` with Intel, load the following modules after loading miniconda and ecflow:

.. code-block:: console
module use /work/noaa/epic-ps/role-epic-ps/spack-stack/spack-stack-1.3.0/envs/unified-env/install/modulefiles/Core
module load stack-intel/2022.0.2
module load stack-intel-oneapi-mpi/2021.5.1
module load stack-python/3.9.7
module available
For ``spack-stack-1.3.0`` with GNU, load the following modules after loading miniconda and ecflow:

.. code-block:: console
module use /work/noaa/epic-ps/role-epic-ps/spack-stack/spack-stack-1.3.0/envs/unified-env/install/modulefiles/Core
module load stack-gcc/10.2.0
module load stack-openmpi/4.0.4
module load stack-python/3.9.7
module available
For ``spack-stack-1.3.1`` with Intel, load the following modules after loading miniconda and ecflow:
For ``spack-stack-1.4.0`` with Intel, load the following modules after loading miniconda and ecflow:

.. code-block:: console
module use /work/noaa/epic-ps/role-epic-ps/spack-stack/spack-stack-1.3.1/envs/unified-env/install/modulefiles/Core
module use /work/noaa/epic-ps/role-epic-ps/spack-stack/spack-stack-1.4.0/envs/unified-env/install/modulefiles/Core
module load stack-intel/2022.0.2
module load stack-intel-oneapi-mpi/2021.5.1
module load stack-python/3.9.7
module available
For ``spack-stack-1.3.1`` with GNU, load the following modules after loading miniconda and ecflow:
For ``spack-stack-1.4.0`` with GNU, load the following modules after loading miniconda and ecflow:

.. code-block:: console
module use /work/noaa/epic-ps/role-epic-ps/spack-stack/spack-stack-1.3.1/envs/unified-env/install/modulefiles/Core
module use /work/noaa/epic-ps/role-epic-ps/spack-stack/spack-stack-1.4.0/envs/unified-env/install/modulefiles/Core
module load stack-gcc/10.2.0
module load stack-openmpi/4.0.4
module load stack-python/3.9.7
Expand Down Expand Up @@ -659,29 +641,34 @@ For ``spack-stack-1.4.0`` with GNU, run the following commands/load the followin
Amazon Web Services Red Hat 8
-----------------------------

Use a c6i.4xlarge instance or similar with AMI "skylab-4.0.0-redhat8-update-20230419" (ami-01e025b0334795ffa in region us-east-1, ami-094361e081de18589 in region us-east-2).
Use a c6i.4xlarge instance or similar with AMI "skylab-5.0.0-redhat8" (ami-02324faac94a9cac9 in region us-east-1, ami-038d9beca351f9005 in region us-east-2).

For ``spack-stack-1.3.0``, run:
For ``spack-stack-1.4.0``, run:

.. code-block:: console
ulimit -s unlimited
scl enable gcc-toolset-11 bash
module use /home/ec2-user/spack-stack/spack-stack-1.3.0/envs/unified-env/install/modulefiles/Core
module use /home/ec2-user/spack-stack/spack-stack-1.4.0/envs/unified-env/install/modulefiles/Core
module load stack-gcc/11.2.1
module load stack-openmpi/4.1.4
module load stack-openmpi/4.1.5
module load stack-python/3.10.8
module available
For ``spack-stack-1.3.1``, run:
-----------------------------
Amazon Web Services Ubuntu 20
-----------------------------

Use a c6i.4xlarge instance or similar with AMI "skylab-5.0.0-ubuntu20" (ami-09a8c9d3775feafcf in region us-east-1, ami-03e47cdb4ced34d7e in region us-east-2).

For ``spack-stack-1.4.0``, run:

.. code-block:: console
ulimit -s unlimited
scl enable gcc-toolset-11 bash
module use /home/ec2-user/spack-stack/spack-stack-1.3.1/envs/unified-env/install/modulefiles/Core
module load stack-gcc/11.2.1
module load stack-openmpi/4.1.4
module use /home/ubuntu/spack-stack/spack-stack-1.4.0/envs/unified-env/install/modulefiles/Core
module load stack-gcc/10.3.0
module load stack-openmpi/4.1.5
module load stack-python/3.10.8
module available
Expand Down

0 comments on commit 02095bc

Please sign in to comment.