From 6ea87a58d3e4eb7e802b7f63dfdb6d4697b2f46f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E2=80=9Cgspetro-NOAA=E2=80=9D?= Date: Wed, 7 Feb 2024 18:07:59 -0500 Subject: [PATCH 1/9] update wflow config docs --- .../CustomizingTheWorkflow/ConfigWorkflow.rst | 45 ++++++++++--------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/docs/UsersGuide/source/CustomizingTheWorkflow/ConfigWorkflow.rst b/docs/UsersGuide/source/CustomizingTheWorkflow/ConfigWorkflow.rst index cbc6ca6b04..f4d24061a8 100644 --- a/docs/UsersGuide/source/CustomizingTheWorkflow/ConfigWorkflow.rst +++ b/docs/UsersGuide/source/CustomizingTheWorkflow/ConfigWorkflow.rst @@ -320,20 +320,14 @@ Directory Parameters ``EXPT_BASEDIR``: (Default: "") The full path to the base directory in which the experiment directory (``EXPT_SUBDIR``) will be created. If this is not specified or if it is set to an empty string, it will default to ``${HOMEdir}/../expt_dirs``, where ``${HOMEdir}`` contains the full path to the ``ufs-srweather-app`` directory. If set to a relative path, the provided path will be appended to the default value ``${HOMEdir}/../expt_dirs``. For example, if ``EXPT_BASEDIR=some/relative/path`` (i.e. a path that does not begin with ``/``), the value of ``EXPT_BASEDIR`` used by the workflow will be ``EXPT_BASEDIR=${HOMEdir}/../expt_dirs/some/relative/path``. -``EXPT_SUBDIR``: (Default: "") - The user-designated name of the experiment directory (*not* its full path). The full path to the experiment directory, which will be contained in the variable ``EXPTDIR``, will be: - - .. code-block:: console - - EXPTDIR="${EXPT_BASEDIR}/${EXPT_SUBDIR}" - - This parameter must be set to a non-null value in the user-defined experiment configuration file (i.e., ``config.yaml``). +``EXPT_SUBDIR``: (Default: 'experiment') + If ``EXPTDIR`` is not specified, ``EXPT_SUBDIR`` represents the name of the experiment directory (*not* the full path). ``EXEC_SUBDIR``: (Default: "exec") The name of the subdirectory of ``ufs-srweather-app`` where executables are installed. ``EXPTDIR``: (Default: ``'{{ [workflow.EXPT_BASEDIR, workflow.EXPT_SUBDIR]|path_join }}'``) - The full path to the experiment directory. By default, this value is equivalent to ``"${EXPT_BASEDIR}/${EXPT_SUBDIR}"``, but the user can define it differently in the configuration file if desired. + The full path to the experiment directory. By default, this value will point to ``"${EXPT_BASEDIR}/${EXPT_SUBDIR}"``, but the user can define it differently in the configuration file if desired. Pre-Processing File Separator Parameters -------------------------------------------- @@ -490,7 +484,7 @@ Experiment Fix File Paths These parameters are associated with the fixed (i.e., static) files. Unlike the file path parameters in :numref:`Section %s `, which pertain to the locations of system data, the parameters in this section indicate fix file paths within the experiment directory (``$EXPTDIR``). -``FIXdir``: (Default: ``'{{ EXPTDIR if rocoto.tasks.get("task_make_grid") else [user.HOMEdir, "fix"]|path_join }}'``) +``FIXdir``: (Default: ``'{{ EXPTDIR }}'``) Location where fix files will be stored for a given experiment. ``FIXam``: (Default: ``'{{ [FIXdir, "fix_am"]|path_join }}'``) @@ -655,8 +649,8 @@ Forecast Parameters Pre-Existing Directory Parameter ------------------------------------ -``PREEXISTING_DIR_METHOD``: (Default: "delete") - This variable determines how to deal with pre-existing directories (resulting from previous calls to the experiment generation script using the same experiment name [``EXPT_SUBDIR``] as the current experiment). This variable must be set to one of three valid values: ``"delete"``, ``"rename"``, ``"reuse"``, or ``"quit"``. The behavior for each of these values is as follows: +``PREEXISTING_DIR_METHOD``: (Default: "quit") + This variable determines how to deal with pre-existing directories (resulting from previous calls to the experiment generation script using the same experiment name [``EXPT_SUBDIR``] as the current experiment). This variable must be set to one of four valid values: ``"delete"``, ``"rename"``, ``"reuse"``, or ``"quit"``. The behavior for each of these values is as follows: * **"delete":** The preexisting directory is deleted and a new directory (having the same name as the original preexisting directory) is created. @@ -1628,16 +1622,19 @@ Verification Parameters Non-default parameters for verification tasks are set in the ``verification:`` section of the ``config.yaml`` file. +General Verification Parameters +--------------------------------- + +``METPLUS_VERBOSITY_LEVEL``: (Default: ``2``) + Logging verbosity level used by METplus verification tools. Valid values: 0 to 5, with 0 quiet and 5 loud. + Templates for Observation Files --------------------------------- This section includes template variables for :term:`CCPA`, :term:`MRMS`, :term:`NOHRSC`, and :term:`NDAS` observation files. -``OBS_CCPA_APCP01h_FN_TEMPLATE``: (Default: ``'{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hrap.conus.gb2'``) - File name template used to obtain the input observation files (in the ``PcpCombine_obs`` tasks) that contain the 1-hour accumulated precipitation (APCP) from which APCP for longer accumulations will be generated. - -``OBS_CCPA_APCPgt01h_FN_TEMPLATE``: (Default: ``'${OBS_CCPA_APCP01h_FN_TEMPLATE}_a${ACCUM_HH}h.nc'``) - File name template used to generate the observation files (in the ``PcpCombine_obs`` tasks) containing accumulated precipitation for accumulation periods longer than 1-hour. +``OBS_CCPA_APCP_FN_TEMPLATE``: (Default: ``'{valid?fmt=%Y%m%d}/ccpa.t{valid?fmt=%H}z.01h.hrap.conus.gb2'``) + File name template for CCPA accumulated precipitation (APCP) observations. This template is used by the workflow tasks that call the METplus *PcpCombine* tool on CCPA obs to find the input observation files containing 1-hour APCP and then generate NetCDF files containing either 1-hour or greater than 1-hour APCP. ``OBS_NOHRSC_ASNOW_FN_TEMPLATE``: (Default: ``'{valid?fmt=%Y%m%d}/sfav2_CONUS_${ACCUM_HH}h_{valid?fmt=%Y%m%d%H}_grid184.grb2'``) File name template for NOHRSC snow observations. @@ -1648,11 +1645,19 @@ This section includes template variables for :term:`CCPA`, :term:`MRMS`, :term:` ``OBS_MRMS_RETOP_FN_TEMPLATE``: (Default: ``'{valid?fmt=%Y%m%d}/EchoTop_18_00.50_{valid?fmt=%Y%m%d}-{valid?fmt=%H%M%S}.grib2'``) File name template for MRMS echo top observations. -``OBS_NDAS_SFCorUPA_FN_TEMPLATE``: (Default: ``'prepbufr.ndas.{valid?fmt=%Y%m%d%H}'``) - File name template for :term:`NDAS` surface and upper air observations. +``OBS_NDAS_ADPSFCorADPUPA_FN_TEMPLATE``: (Default: ``'prepbufr.ndas.{valid?fmt=%Y%m%d%H}'``) + File name template for :term:`NDAS` surface and upper air observations. This template is used by the workflow tasks that call the METplus *Pb2nc* tool on NDAS obs to find the input observation files containing ADP surface (ADPSFC) or ADP upper air (ADPUPA) fields and then generate NetCDF versions of these files. ``OBS_NDAS_SFCorUPA_FN_METPROC_TEMPLATE``: (Default: ``'${OBS_NDAS_SFCorUPA_FN_TEMPLATE}.nc'``) - File name template for NDAS surface and upper air observations after processing by MET's ``pb2nc`` tool (to change format to NetCDF). + File name template for NDAS surface and upper air observations after processing by MET's *pb2nc* tool (to change format to NetCDF). + +``OBS_CCPA_APCP_FN_TEMPLATE_PCPCOMBINE_OUTPUT``: (Default: ``'${OBS_CCPA_APCP_FN_TEMPLATE}_a${ACCUM_HH}h.nc'``) + Template used to specify the names of the output NetCDF observation files generated by the workflow verification tasks that call the METplus *PcpCombine* tool on CCPA observations. (These files will contain observations of accumulated precipitation [APCP], both for 1 hour and for > 1 hour accumulation periods, in NetCDF format.) + +``OBS_NDAS_ADPSFCorADPUPA_FN_TEMPLATE_PB2NC_OUTPUT``: (Default: ``'${OBS_NDAS_ADPSFCorADPUPA_FN_TEMPLATE}.nc'``) + Template used to specify the names of the output NetCDF observation files generated by the workflow verification tasks that call the METplus Pb2nc tool on NDAS observations. (These files will contain obs ADPSFC or ADPUPA fields in NetCDF format.) + + VX Forecast Model Name ------------------------ From be2fac63ea0e990c922b550ab1982b8ba04a46b0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E2=80=9Cgspetro-NOAA=E2=80=9D?= Date: Wed, 7 Feb 2024 18:08:21 -0500 Subject: [PATCH 2/9] update path to UW docs --- docs/UsersGuide/source/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/UsersGuide/source/conf.py b/docs/UsersGuide/source/conf.py index 2c51953f61..dcac0d30e5 100644 --- a/docs/UsersGuide/source/conf.py +++ b/docs/UsersGuide/source/conf.py @@ -233,7 +233,7 @@ def setup(app): 'ccpp-techdoc': ('https://ccpp-techdoc.readthedocs.io/en/ufs_srw_app_v2.2.0/', None), 'stochphys': ('https://stochastic-physics.readthedocs.io/en/latest/', None), 'srw_v2.2.0': ('https://ufs-srweather-app.readthedocs.io/en/release-public-v2.2.0/', None), - 'uw': ('https://uwtools.readthedocs.io/en/develop', None), + 'uw': ('https://uwtools.readthedocs.io/en/main', None), } # -- Options for todo extension ---------------------------------------------- From 98f7386fc8c3d01d74488be968c371b30bbced9c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E2=80=9Cgspetro-NOAA=E2=80=9D?= Date: Wed, 7 Feb 2024 18:08:47 -0500 Subject: [PATCH 3/9] update flow of container chapter; modify xlinks --- .../ContainerQuickstart.rst | 127 ++++++++++-------- .../source/BuildingRunningTesting/RunSRW.rst | 4 +- 2 files changed, 75 insertions(+), 56 deletions(-) diff --git a/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst b/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst index 9eafa7c5f3..9ec125c9db 100644 --- a/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst +++ b/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst @@ -21,54 +21,48 @@ Download the Container Prerequisites ------------------- -Users must have an **Intel** compiler and :term:`MPI` (available for free `here `__) in order to run the SRW App in the container provided using the method described in this chapter. Additionally, it is recommended that users install the `Rocoto workflow manager `__ on their system in order to take advantage of automated workflow options. Although it is possible to run an experiment without Rocoto, and some tips are provided, the only fully-supported and tested container option assumes that Rocoto is pre-installed. +**Intel Compiler and MPI** -Install Singularity/Apptainer -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Users must have an **Intel** compiler and :term:`MPI` (available for free `here `__) in order to run the SRW App in the container provided using the method described in this chapter. Additionally, it is recommended that users install the `Rocoto workflow manager `__ on their system in order to take advantage of automated workflow options. Although it is possible to run an experiment without Rocoto, and some tips are provided, the only fully-supported and tested container option assumes that Rocoto is preinstalled. + +**Install Singularity/Apptainer** + +To build and run the SRW App using a Singularity/Apptainer container, first install the software according to the `Apptainer Installation Guide `__. This will include the installation of all dependencies. .. note:: As of November 2021, the Linux-supported version of Singularity has been `renamed `__ to *Apptainer*. Apptainer has maintained compatibility with Singularity, so ``singularity`` commands should work with either Singularity or Apptainer (see compatibility details `here `__.) -To build and run the SRW App using a Singularity/Apptainer container, first install the software according to the `Apptainer Installation Guide `__. This will include the installation of all dependencies. - -.. warning:: +.. attention:: Docker containers can only be run with root privileges, and users cannot have root privileges on :term:`HPCs `. Therefore, it is not possible to build the SRW App, which uses the spack-stack, inside a Docker container on an HPC system. However, a Singularity/Apptainer image may be built directly from a Docker image for use on the system. +.. _work-on-hpc: + Working in the Cloud or on HPC Systems ----------------------------------------- -For users working on systems with limited disk space in their ``/home`` directory, it is recommended to set the ``SINGULARITY_CACHEDIR`` and ``SINGULARITY_TMPDIR`` environment variables to point to a location with adequate disk space. For example: +Users working on systems with limited disk space in their ``/home`` directory may need to set the ``SINGULARITY_CACHEDIR`` and ``SINGULARITY_TMPDIR`` environment variables to point to a location with adequate disk space. For example: .. code-block:: export SINGULARITY_CACHEDIR=/absolute/path/to/writable/directory/cache export SINGULARITY_TMPDIR=/absolute/path/to/writable/directory/tmp -where ``/absolute/path/to/writable/directory/`` refers to a writable directory (usually a project or user directory within ``/lustre``, ``/work``, ``/scratch``, or ``/glade`` on NOAA Level 1 systems). If the ``cache`` and ``tmp`` directories do not exist already, they must be created with a ``mkdir`` command. - -On NOAA Cloud systems, the ``sudo su`` command may also be required. For example: - -.. code-block:: - - mkdir /lustre/cache - mkdir /lustre/tmp - sudo su - export SINGULARITY_CACHEDIR=/lustre/cache - export SINGULARITY_TMPDIR=/lustre/tmp - exit - -.. note:: - ``/lustre`` is a fast but non-persistent file system used on NOAA Cloud systems. To retain work completed in this directory, `tar the files `__ and move them to the ``/contrib`` directory, which is much slower but persistent. +where ``/absolute/path/to/writable/directory/`` refers to the absolute path to a writable directory with sufficient disk space. If the ``cache`` and ``tmp`` directories do not exist already, they must be created with a ``mkdir`` command. See :numref:`Section %s ` to view an example of how this can be done. .. _BuildC: Build the Container ------------------------ +* :ref:`On Level 1 Systems ` (see `list `__) +* :ref:`On Level 2-4 Systems ` + .. hint:: If a ``singularity: command not found`` error message appears when working on Level 1 platforms, try running: ``module load singularity`` or (on Derecho) ``module load apptainer``. +.. _container-L1: + Level 1 Systems ^^^^^^^^^^^^^^^^^^ @@ -120,6 +114,8 @@ When making a writable sandbox on Level 1 systems, the following warnings common WARNING: integrity: signature not found for object group 1 WARNING: Bootstrap image could not be verified, but build will continue. +.. _container-L2-4: + Level 2-4 Systems ^^^^^^^^^^^^^^^^^^^^^ @@ -132,7 +128,7 @@ On non-Level 1 systems, users should build the container in a writable sandbox: Some users may prefer to issue the command without the ``sudo`` prefix. Whether ``sudo`` is required is system-dependent. .. note:: - Users can choose to build a release version of the container (SRW App |version|) using a similar command: + Users can choose to build a release version of the container using a similar command: .. code-block:: console @@ -144,36 +140,6 @@ For easier reference, users can set an environment variable to point to the cont export img=/path/to/ubuntu20.04-intel-srwapp - -.. _WorkOnHPC: - -Allocate a Compute Node --------------------------- - -Users working on HPC systems that do **not** have Rocoto installed must `install Rocoto `__ or allocate a compute node. All other users may skip to the :ref:`next step `. - -.. note:: - - All NOAA Level 1 systems have Rocoto pre-installed. - -The appropriate commands for allocating a compute node will vary based on the user's system and resource manager (e.g., Slurm, PBS). If the user's system has the Slurm resource manager, the allocation command will follow this pattern: - -.. code-block:: console - - salloc -N 1 -n -A -t