Merge release hotfix into develop (#45)

* Set default values for model_configure variables (#18)

RUN_CONTINUE = .false.
 ENS_SPS = .false.
 TOTAL_MEMBER = 1

* fixes problem with MPI_Comm_Split in openmpi when hash is negative

* removed redundant check on hash > 0

* Add GNU LPGL (#32)

* Add NEMS documentation (#33)

* Add nems documentation in doc directory

*Include in .gitignore NEMS/doc/html/ and NEMS/doc/webpage.tar.gz
*Remove milestones.dox which is no longer needed
*Modify Makefile to include doxygen command on Hera
*Remove in prep_inputs.py two functions related to milestones

* Remove webpage.tar.gz file

* Update NEMS documentation

*Add Doxyfile in NEMS/.gitignore

* Update based on documentation meeting

* Updates to the documentation (#35)

* Documentation update (#36)

* Modify license file (#37)

* Modify license file

* Fix typo in license

Co-authored-by: Dusan Jovic <[email protected]>
Co-authored-by: Mark A. Potts <[email protected]>
Co-authored-by: Mark Potts <[email protected]>
Co-authored-by: Mark Potts <[email protected]>
Co-authored-by: arun chawla <[email protected]>

.gitignore

@@ -1,2 +1,5 @@
 *.pyc
 __pycache__
+/doc/html/
+/doc/webpage.tar.gz
+/doc/Doxyfile

doc/Makefile

@@ -0,0 +1,64 @@
+LOCAL_DOCS=$(wildcard *.md) $(wildcard *.dox)
+CONTROL_FILES=Makefile prep_inputs.py Doxyfile.IN DoxygenLayout.xml.IN
+
+
+# Find Doxygen
+
+ifndef DOXYGEN
+DOXYGEN=doxygen
+endif
+
+ifeq ("$(DOXYGEN)","doxygen")
+  $(info Attempting to auto-detect environment.)
+  THEIA_DOXYGEN=/scratch3/NCEPDEV/hwrf/save/Samuel.Trahan/doxygen-1.8.10/bin/doxygen
+  JET_DOXYGEN=/contrib/doxygen/1.8.10/bin/doxygen
+  LUNA_DOXYGEN=/gpfs/hps3/emc/hwrf/noscrub/soft/doxygen-1.8.10/bin/doxygen
+  TIDE_DOXYGEN=/hwrf/noscrub/soft/doxygen-1.8.10/bin/doxygen
+  HERA_DOXYGEN=/usr/bin/doxygen
+
+  ifneq ($(wildcard $(THEIA_DOXYGEN)*),)
+    $(info On Theia.)
+    DOXYGEN=$(THEIA_DOXYGEN)
+    TARGET ?= [email protected]:/home/www/emc/htdocs/projects/nems-sample/.
+  else ifneq ($(wildcard $(JET_DOXYGEN)*),)
+    $(info On Jet.)
+    DOXYGEN=$(JET_DOXYGEN)
+    TARGET ?= [email protected]:/home/www/emc/htdocs/projects/nems-sample/.
+  else ifneq ($(wildcard $(LUNA_DOXYGEN)*),)
+    $(info on Luna or Surge.)
+    DOXYGEN=$(LUNA_DOXYGEN)
+    TARGET ?= [email protected]:/home/www/emc/htdocs/mmb/rvasic/nems-sample/.
+  else ifneq ($(wildcard $(TIDE_DOXYGEN)*),)
+    $(info on Tide or Gyre.)
+    DOXYGEN=$(TIDE_DOXYGEN)
+    TARGET ?= [email protected]:/home/www/emc/htdocs/mmb/rvasic/nems-sample/.
+  else ifneq ($(wildcard $(HERA_DOXYGEN)*),)
+    $(info on Local machine.)
+    DOXYGEN=$(HERA_DOXYGEN)
+  else
+    $(error Could not detect environment.  You must set TARGET and DOXYGEN manually)
+  endif
+else
+  $(info DOXYGEN set in environment.  Will not set TARGET variable.  You must set it manually.  DOXYGEN=$(DOXYGEN))
+endif
+
+TARGET ?= /please/update/TARGET/in/Makefile
+
+default:
+	@echo Specify build mode:
+	@echo doc        = build documentation just for this directory
+	@echo clean      = delete outputs
+	@echo deliver    = copy to website
+	exit 19
+
+clean:
+	rm -rf html webpage.tar.gz repo_info.sh.inc Doxyfile DoxygenLayout.xml
+
+doc: clean
+	./prep_inputs.py ..
+	set -x ; $(DOXYGEN)
+	tar -cpzf webpage.tar.gz html Doxyfile
+
+deliver: html
+	cd html && rsync -arv . "$(TARGET)/."
+

doc/architecture.md

@@ -0,0 +1,48 @@
+Architecture {#architecture}
+============
+
+The NEMS architecture is based on an ESMF component hierarchy with the
+application driver `MAIN_NEMS` at the top, calling into the
+`NEMS_COMP` component, which in turn drives the `EARTH_COMP`
+component. The `EARTH_COMP` drives the `ATM` and other components.
+The architecture allows for multiple `EARTH_COMP` instances.
+
+NEMS includes atmosphere, ocean, ice, wave, land,
+aerosol/chemistry, and hydrologic models, with coupling interface and
+utilities based on the 
+[Earth System Modeling Framework (ESMF)](https://www.earthsystemcog.org/projects/esmf/).
+The NEMS applications also utilize intreopereability conventions
+introduced by the 
+[National Unified Operational Prediction Capability (NUOPC)](https://www.earthsystemcog.org/projects/nuopc/).
+
+Key architecture features are:
+
+* Data exchanges between major model components go through a central
+  (NEMS) mediator component. There
+  may be additional specialized mediators (e.g. for the 3D
+  interactions associated with space weather coupling).
+
+* The NEMS mediator component is an integral part of the NEMS
+  software. The mediator source code is managed alongside the NEMS
+  source code, is integrated into the NEMS make system and is built
+  when the NEMS executable is built.
+
+* Component models are treated by NEMS as external
+  dependencies. Their source code is managed outside of NEMS,
+  typically in the proximity of the organization maintaining the
+  official version of the model. Each model maintains its own separate
+  make system with a NUOPC compliant build option. NEMS requires that
+  the pre-built models are available when the NEMS executable is being
+  built.
+
+* All of the components driven by they EARTH_COMP are NUOPC-compliant
+  components.
+
+* All of the components driven by they EARTH_COMP can be configured to
+  run on a specific set of PETs (MPI processes), supporting concurrent
+  or sequential component execution.
+
+* Data exchanges between components are handled by generic
+NUOPC_Connector components.
+* The generic connectors perform basic regrid and redist operations as needed to
+take field data from one side to the other.

doc/cap-doc.md

@@ -0,0 +1,15 @@
+Cap Documentation {#cap-page}
+=================
+
+Caps are codes that act as adaptors, enabling models to use 
+[NUOPC](https://www.earthsystemcog.org/projects/nuopc/)
+standard component interfaces. They are called caps because they "sit
+on top" of user code and call into it. They contain translations of
+native model data structures into ESMF data structures.
+
+For documentation of caps, see the 
+[Earth System Prediction Suite (ESPS)](https://www.earthsystemcog.org/projects/esps/)
+site. Find the components on the table on the landing page, and
+follow the corresponding Code Access and Documentation link. Not all
+listed components have documented caps. The ESPS is a collection of
+components that are compliant with NUOPC conventions.

doc/configuring.md

@@ -0,0 +1,95 @@
+Configuring {#configuring}
+===========
+
+The NEMS application is highly configurable. During
+build-time, i.e. when the NEMS executable is being compiled and
+linked, choices are made as to which model and mediator components are
+built into the system. The built-in components become accessible
+during run-time, allowing the execution of different run
+configurations without the need to rebuild NEMS.
+
+Often the build and run configurability of NEMS is hidden from the
+user by application or test scripts. However, there are times where it
+is useful to understand the technical details on the level of the NEMS
+executable.
+
+Run Configuration
+-----------------
+
+During run-time of the NEMS executable, it accesses a file called
+nems.configure, which it expects to find in the run directory. This
+file specifies the dynamic component selection, and the exact run
+sequence to be used. Only models built into the executable at
+build-time are accessible during run-time. An error will be triggered
+if an unavailable model is selected in nems.configure. The component
+selection is based on two variables:
+
+    xxx_model:          abc
+    xxx_petlist_bounds: lower upper
+
+Here `xxx` can be `atm`, `ocn`, `ice`, `ipm`, `med`. The `abc`
+stands for the actual instance name, e.g. `fv3` or `mom6`. The
+lower and upper bounds of the petList specification are integer PET
+numbers.
+
+The specification of the run sequence provides the flexibility needed
+to cover different coupling scenarios. The format is line based
+between special tags:
+
+    runSeq::
+        line1
+        line2
+        ...
+        lineN
+    ::
+
+There are a number of format options for each line:
+
+* A time loop is introduced by a `@` symbol, followed immediatly by
+  the number of seconds of the associated time step.
+
+* A time loop is closed by a single `@` symbol on a line.
+
+* The `RUN` method of model component `xxx` is called by specifying
+  just `xxx` on a line. The supported values of `xxx` are the same
+  as for the model specification above. A specific RUN phase can be
+  provided by adding the phase label to the same line, following the
+  model string.
+
+* A connector going from component `xxx` to component `yyy` is
+  specified by a line like this: `xxx -> yyy`. An additional
+  argument on the same line can be used to specify connection options
+  for all of the field handled by the connector. The format and
+  supported values of the connection options is documented in the
+  NUOPC reference manual.
+
+A very simple run sequence is running only the atmospheric uncoupled model.
+The run sequence looks like:
+
+    # Run Sequence #
+    runSeq::
+        ATM
+    ::
+
+A more complex example is when components are exchanging information at two time scales:
+
+    # Run Sequence #
+    runSeq::
+        @7200.0
+        OCN -> MED
+        MED MedPhase_slow
+        MED -> OCN
+        OCN
+        @3600.0
+            MED MedPhase_fast_before
+            MED -> ATM
+            ATM
+            ATM -> MED
+            MED MedPhase_fast_after
+          @
+        @
+    ::
+
+Anything on a line after the `#` symbol is treated as a comment and
+ignored. Indentation in the formatting does not change the meaning of
+a line, but is purely used to increase readability.

doc/documentation.dox

@@ -0,0 +1,12 @@
+/**@page documentation Users Guide and Reference
+
+These pages contain general documentation for the NEMS, such as
+guides, howtos and system descriptions.
+
+  + @subpage introduction
+  + @subpage architecture
+  + @subpage configuring
+  + @subpage mediator
+  + @subpage cap-page
+
+*/

doc/introduction.md

@@ -0,0 +1,76 @@
+Introduction to NEMS {#introduction}
+====================
+
+The NOAA Environmental Modeling System (NEMS) is the infrastructure
+underlying NOAA's [Unified Forecast System (UFS)] (https://ufscommunity.org/) a fully coupled modeling framework that supports predictions of
+Earth's environment at a range of time scales. Examples of other
+coupled modeling systems are the 
+[Community Earth System Model (CESM)](http://www.cesm.ucar.edu)
+and the
+[Met Office Unified Model] (http://www.metoffice.gov.uk/research/modelling-systems/unified-model).
+
+NEMS includes infrastructure for coupling model components
+representing major Earth system domains and processes.  
+**A model component** is a software representation of a physical
+domain or process, for example sea ice. It is often developed by a
+team of specialists in that domain. Model coupling is a software
+representation of feedbacks between physical processes. It involves
+modifying the exported fields of a component through grid, unit,
+temporal, and other transformations so that they can be used as the
+inputs for another component. These components are managed through repositories, primarily on GitHub.
+
+In general, model components are
+coupled through the NEMS mediator (in other coupled modeling systems
+this is often called the "coupler").  NEMS also includes some
+specialized mediators; for example, for space weather. In some cases
+in NEMS, the model components are coupled "in-line", meaning that they
+are called directly from another model component instead of having
+fields sent through the mediator.
+
+NEMS can be assembled into a number of different **modeling
+applications** (often shortened to just applications). Modeling
+applications are associated with a purpose, like medium-range
+forecasting; a set of model components; and a set of parameters that
+represent a range of supported options, including grids and
+resolutions. Different NEMS modeling applications can have different
+types and numbers of model components. Also, the same physical domain
+may be represented by different model components in different modeling
+applications. For example, in some NEMS modeling applications the
+ocean component may be the HYbrid Coordinate Ocean Model (HYCOM) and
+in others it may be the Modular Ocean Model (MOM).
+
+Infrastructure
+--------------
+
+[NEMS is built using the Earth System Modeling Framework (ESMF)](https://www.earthsystemcog.org/projects/esmf/)
+infrastructure software. ESMF provides utilities like generation of
+interpolation weights and utilities for calendar and timee management,
+and also wrappers that create a standard component calling
+interface. This enables model components developed at different sites
+to be coupled more easily.
+
+[The National Unified Operational Prediction Capability (NUOPC) Layer] (https://earthsystemcog.org/projects/nuopc/)
+adds additional rules about how ESMF models interact and increases
+their interoperability. The NUOPC Layer covers aspects from the level
+of build dependencies, to standardization of initialization phases,
+all the way to standard names of the exchanged fields. NEMS is an
+example of a modeling system built using the NUOPC Layer architecture.
+
+Architecture 
+------------
+
+The NEMS architecture is based on an ESMF component hierarchy with the
+application driver `MAIN_NEMS` at the top, calling into the
+`NEMS_COMP` component, which in turn drives the `EARTH_COMP`
+component. The `EARTH_COMP` drives the `ATM` and other components.
+The architecture allows for
+multiple `EARTH_COMP` instances, supporting ensemble applications such
+as the Global Ensemble Forecast System (GEFS).
+
+Coupled NEMS includes atmosphere, ocean, ice, wave, land,
+aerosol/chemistry, and hydrologic models, with coupling interface and
+utilities based on the 
+[Earth System Modeling Framework (ESMF)](https://www.earthsystemcog.org/projects/esmf/).
+The NEMS applications also utilize intreopereability conventions
+introduced by the 
+[National Unified Operational Prediction Capability (NUOPC)](https://www.earthsystemcog.org/projects/nuopc/).

doc/make-images-fit.css

@@ -0,0 +1,3 @@
+img {
+	max-width: 100%
+}