Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User guide update (includes #32) #40

Merged
merged 13 commits into from
Jan 31, 2020
Merged

User guide update (includes #32) #40

Changes from 10 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
e0c82bf
update chapter about model development with info from Wiki
valbonakunkel-noaa Jan 13, 2020
3a58dfa
fixed all bugs
valbonakunkel-noaa Jan 16, 2020
888c753
Delete .DS_Store
valbonakunkel-noaa Jan 17, 2020
9a02ea6
Delete .DS_Store
valbonakunkel-noaa Jan 17, 2020
55d6a4e
update the file ContributingDevelopment.rst
valbonakunkel-noaa Jan 28, 2020
a8bd901
Merge pull request #1 from valbonakunkel-noaa/documentation
valbonakunkel-noaa Jan 28, 2020
9b80926
Delete ContributingDevelopment.rst
valbonakunkel-noaa Jan 28, 2020
46d14dd
Add files via upload
valbonakunkel-noaa Jan 28, 2020
a144a2f
Update ContributingDevelopment.rst
valbonakunkel-noaa Jan 29, 2020
219d576
Merge branch 'UserGuideUpdate' of https://github.com/valbonakunkel-no…
climbfuji Jan 30, 2020
196ffea
doc/UsersGuide/source/ContributingDevelopment.rst: fixing the biggest…
climbfuji Jan 30, 2020
22765ab
Fix more errors, update usage in rt.sh
climbfuji Jan 30, 2020
01d4ad9
Document -k in rt.sh, remove another <model> comment
climbfuji Jan 30, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
216 changes: 215 additions & 1 deletion doc/UsersGuide/source/ContributingDevelopment.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,218 @@
Contributing Development
*************************

This chapter is being developed here: https://github.com/ufs-community/ufs-weather-model/pull/32
The ufs-weather-model repository contains the model code and external links needed to build the Unified Forecast System (UFS) atmosphere model and associated components, including the WaveWatch III model. This weather model is used in several of the UFS applications, including the medium-range weather application, the short-range weather application, and the sub-seasonal to seasonal application.

------------------------
Prerequisite libraries:
------------------------

Below is the list of libraries that are required to compile the model and run regression tests:

+-------------+----------------------------------------------------------------------+---------+----------------+
| Library name| source | version | env variables |
+=============+======================================================================+=========+================+
| bacio | https://github.com/NOAA-EMC/NCEPLIBS-bacio | 2.0.3 | $BACIO_LIB4 |
+-------------+----------------------------------------------------------------------+---------+----------------+
| sp | https://github.com/NOAA-EMC/NCEPLIBS-sp | 2.0.3 | $SP_LIBd |
+-------------+----------------------------------------------------------------------+---------+----------------+
| w3nco | https://github.com/NOAA-EMC/NCEPLIBS-w3nco | 2.0.7 | $W3NCO_LIBd |
+-------------+----------------------------------------------------------------------+---------+----------------+
| w3emc | https://github.com/NOAA-EMC/NCEPLIBS-w3emc | 2.3.1 | $W3EMC_LIBd |
+-------------+----------------------------------------------------------------------+---------+----------------+
| nemsio | https://github.com/NOAA-EMC/NCEPLIBS-nemsio | 2.2.4 | $NEMSIO_INC |
+ + + + +
| | | | $NEMSIO_LIB |
+-------------+----------------------------------------------------------------------+---------+----------------+
| g2 | https://github.com/NOAA-EMC/NCEPLIBS-g2 | 3.1.1 | $G2_LIB4 |
+-------------+----------------------------------------------------------------------+---------+----------------+
| g2tmpl | https://github.com/NOAA-EMC/NCEPLIBS-g2tmpl(required for inline post)| 1.5.1 | $G2TMPL_LIB |
+-------------+----------------------------------------------------------------------+---------+----------------+
| crtm | https://github.com/NOAA-EMC/EMC_crtm (required for inline post) | 2.2.6 | $CRTM_LIB |
+-------------+----------------------------------------------------------------------+---------+----------------+
| post | http://github.com/NOAA-EMC/EMC_post (required for inline post) | 8.0.0 | $POST_INC |
+ + + + +
| | | | $POST_LIB |
+-------------+----------------------------------------------------------------------+---------+----------------+

External libraries:

+-------------+--------------------------------------------------------------+---------+-------------------------+
| Library name| source | version | env variables |
+=============+==============================================================+=========+=========================+
| esmf | https://www.earthsystemcog.org/projects/esmf/download_800 | 8.0.0 | $ESMFMKFILE |
+-------------+--------------------------------------------------------------+---------+-------------------------+
| | | 4.5.0 | |
+ + + + +
| netCDF | https://www.unidata.ucar.edu/downloads/netcdf/index.jsp | 4.6.1 | $NETCDF |
+ + + + +
| | | 4.6.3 | |
+ + + + +
| | | 4.7.0 | |
+-------------+--------------------------------------------------------------+---------+-------------------------+
| | | 1.8.14 | |
+ + + + +
| | | 1.8.16 | |
+ + + + +
| HDF5 | https://www.hdfgroup.org/downloads/hdf5/ | 1.10.1 | $HDF5_ROOT |
+ + + + +
| | | 1.10.4 | |
+ + + + +
| | | 1.10.5 | |
+-------------+--------------------------------------------------------------+---------+-------------------------+
| jasper | https://github.com/mdadams/jasper | 1.900.1 | $JASPER_ROOT |
+-------------+--------------------------------------------------------------+---------+-------------------------+
| zlib | https://github.com/madler/zlib | 1.2.11 | $ZLIB_ROOT |
+-------------+--------------------------------------------------------------+---------+-------------------------+
| libpng | http://www.libpng.org/pub/png/libpng.html | 1.2.44 | $LIBPNG_ROOT |
+-------------+--------------------------------------------------------------+---------+-------------------------+

Compiler :

+--------------------+------------------------------------------------------------------+
|Compiler name | Compiler version |
+====================+==================================================================+
| Intel | 16.0.3.210, 16.3.210, 18.0.1.163, 18.0.3.222, 18.0.5.274, 19.0.2 |
+--------------------+------------------------------------------------------------------+
| GNU | 8.3.0, 9.1.0, 9.2.0 |
+--------------------+------------------------------------------------------------------+

MPI library :

+--------------------+------------------------------------------------------------------+
| MPI Library name | MPI library version |
+====================+==================================================================+
| Intel iMPI | 2018.0.4, 2018.4.274,18.0.1, 2019.2.187 |
+--------------------+------------------------------------------------------------------+
| MPICH | 3.3.1, 3.3.2 |
+--------------------+------------------------------------------------------------------+
| Open MPI | 4.0.2 |
+--------------------+------------------------------------------------------------------+
| SGI MPT | 2.19 |
+--------------------+------------------------------------------------------------------+

These libraries are available on several platforms including Hera, Cheyenne and WCOSS. Installing these libraries on jet and gaea is a work in progress. Information how to build the libraries can be found at [https://github.com/NOAA-EMC/NCEPLIBS/wiki/Cloning-and-Compiling-NCEPLIBS]

In addition to these libraries, other software is required such as a compiler (Intel or GNU), MPI library (impi), HDF5, cmake, Python, Perl and Ruby to compile the model and run regression tests.

---------------------------------------------
Making code changes using a forking workflow
---------------------------------------------

If developers would like to make code changes, they need to make a personal fork, set up upstream remote (for merging with the original ufs-weather-model), and create a branch for ufs-weather-model and each of the subcomponent repositories they want to change. They can then make code changes, perform testing and commit the changes to the branch in their personal fork. It is suggested that they update their branch by merging the develop branch with the develop branch of the original repositories periodically to get the latest updates and bug fixes.

If developers would like to get their code committed back to the original repository, it is suggested to follow the steps below:

1. Create an issue in the authoritative repository. For example to commit code changes to fv3atm, please go to https://github.com/NOAA-EMC/fv3atm, under NOAA-EMC/fv3atm and find the “Issues” tab next to the “Code” tab. Click on “Issues” and a new page will appear. On the right side of the page, there is a green “New issue” button. Clicking on that will lead to a new issue page. Fill out the title, comments to describe the code changes, and also please provide personal fork and branch information. Lastly, click on the “Submit new issue” button, so that the new issue is created.

2. When the development is mature, tests have been conducted, and the developer is satisfied with the results, create a pull request to commit the code changes.

* Merge developer’s branch to the latest ufs-weather-model develop branch in authoritative repository. If changes are made in model sub-components, developers need to merge their branches to branches with the corresponding authoritative repository (or original repository for some components). For this, code management practices of the subcomponents need to be followed.

* Regression tests associated with the ufs-weather-model are available in Tier 1 and Tier 2 platforms as described in https://github.com/ufs-community/ufs-weather-model/wiki/Weather-Model-Platform-and-Compiler-Support. If the developer has access to these platforms, the developer should pass the regression test on at least one supported platform. If the developer does not have access to these platforms, this should be stated in the PR so the code manager(s) can conduct the tests.

* For each component branch where developers make changes, developers need to go to their personal fork on GitHub and click on the “New pull request” button. When a new page “Compare changes” appears, developers will choose the branch in their fork with code changes to commit and the branch in upstream repository that the changes will be committed to. Also developers in the commit comment must add the github issue title and number created in 1) in the comment box. The code differences between the two branches will be displayed. Developers can review the differences and click on “submit pull request” to make the pull request. After code changes are committed to the component repository, developers will make pull requests to ufs-weather-model repository.

3. When PRs are created, the creator must temporarily modify .gitmodules to point to his/her fork and branch if updates are required for submodules.

4. Merging code from PRs with submodules requires coordination with the person making the PRs. From the "innermost" nested PR up to the top-level PR, the PRs need to be merged as-is. After each merge, the person creating the PRs has to update his/her local code to check out the merged version, revert the change to .gitmodules, and push this to GitHub to update the PR. And so on and so forth.

5. Checking out the code ufs_release_1.0 should always be as follows:

.. code-block:: console

git clone https://github.com/ufs-community/ufs-weather-model
cd ufs-weather-model
git checkout ufs_release_1.00
git submodule update --init --recursive

6. Checking out a PR with id ID for testing it should always be as follows:

.. code-block:: console

git clone https://github.com/ufs-community/ufs-weather-model
cd ufs-weather-model
git fetch origin pull/ID/head:BRANCHNAME
git checkout BRANCHNAME
git submodule update --init --recursive

It is suggested that the developers inform all the related code managers as the hierarchy structure of the ufs-weather-model repository may require collaboration among the code managers.

-----------------------------------
Engaging in the code review process
-----------------------------------

When code managers receive a pull request to commit the code changes, it is recommended that they add at least two code reviewers to review the code and at least one of the reviewers has write permission. The reviewers will write comments about the code changes and give a recommendation as to whether the code changes can be committed. What kinds of code changes will be accepted in the repository is beyond the scope of this document; future ufs-weather-model code management documents may have a detailed answer for that.

Reviewers may suggest some code changes during the review process. Developers need to respond to these comments in order to get code changes committed. If developers make further changes to their branch, reviewers need to check the code changes again. When both reviewers give recommendation to commit the code, code managers will merge the changes into the repository.

----------------------------
Conducting regression tests
----------------------------

Only developers using Tier 1 and 2 platforms can run the ufs-weather-model regression tests. Other developers need to work with the code managers to assure completion of the regression tests.


To run regression test using rt.sh

rt.sh is a bash shell file to run the RT and has the following options:

.. code-block:: console

Usage: $0 -c <model> | -f | -s | -l <file> | -m | -r | -e | -h
-c create new baseline results for <model>
-f run full suite of regression tests
-s run standard suite of regression tests
-l run test specified in <file>
-m compare against new baseline results
-r use Rocoto workflow manager
-e use ecFlow workflow manager
-h display this help

.. code-block:: console

% cd ufs-weather-model/tests
% ./rt.sh -f

This command can only be used on a NOAA machine using the Intel compiler, where an official baseline is available. For information on testing the CCPP code, or using alternate computational platforms, see the following sections.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't ./rt.sh be run on non-NOAA machines, i.e. Cheyenne? And with the GNU compiler?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is wrong. My job was not to work on the documentation but to merge what is there now. Hmpf. I will correct it.


This command and all others below produce log output in ./tests/log_machine.compiler. These log files contain information on the location of the run directories that can be used as templates for the user. Each rt*.conf contains one or more compile commands preceding a number of tests.

Regression test log files (ufs-weather-model/tests/Compile_$(MACHINE_ID).log and ufs-weather-model/tests/RegressionTests_$(MACHINE_ID).log ) will be updated.

Regression testing is only possible on machines for which baselines exist, that is Tier 1 and 2 platforms.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regression testing is possible on a machine without a baseline. Baselines should be available on Tier 1 and possibly on Tier 2 platforms (please clarify). ./rt.sh should run on Tier 1 and 2 platforms, in which case, a baseline may be available. If not, one can be created as described below.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JulieSchramm @DusanJovic-NOAA As I am reading the paragraph starting at line 180, this is wrong! You can't just create a new baseline on a different platform using rt.sh. The script contains machine-specific sections and uses several scripts underneath that contain machine-specific sections.

And it contradicts several other sections in the document.


When porting the code to a new machine, it is useful to start by establishing a personal baseline. Future runs of the RT can then be compared against the personal baseline to ascertain that the results have not been inadvertently affected by code developments. The rt.sh -c option is used to create a personal baseline.

./rt.sh -l rt.conf -c fv3 # create own reg. test baseline
Once the personal baseline has been created, future runs of the RT should be compared against the personal baseline using the -m option.

./rt.sh -l rt.conf -m # compare against own baseline

To create new baseline:

.. code-block:: console

% cd ufs-weather-model/tests
% ./rt.sh -f -c

* To run regression test using NEMSCompsetRun

.. code-block:: console

% cd ufs-weather-model
% ./NEMS/NEMSCompsetRun -f

Regression test log files (ufs-weather-model/log/$MACHINE_ID/* ) will be updated.

To create new baseline:

.. code-block:: console

% cd ufs-weather-model
% ./NEMS/NEMSCompsetRun--baseline fv3 --platform=${PLATFORM}


The value of ${PLATFORM} can be found in ufs-weather-model/compsets/platforms.input.

Developers need to commit the regression test log files to their branch before making pull request.