Dev Guide updates #440

forsyth2 · 2023-06-19T12:42:58Z

Dev Guide updates

forsyth2 · 2023-06-19T12:44:46Z

I will probably merge this after the summer 2023 release. That way I can update it throughout the release process as difficulties/unclear steps are encountered.

Then I will make an equivalent pull request for zstash.

xylar · 2023-06-19T14:00:07Z

Good plan!

docs/source/dev_guide/release.rst

docs/source/dev_guide/pre_release_testing.rst

.github/workflows/build_workflow.yml

docs/source/dev_guide/index.rst

docs/source/dev_guide/pre_release_testing.rst

forsyth2 · 2023-08-15T14:10:17Z

docs/source/dev_guide/pre_release_testing.rst

+   * `Compy <https://github.com/E3SM-Project/zppy/blob/main/tests/integration/generated/directions_compy.md>`_
+   * `Perlmutter <https://github.com/E3SM-Project/zppy/blob/main/tests/integration/generated/directions_pm-cpu.md>`_
+
+9. Run the integration tests with ``python -u -m unittest tests/integration/test_*.py``. Note that ``test_complete_run.py`` takes approximately 75 minutes to run on Compy.


About 20 minutes to run entire test suite on Chrysalis. The zppy run of test_complete_run_chrysalis.cfg takes a little over an hour.

Compy: Ran 13 tests in 5836.885s [97.3 minutes] for python -u -m unittest tests/integration/test_*.py

Compy, after updating expected files: Ran 1 test in 669.627s [11.2 minutes] for python -u -m unittest tests/integration/test_complete_run.py (the longest-running test).

I suspect the printing of error messages (e.g., the image diff information) may be what slowed the previous run down so much.

Compy, after updating expected files: Ran 13 tests in 2854.441s [47.6 minutes] for python -u -m unittest tests/integration/test_*.py (all integration tests)

forsyth2 · 2023-08-16T21:48:33Z

docs/source/dev_guide/pre_release_testing.rst

+Testing directions for making a release
+***************************************
+
+1. Have three shells open: one on Chrysalis, one on Compy, and one on Perlmutter. Do the following steps on each machine.


It might be a good idea to have a slightly different document for testing Unified releases (i.e., using Unified RCs rather than a zppy dev environment)

Step 3: git checkout -b test_unified_rc12 upstream/main
Step 6: Load the relevant Unified RC. E.g.,

Chrysalis: source /lcrc/soft/climate/e3sm-unified/test_e3sm_unified_1.9.0rc12_chrysalis.sh Compy: source /share/apps/E3SM/conda_envs/test_e3sm_unified_1.9.0rc12_compy.sh Perlmutter: source /global/common/software/e3sm/anaconda_envs/test_e3sm_unified_1.9.0rc12_pm-cpu.sh

Step 7: if there are any failures, make a fix and use a zppy dev environment until the fix can be incorporated into the next Unified RC.

Include the machine name in git checkout -b test_unified_rc12 upstream/main. That way I can push a branch for each machine for the testing-fixes PRs that are typical in the release process (e.g., #486)

forsyth2 · 2023-08-18T17:23:41Z

docs/source/dev_guide/pre_release_testing.rst

+   * `Chrysalis <https://github.com/E3SM-Project/zppy/blob/main/tests/integration/generated/directions_chrysalis.md>`_
+   * `Compy <https://github.com/E3SM-Project/zppy/blob/main/tests/integration/generated/directions_compy.md>`_
+   * `Perlmutter <https://github.com/E3SM-Project/zppy/blob/main/tests/integration/generated/directions_pm-cpu.md>`_
+


Between these steps, check the status of the files. Probably best to add that to the testing direction files above:
e.g., for Chrysalis:

$ cd /lcrc/group/e3sm/ac.forsyth2/zppy_test_complete_run_output/v2.LR.historical_0201/post/scripts $ grep -v "OK" *status $ cd /lcrc/group/e3sm/ac.forsyth2/zppy_test_bundles_output/v2.LR.historical_0201/post/scripts $ grep -v "OK" *status

Compy:

$ cd /compyfs/fors729/zppy_test_complete_run_output/v2.LR.historical_0201/post/scripts/ $ grep -v "OK" *status $ cd /compyfs/fors729/zppy_test_bundles_output/v2.LR.historical_0201/post/scripts/ $ grep -v "OK" *status

Perlmutter:

$ cd /global/cfs/cdirs/e3sm/forsyth/zppy_test_complete_run_output/v2.LR.historical_0201/post/scripts $ grep -v "OK" *status $ cd /global/cfs/cdirs/e3sm/forsyth/zppy_test_bundles_output/v2.LR.historical_0201/post/scripts $ grep -v "OK" *status

Currently, the placement of the image_check_failures directory results in it being mistakenly included in the expected files, when the expected files are updated. This should be resolved by running rm -rf /global/cfs/cdirs/e3sm/www/forsyth/zppy_test_bundles_www/v2.LR.historical_0201/image_check_failures* (and similar for complete run, and on different machines)before running thecp -r` line.

Also, try to have two sets of expected -- one from the latest official release, one from the latest main.

=> Make a note before the rm -rf /lcrc/group/e3sm/public_html/zppy_test_resources/expected_complete_run line to copy to a expected_complete_run_v2.3.0 directory (if a release has just been made).

forsyth2 · 2023-08-29T16:46:34Z

docs/source/dev_guide/pre_release_testing.rst

+     $ mamba clean --all
+     $ mamba env create -f conda-env/dev.yml -n e3sm_diags_<date>
+     $ conda activate e3sm_diags_<date>
+     $ pip install .


cd back to zppy directory

forsyth2 · 2023-09-14T17:04:23Z

docs/source/dev_guide/post_release.rst

+At https://github.com/E3SM-Project/zppy/branches, delete any branches that are no longer needed.
+
+Once there is a final release of E3SM Unified, run the pre-test ``zppy`` runs to establish baseline expected images. Run the "Commands to run before running integration tests" for the current machine. To ensure you don't encounter issues from running ``zppy`` commands simultaneously, wait for all the automatically launched jobs from one run to finish before running ``zppy`` again.
+


Actually, probably best to just follow the same directions from docs/source/dev_guide/pre_release_test_unified.rst, just with the final release rather than an RC

Except, set generate_cfgs(unified_testing=False), so the latest Unified is in fact used

No need to update environment_commands_test

Step 6 needs to be the latest Unified script

forsyth2 · 2023-09-14T18:01:52Z

docs/source/dev_guide/pre_release_test_unified.rst

+
+3. Check out a branch to test on. ::
+
+     git fetch upstream main


Note that on Compy, this might not always get the latest commits from main. It is better to do git fetch upstream to fetch all branches.

forsyth2 · 2023-09-14T18:35:37Z

docs/source/dev_guide/pre_release_test_dev.rst

+
+Use these directions to test a ``zppy`` dev environment prior to making a release candidate (RC).
+
+TODO: match up wording with the other doc


Maybe make it all one doc with sub-steps A) dev, B) pre-release, C) testing Unified release

forsyth2 · 2023-09-14T23:08:03Z

docs/source/dev_guide/pre_release_test_unified.rst

@@ -0,0 +1,61 @@
+***************************************


Render docs:

$ cd /home/ac.forsyth2/zppy/docs $ make html $ mv _build/html/ /lcrc/group/e3sm/public_html/diagnostic_output/ac.forsyth2/html_20230914

forsyth2 added DevOps CI/CD, configuration, etc. Documentation Files in `docs` modified labels Jun 19, 2023

forsyth2 self-assigned this Jun 19, 2023

forsyth2 marked this pull request as draft June 19, 2023 12:43

forsyth2 mentioned this pull request Jun 19, 2023

Update release docs #433

Merged

forsyth2 force-pushed the dev-guide-updates branch from d76a265 to ca8f498 Compare June 19, 2023 12:49

forsyth2 commented Jun 21, 2023

View reviewed changes

docs/source/dev_guide/release.rst Outdated Show resolved Hide resolved

forsyth2 commented Jun 21, 2023

View reviewed changes

docs/source/dev_guide/release.rst Show resolved Hide resolved

forsyth2 commented Jun 21, 2023

View reviewed changes

docs/source/dev_guide/release.rst Show resolved Hide resolved

forsyth2 force-pushed the dev-guide-updates branch from ca8f498 to ed85191 Compare July 25, 2023 21:48

forsyth2 mentioned this pull request Jul 25, 2023

Dev Guide updates E3SM-Project/zstash#285

Merged

forsyth2 commented Jul 26, 2023

View reviewed changes

docs/source/dev_guide/release.rst Show resolved Hide resolved

forsyth2 commented Jul 28, 2023

View reviewed changes

docs/source/dev_guide/pre_release_testing.rst Outdated Show resolved Hide resolved

forsyth2 commented Aug 7, 2023

View reviewed changes

docs/source/dev_guide/pre_release_testing.rst Outdated Show resolved Hide resolved

forsyth2 force-pushed the dev-guide-updates branch 2 times, most recently from 007ad76 to 074c781 Compare August 10, 2023 18:45

forsyth2 commented Aug 10, 2023

View reviewed changes

.github/workflows/build_workflow.yml Show resolved Hide resolved

forsyth2 commented Aug 10, 2023

View reviewed changes

docs/source/dev_guide/index.rst Outdated Show resolved Hide resolved

forsyth2 commented Aug 10, 2023

View reviewed changes

docs/source/dev_guide/pre_release_testing.rst Outdated Show resolved Hide resolved

forsyth2 commented Aug 15, 2023

View reviewed changes

forsyth2 commented Aug 16, 2023

View reviewed changes

forsyth2 commented Aug 18, 2023

View reviewed changes

forsyth2 commented Aug 29, 2023

View reviewed changes

forsyth2 force-pushed the dev-guide-updates branch from b61ec81 to 678c424 Compare September 13, 2023 22:32

forsyth2 commented Sep 14, 2023

View reviewed changes

Dev Guide updates

51e78ed

forsyth2 force-pushed the dev-guide-updates branch from c160443 to 51e78ed Compare September 15, 2023 22:03

forsyth2 marked this pull request as ready for review September 15, 2023 22:43

forsyth2 merged commit 23065c9 into main Sep 15, 2023

forsyth2 deleted the dev-guide-updates branch September 15, 2023 22:43

This was referenced Sep 26, 2023

Automated doc updates #515

Merged

Further automate tests #516

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dev Guide updates #440

Dev Guide updates #440

forsyth2 commented Jun 19, 2023

forsyth2 commented Jun 19, 2023

xylar commented Jun 19, 2023

forsyth2 Aug 15, 2023 •

edited

Loading

forsyth2 Aug 17, 2023

forsyth2 Aug 18, 2023 •

edited

Loading

forsyth2 Aug 18, 2023

forsyth2 Aug 16, 2023

forsyth2 Aug 16, 2023

forsyth2 Aug 19, 2023

forsyth2 Aug 18, 2023

forsyth2 Aug 18, 2023

forsyth2 Aug 30, 2023

forsyth2 Aug 31, 2023

forsyth2 Sep 1, 2023

forsyth2 Aug 29, 2023

forsyth2 Sep 14, 2023

forsyth2 Sep 14, 2023

forsyth2 Sep 14, 2023

forsyth2 Sep 14, 2023

forsyth2 Sep 14, 2023

forsyth2 Sep 14, 2023

forsyth2 Sep 14, 2023

		At https://github.com/E3SM-Project/zppy/branches, delete any branches that are no longer needed.

		Once there is a final release of E3SM Unified, run the pre-test ``zppy`` runs to establish baseline expected images. Run the "Commands to run before running integration tests" for the current machine. To ensure you don't encounter issues from running ``zppy`` commands simultaneously, wait for all the automatically launched jobs from one run to finish before running ``zppy`` again.


		Use these directions to test a ``zppy`` dev environment prior to making a release candidate (RC).

		TODO: match up wording with the other doc

Dev Guide updates #440

Dev Guide updates #440

Conversation

forsyth2 commented Jun 19, 2023

forsyth2 commented Jun 19, 2023

xylar commented Jun 19, 2023

forsyth2 Aug 15, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

forsyth2 Aug 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

forsyth2 Aug 15, 2023 •

edited

Loading

forsyth2 Aug 18, 2023 •

edited

Loading