Merge fates_main_api into master #1264

glemieux · 2021-01-27T09:00:52Z

Description of changes

This PR merges fates_main_api into master. The fates_main_api branch has integrated updates from master up to tag ctsm5.1.dev020 per PR #1257. The purpose of this merge is to bring the most recent fates updates into master and for fates future development to continue from ctsm master instead of on fates_main_api.

Specific notes

The aux_clm suite tests were compared against ctsm5.1.dev021 baseline. The fates suite tests were compared against fates-sci.1.43.2_api.14.2.0-ctsm5.1.dev020-C5f348cac-F5534a940 baseline.

Contributors other than yourself, if any: @ekluzek

CTSM Issues Fixed (include github issue #): none

Are answers expected to change (and if so in what way)? Only for aux_clm fates testmods only. All else b4b.

Any User Interface Changes (namelist or namelist defaults changes)? none

Testing performed, if any:

aux_clm on cheyenne: /glade/u/home/glemieux/scratch/tests_0126-161304ch
aux_clm on izumi: /scratch/cluster/glemieux/tests_0204-142628iz
fates on cheyenne: /glade/u/home/glemieux/scratch/clmed-tests/basegen.fates-sci.1.43.2_api.14.2.0-ctsm5.1.dev021-C4c9a4c15-F5534a940

NOTE: Be sure to check your coding style against the standard
(https://github.com/ESCOMP/ctsm/wiki/CTSM-coding-guidelines) and review
the list of common problems to watch out for
(https://github.com/ESCOMP/CTSM/wiki/List-of-common-problems).

gnu test suite. Changing highest res grid test to clm5.

… hydro is active.

…uper-saturation during plant hydraulics.

…prtglobals

…ng wrfs).

variables that track size and age dependent mortality. Add coage dimension to clmfates_interfaceMod.F90 and histFileMod.F90. This allows cohort age to be tracked. Also add variables that allow size and age dependent mortality to be tracked. These mortality functions have been added to the fates code.

[Cohort age tracking is now a flag like spitfire or hydro that can be turned on with the flag hlm_use_cohort_age_tracking. These changes add this flag to ctsm code. This is for use when FATES is running.] Fixes: [NGT-ED Github issue #] User interface changes?: [No - only on the FATES side of things] Code review: [Names] Test suite: [suite name, machine, compilers] Test baseline: Test namelist changes: Test answer changes: [bit for bit, roundoff, climate changing] Test summary: No testing.

[ Add NPLANT_CAPF to user_nl_clm - number of plants by cohort age class and pft. ] Fixes: [NGT-ED Github issue #] User interface changes?: [No] Code review: [Names] Test suite: [suite name, machine, compilers] Test baseline: Test namelist changes: Test answer changes: [bit for bit, roundoff, climate changing] Test summary:No testing.

…olidate the initialization sequence, and also reduce fates side circular dependencies. This also fixes a bug where some parteh structures were not being passed to this history dimensions properly.

ekluzek · 2021-02-01T18:31:18Z

I've done some work on the build-namelist tests, and found some issues that definitely need some attention. So it'll be good to get these changes in place. A lot of the fates tests weren't working because of this, so this will be good to have in.

…ename

glemieux · 2021-02-02T07:26:02Z

Rerunning all tests.
aux_clm on cheyenne: /glade/u/home/glemieux/scratch/tests_0201-181023ch

All non-fates testmods PASS b4b.
All fates testmods run with expected DIFFs

fates on cheyenne: /glade/u/home/glemieux/scratch/clmed-tests/basegen.fates-sci.1.43.2_api.14.2.0-ctsm5.1.dev021-Cc8b23395-F5534a940

Results: All expected PASS b4b

aux_clm on izumi: /scratch/cluster/glemieux/tests_0202-002203iz

In progress

glemieux · 2021-02-02T20:09:17Z

There are a number of failures for the aux_clm test on izumi. They seem to fall into three categories:

RUN: I'm not sure how to parse the error here. It's only showing up on ERP_D_Ld5.f10_f10_musgs.I1850Clm50Bgc.izumi_intel.clm-nlevgrnd_small
SUBMIT: These all appear to be due to the fates parameter needing to be added to /fs/cgd/csm/inputdata/lnd/clm2/paramdata/
SHAREDLIB_BUILD: I think this might be due to the nag compiler complaining about mixed argument types in PRTAllometricCNPMod. Testing this now.
GENERATE: This is just because I passed -g to run_sys_test and didn't have permission to write to the baseline dir.

glemieux · 2021-02-02T20:25:22Z

@ekluzek noted that the RUN error was just a glitch with PBS. Re-submitting the test results in b4b PASS and I was able to write to baseline.

…spitfire settings that don't need it to 'none' when light_res is none do not set any of the lighting settings, remove use_cn setting for lightning as not needed

…x/CTSM into fates_main_api-merge-ctsm_master

…on't have to have a different copy for each

glemieux · 2021-02-03T21:57:09Z

The SHAREDLIB_BUILD actually appears to be a nag panic and is not due to the mixed argument types:

Panic: /home/glemieux/ctsm/src/fates/parteh/PRTAllometricCNPMod.F90: Unexpected expr node type 432
Internal Error -- please report this bug
Abort
gmake: *** [PRTAllometricCNPMod.o] Error 3
gmake: *** Waiting for unfinished jobs....

NOTE: the debug version of these tests PASS, although only a subset of the failing list have debug versions.

@ekluzek and I chatted a bit about this. One test was to see if we could simply update the nag compiler to 7.0. I attempted to do this for both this PR branch and master, but both of them are failing at the same point in the mct build process. Troubleshooting this line is beyond my capabilities I think.

The other option we considered was trying to see if there is a version of fates_main_api with PRTAllometricCNPMod that might pass this test on izumi. I'm currently looking into this.

glemieux · 2021-02-04T17:51:41Z

@billsacks per our conversation at the ctsm software meeting here are the nag tests that are failing SHAREDLIB_BUILD:

FAIL ERP_Ld5_P48x1.f10_f10_musgs.I1850Clm50Bgc.izumi_nag.clm-ciso SHAREDLIB_BUILD time=170
FAIL ERP_Ld5_P48x1.f10_f10_musgs.I1850Clm50Bgc.izumi_nag.clm-default SHAREDLIB_BUILD time=3
FAIL ERP_Ld5_P48x1.f10_f10_musgs.I2000Clm50BgcCru.izumi_nag.clm-flexCN_FUN SHAREDLIB_BUILD time=5
FAIL ERP_Ld5_P48x1.f10_f10_musgs.I2000Clm50BgcCru.izumi_nag.clm-luna SHAREDLIB_BUILD time=6
FAIL ERP_Ld5_P48x1.f10_f10_musgs.I2000Clm50BgcCru.izumi_nag.clm-noFUN_flexCN SHAREDLIB_BUILD time=4
FAIL ERP_Ld5_P48x1.f10_f10_musgs.I2000Clm50BgcCru.izumi_nag.clm-reduceOutput SHAREDLIB_BUILD time=4
FAIL ERS_Ly5_Mmpi-serial.1x1_numaIA.I2000Clm50BgcCropQianRs.izumi_nag.clm-monthly SHAREDLIB_BUILD time=96
FAIL SMS_Ld1_Mmpi-serial.f45_f45_mg37.I2000Clm50Sp.izumi_nag.clm-ptsRLA SHAREDLIB_BUILD time=3

Note that it looks like only ciso and reduceOutput have exact debug versions.

There is a compiler error with nag when building in non-debug mode with the latest FATES code. The non-debug nag tests don't seem to have much value, so I'm removing them - either changing them to debug tests or removing them entirely, depending on whether a given configuration seems to have sufficient test coverage.

billsacks · 2021-02-04T21:03:33Z

@glemieux @ekluzek I just pushed a commit that removes nag non-debug tests - either changing them to debug tests or removing them entirely, based on an analysis of existing test coverage.

billsacks · 2021-02-04T21:06:55Z

@glemieux you'll get BFAIL results (due to missing baselines) for the new tests. I checked the test list, and I think we have sufficient coverage of these configurations in other tests that you don't need to worry about this - i.e., you do not need to generate baselines for these tests from master.

glemieux · 2021-02-04T21:09:27Z

@billsacks thanks! @ekluzek I'll rerun the aux_clm and generate dev022 baselines on izumi with this.

glemieux · 2021-02-04T22:54:44Z

Re-ran aux_clm on izumi. All expected pass b4b: /scratch/cluster/glemieux/tests_0204-142628iz. Only fates testmods have DIFFs which are expected.

…template

ekluzek

OK, most things I asked for were addressed. We moved a few things to their own issues to be done later. But, this is an exciting development for CTSM and FATES so good to have this tag in place!

bld/namelist_files/namelist_defaults_ctsm.xml

bld/unit_testers/build-namelist_test.pl

src/biogeophys/WaterStateType.F90

…x/CTSM into fates_main_api-merge-ctsm_master

wwieder · 2021-02-05T12:47:24Z

WHAT! This is awesome news. Thanks for all your hard work on this.

…

On Fri, Feb 5, 2021 at 12:06 AM Erik Kluzek ***@***.***> wrote: Merged #1264 <#1264> into master. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1264 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB5IWJHSK5D56PESGLVMGPLS5OKJDANCNFSM4WU4HOMA> .

glemieux and others added 30 commits June 22, 2020 15:31

Removing calls to retired machine edison. Adding a SMS test to the

f80b00f

gnu test suite. Changing highest res grid test to clm5.

reinstating edison tests to prebeta

daca97b

Initialize soil moisture to a higher value for cold-starts when fates…

728d9d7

… hydro is active.

Added surface runoff modifier to the fates interface to account for s…

b7f1b4c

…uper-saturation during plant hydraulics.

Condition the zeroing of qflx bc_in upon having hydro on.

b827ac7

Update fates interface to use a call to fatesglobals instead of just …

812112a

…prtglobals

Updating fates tag and default parameter file match api 8.1

97f2471

Provisions for fates-hydro, supersaturating soils.

d18d552

Allowing soilpsi_col diagnostic with fates

f49862a

Allowing soilpsi_col diagnostic with fates

41a61b2

fates-hydro: changed subroutine arguments

e906294

Fixed hydro restart bug (needed to initialize bcs prior to initializi…

5f3c439

…ng wrfs).

fates-hydro: removed dependence on qflx_drain_vr_col

5e6fdab

fixed reverting files to the wrong version of fates_next_api

f76d039

Added FATESSizeAge test protocol

d81ca0e

Fixed fates size-age-mortality test length to be splitable

4bed4f9

update manage externals config file to pull correect fates tag

f1b579a

remove temp writes to iulog in clmfates_interface

9eda517

removing remnant of old code from htape_addfld

415f89f

fates-hydro, api update for hydro refactors, bug fix

683d14e

added time-step argument to fix nep calculations

df9e930

fates-hydro: updated test user_nl_clm

6a7d5b5

updated fates parameter file, backwards compatible

c669928

Updating fates tag in the externals pointer

5f71faf

no for real this time, fixing merge conflicts

31a59ef

fates interface refactor. this interface refactor is intended to cons…

77a21d4

…olidate the initialization sequence, and also reduce fates side circular dependencies. This also fixes a bug where some parteh structures were not being passed to this history dimensions properly.

Removed site level veg24 since it was meaningless

93b03e1

glemieux added 5 commits February 1, 2021 10:39

create FATES version of fire factory

08b1f87

adding FATESFireFactoryMod

deb57e0

Merge branch 'fates_main_api-merge-ctsm_master' into fates_firemods_r…

1ab763c

…ename

fixing missing end module name

b8abb88

removing old call to fire_method = spitfire in ignitions testmod

c8b2339

ekluzek added 3 commits February 2, 2021 23:07

Set T62 lightning dataset for fates as well, set light_res for fates …

9f0916b

…spitfire settings that don't need it to 'none' when light_res is none do not set any of the lighting settings, remove use_cn setting for lightning as not needed

Merge branch 'fates_main_api-merge-ctsm_master' of github.com:glemieu…

0e645ea

…x/CTSM into fates_main_api-merge-ctsm_master

Remove the use_cn and use_fates settings for lightning variables so d…

2f27c68

…on't have to have a different copy for each

updating changelog template

089eb79

updating changelog notes for dev022 and removing fates checkbox from …

0d52de5

…template

ekluzek mentioned this pull request Feb 4, 2021

Some build-namelist refactoring for FATES/CN fire data #1272

Open

ekluzek approved these changes Feb 4, 2021

View reviewed changes

bld/namelist_files/namelist_defaults_ctsm.xml Show resolved Hide resolved

bld/namelist_files/namelist_defaults_ctsm.xml Show resolved Hide resolved

bld/unit_testers/build-namelist_test.pl Show resolved Hide resolved

src/biogeophys/WaterStateType.F90 Show resolved Hide resolved

ekluzek added 3 commits February 4, 2021 17:00

Merge branch 'fates_main_api-merge-ctsm_master' of github.com:glemieu…

50d5e71

…x/CTSM into fates_main_api-merge-ctsm_master

Add two fail tests for light-res set to none, but fire required to be on

5ee6ce2

Some more work with the Change files

c49dcbc

ekluzek merged commit b93a60e into ESCOMP:master Feb 5, 2021

glemieux mentioned this pull request Feb 5, 2021

FATES testing update and cleanup #1275

Merged

4 tasks

samsrabin added the science Enhancement to or bug impacting science label Aug 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Merge fates_main_api into master #1264

Merge fates_main_api into master #1264

glemieux commented Jan 27, 2021 •

edited

Loading

ekluzek commented Feb 1, 2021

glemieux commented Feb 2, 2021 •

edited

Loading

glemieux commented Feb 2, 2021 •

edited

Loading

glemieux commented Feb 2, 2021

glemieux commented Feb 3, 2021 •

edited

Loading

glemieux commented Feb 4, 2021 •

edited

Loading

billsacks commented Feb 4, 2021

billsacks commented Feb 4, 2021

glemieux commented Feb 4, 2021

glemieux commented Feb 4, 2021

ekluzek left a comment

wwieder commented Feb 5, 2021 via email

Merge fates_main_api into master #1264

Merge fates_main_api into master #1264

Conversation

glemieux commented Jan 27, 2021 • edited Loading

Description of changes

Specific notes

ekluzek commented Feb 1, 2021

glemieux commented Feb 2, 2021 • edited Loading

glemieux commented Feb 2, 2021 • edited Loading

glemieux commented Feb 2, 2021

glemieux commented Feb 3, 2021 • edited Loading

glemieux commented Feb 4, 2021 • edited Loading

billsacks commented Feb 4, 2021

billsacks commented Feb 4, 2021

glemieux commented Feb 4, 2021

glemieux commented Feb 4, 2021

ekluzek left a comment

Choose a reason for hiding this comment

wwieder commented Feb 5, 2021 via email

glemieux commented Jan 27, 2021 •

edited

Loading

glemieux commented Feb 2, 2021 •

edited

Loading

glemieux commented Feb 2, 2021 •

edited

Loading

glemieux commented Feb 3, 2021 •

edited

Loading

glemieux commented Feb 4, 2021 •

edited

Loading