More thoroughly investigate answer changes in ctsm1.0.dev001

[billsacks]

ctsm1.0.dev001 included the following answer changes:

• Larger than roundoff-level changes expected from the following:
  • Include qflx_h2osfc_surf in BGC code: expected to be same climate
  • Two changes just affecting VIC
  • A change just affecting the QOVER diagnostic field
• Additional changes believed to be roundoff-level

June 28, 2018, @olyson ran the ensemble consistency test against release-clm5.0.01 (I'm not sure if this was the 9-timestep F compset UF-ECT, or the 1-year B compset ECT).

Should we do more thorough testing on this tag to ensure that it hasn't changed answers relative to release-clm5.0.01? e.g., should we do longer runs and run the diagnostics package on this, or should we run the 1-year ECT if only the UF-ECT was run?

[billsacks]

@dlawrenncar @olyson feel free to mark this as a wontfix if you feel like this has been tested sufficiently.

[olyson]

I ran the 9step test on this just now and it passes overall but fails one PC for one ensemble member.

[billsacks]

@olyson what was the baseline tag for this?

[olyson]

It is supposed to be pointing to this:
/gpfs/fs1/p/cesmdata/cseg/inputdata/validation/uf_ensembles/cesm2.0

[olyson]

There is an attribute added to the netcdf files that I uploaded that include the model version:

model_version=cesm2.0.0

I used release-cesm2.0.0 but with ctsm1.0.dev001 to run the simulations required.

[billsacks]

Huh, that's strange. cesm2.0.0 uses release-clm5.0.01, which is what you used as a baseline last June (I think). But you're getting different conclusions now vs. before?

[olyson]

Yes, I replaced release-clm5.0.01 in my checkout of release-cesm2.0.0 with ctsm1.0.dev001
The difference is limited to 1 PC score for 1 run. Maybe due to compiler or module changes? I had to update the modules to get this to work.

[billsacks]

Oh, okay: now that I look more closely, I see that the overall result is a pass, like before.

So I guess the question remains, for you and/or @dlawrenncar , whether this is sufficient or we should do some additional confirmation.

[dlawrenncar]

I think this is sufficient. On Apr 24, 2019, at 5:08 PM, Bill Sacks <[email protected]> wrote: Oh, okay: now that I look more closely, I see that the overall result is a pass, like before. So I guess the question remains, for you and/or @dlawrenncar <https://github.com/dlawrenncar> , whether this is sufficient or we should do some additional confirmation. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#690 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFABYVDQT25U4VMFLEIAZXLPSDR6VANCNFSM4HH5VCHQ> .

[billsacks]

Thanks, @dlawrenncar . I'll close this issue then.