-
Notifications
You must be signed in to change notification settings - Fork 245
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
PGI answers for OM4 do not match across restart #257
Comments
When I set #override DEBUG = True the model came down with: FATAL from PE 20: NaN detected: after ePBL Kd_ePBL stdout: |
The restart issue also exists on c2 with pgi14 |
The problem exists in the much smaller (32 pes) test case MOM6_SIS2_bergs_cgrid Also, I think the issue exists only in prod mode (-O3) and goes away in repro mode (-O2). |
- Kd_ePBL has intent(out) from the PBL routine but was being initialized by the calling routine and conditionally on whether a diagnostic is active: - Kd_ePBL is a mandatory output, no-longer conditional on the diagnostic; - initializing values by the caller is not guaranteed to work when the callee has intent(out); - ePBL was only setting values for non-land points. - The conditional initialization has been removed entirely. - ePBL now sets values on all computational points. - Addresses issue discussed in #257 where DEBUG=True was detecting NaN's in Kd_ePBL.
DEBUG=True issue has been fixed in b4f8cdb -A. Dr Alistair Adcroft ([email protected]) On Wed, Mar 16, 2016 at 4:38 PM, Niki Zadeh [email protected]
|
* Move mct_cap/ to STALE_mct_cap/. mct cap is no longer supported and will soon be removed for good. * remove mct from CI testing * Remove mct test from github workflows
MOM6 tag 2016.03.02 (the issue is probably much older)
When compiled with pgi15 on c3 the experiment OM4_SIS2_low_mixing3 does not reproduce when a restart is involved
(i.e., 2 days restart files do not match 1+1 days restart files) (irrespective of mask_table being used or not).
There is no such issue for the intel15 runs on c3.
I have not seen a non-repeatability issue (answer changes between dual runs) with pgi15 for this experiment (3 runs so far) . So this is most probably a restart issue (75%).
The text was updated successfully, but these errors were encountered: