-
Notifications
You must be signed in to change notification settings - Fork 259
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a RT for SFS with hydrostatic options #2255
Add a RT for SFS with hydrostatic options #2255
Conversation
@XiaqiongZhou-NOAA running ORTs against the new cpld_gfsv17_sfs case with gnu compilers. Right now getting some failures on the bit, debug, standard, decomp, and restart ORTs. The compile is fine, the failure occurs during the test run. "An MPI communication peer process has unexpectedly disconnected." 180: *** WAVEWATCH III ERROR IN W3IOGR : Example test experiment directory: /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_2606214/bit_base_bit_base/ looking at err and out logs. |
Check that the mod_def file from WW3 is the correct one |
@BrianCurtis-NOAA is correct. Based on rt.sh and your test, you should be using /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20221101/WW3_input_data_20240214/mod_def.exp.global_270k for your mod_def, but: Binary files /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_2606214/bit_base_bit_base/mod_def.ww3 and /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20221101/WW3_input_data_20240214/mod_def.exp.global_270k differ I think you're using the wrong WW3_input data somehow as: /scratch1/NCEPDEV/stmp2/Zachary.Shrader/FV3_OPNREQ_TEST/opnReqTest_2606214/bit_base_bit_base/mod_def.ww3 /scratch2/NAGAPE/epic/UFS-WM_RT/NEMSfv3gfs/input-data-20221101/WW3_input_data_20220624/mod_def.exp.global_270k |
ORTs passed with WW3_input_data_20240214. |
@XiaqiongZhou-NOAA @NickSzapiro-NOAA Can you please sync up the branch for this PR? |
The branch is synced.
…On Mon, May 13, 2024 at 1:55 PM zach1221 ***@***.***> wrote:
@XiaqiongZhou-NOAA <https://github.com/XiaqiongZhou-NOAA>
@NickSzapiro-NOAA <https://github.com/NickSzapiro-NOAA> Can you please
sync up the branch for this PR?
—
Reply to this email directly, view it on GitHub
<#2255 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ALQE7UV5MGNEKEKFYEHHMCLZCD5CFAVCNFSM6AAAAABG22ZRWGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBYGQ2DGMJZHE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Looks like FV3ATM is supposed to be at: XiaqiongZhou-NOAA/fv3atm@fc2630c This hash was after the indentation change I requested. |
Pointed to new Fv3atm.
…On Mon, May 13, 2024 at 2:11 PM Brian Curtis ***@***.***> wrote:
Looks like FV3ATM is supposed to be at: ***@***.***
<XiaqiongZhou-NOAA/fv3atm@fc2630c>
This hash was after the indentation change I requested.
—
Reply to this email directly, view it on GitHub
<#2255 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ALQE7UQNTRKC5G6QUTWIYUDZCD65HAVCNFSM6AAAAABG22ZRWGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBYGQ4TQNRTGU>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
@zach1221 This PR is not ready for the commit Q unless all ORT tests are completed and passed. This test is a possible implementation target and must successfully pass all ORTs at the time of commit. |
Disregard the Orion failure message above. |
I'll follow up on the cubed-sphere sub-pr and we can begin the merge process. |
cpld_warmstart_c48 should need approximately same amount of memory as cpld_control_c48? How did this test work before? Which change in this PR is causing this test to require more memory now? |
I had the same questions. Denise mentioned the same to me earlier as well. The run completed fully (as in generated all the sfcf and atmf files it needed) but right at the end it had this, and the comparisons were not run:
|
This does not indicate any error. If the model finished successfully then the issue is not that the test is using too much memory. There must be some other reason why the comparisons were not run. Please revert to original setup and try again. |
the E in PBS is not for "error" but "Exiting" and caused the ecflow job to terminate before finishing even when the model did finish |
Hmm. This was never an issue. Before the recent 'cleanup' code that handled the 'E' status looked like:
That part of code has been completely rewritten. I wonder how many more subtle error like this we are going to find in the future. For example the code that checks the 'Exit_status' of the finished job:
is completely removed. Why? |
Also, this logic has been removed:
|
@DusanJovic-NOAA I don't think any of this pertains to this PR, we can move forward with this PR at this point. Please feel free to start an e-mail discussion on recent rt.sh and rt_utils.sh changes. |
@XiaqiongZhou-NOAA Fv3atm merged. Please update submodule hash and revert change to .gitmodule url. |
done.
…On Wed, May 15, 2024 at 4:38 PM zach1221 ***@***.***> wrote:
@XiaqiongZhou-NOAA <https://github.com/XiaqiongZhou-NOAA> Fv3atm merged.
Please update submodule hash and revert change to .gitmodule url.
Hash: ***@***.***
<NOAA-EMC/fv3atm@9534e14>
—
Reply to this email directly, view it on GitHub
<#2255 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ALQE7UX2AAEHTIACLDPFGVTZCPBULAVCNFSM6AAAAABG22ZRWGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJTGQYTMNJYGE>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
@DeniseWorthen @NickSzapiro-NOAA can you please review your unresolved conversation? |
@zach1221 I marked my comment as resolved. I don't see any other options... This PR still needs commit messages, fwiw |
@XiaqiongZhou-NOAA I think Fernando mentioned this previously, but can you please fill out the commit message section of the PR template, which summarizes briefly your code changes included in this PR and sub-component PRs? |
Sorry, I did not notice. Commit message is added. |
Commit Queue Requirements:
Description:
Add a RT for the seasonal forecast system with hydrostatic options.
Compiling flags -DUSE_COND and -DMOIST_CAPPA are removed if this is a hydrostatic run.
inline post is modified to output omega and HGT for hydrostatic runs.
Commit Message:
Priority:
Sub component Pull Requests:
FV3: Change compiling flag for hydrostatic options NOAA-EMC/fv3atm#828
GFDL_atmos_cubed_sphere Remove -DUSE_COND and -DMOIST_CAPPA for hydrostatic runs NOAA-GFDL/GFDL_atmos_cubed_sphere#337
Closes turn threaded tests back on for Derecho #2242
Changes
Regression Test Changes (Please commit test_changes.list):
Input data Changes:
Library Changes/Upgrades:
Testing Log: