ufs.configure.s2sw_slow.IN is not using ESMF threading

[JessicaMeixner-NOAA]

Description

Perhaps not a "bug" but an observation I made is that ufs.configure.s2sw_slow.IN is not templated to use ESMF managed threading. I think this is simply an oversight not necessarily intentional. I noticed this when updating the model in the workflow as this is inconsistent between the two.

Additional Info

Look at the https://github.com/ufs-community/ufs-weather-model/blob/develop/tests/parm/ufs.configure.s2sw_slow.IN and note we do not have omp for each component. Since this is used in the GFSv17 tests, which should likley use esmf managed threading since that's currently the goal and what's used in the workflow.

[junwang-noaa]

Thanks for sharing your observations. Actually we are considering to have one traditional threading test depending on the HR3b timing testing. At this time we can update the template here to be consistent with GW if needed.

[junwang-noaa]

We will add a new template ufs.configure.s2swa_slow.IN corresponding to ufs.configure.s2sw_slow.IN and have the s2swa coupled tests to use this template until the issue #1764 is resolved.

[junwang-noaa]

@JessicaMeixner-NOAA Currently ufs.configure.s2sw.IN has ESMF managed threading, the difference between ufs.configure.s2sw_slow.IN and ufs.configure.s2sw.IN is that wave is running in slow or fast coupling loop. If I remember correctly, your group is testing the timing of wave coupling in slow and fast loop, may I ask if you have the results yet? My understanding is the moving wave to slow loop does not improve the computational/scientific performance, if that is the case, can we remove the ufs.configure.s2sw_slow.IN? Otherwise, maybe we can remove the ufs.configure.s2sw.IN and rename ufs.configure.s2sw_slow.IN to ufs.configure.s2sw.IN rather than keeping two templates? Thanks

[JessicaMeixner-NOAA]

@junwang-noaa I will search for the results to post here. The summary is that it was computationally faster to use the slow loop in the GEFS and GFS configurations we tested. I think the reason some tests might not have seen this advantage was because of resolutions being tested or WW3 internal time-steps. Both GFSv17 and GEFSv13 are moving forward with the wave component in the slow-loop at this time.

[JessicaMeixner-NOAA]

Here's a slide I found with one figure and some results:

I have another spreadsheet from some old P8c runs where with 1/4 deg inner loop, it took ~ 3964s/day whereas outerloop was only 600s per day. Again, a lot of this depends on what the inner versus outerloop timesteps are and what the corresponding WW3 time steps in the two scenarios are. I do not have recent timing results. I believe @DeniseWorthen wanted the innerloop configuration for some ice-coupling work so I'd double check with her before removing the inner-loop option.

[junwang-noaa]

Thanks for confirming that the wave grid or internal time step themselves do not have impact on the total timing, the outloop P8c runs faster because the wave runs with larger time step. We will make plan to remove the wave fast loop coupling then. Thanks.

[junwang-noaa]

@JessicaMeixner-NOAA a quick question before we remove the wave fast loop template, can the unstructured mesh grid use the larger time step too? Just want to confirm we don't need to keep two templates.

[DeniseWorthen]

@junwang-noaa We need to keep at least one configuration with waves in the fast loop for the wave-ice coupling.

[junwang-noaa]

@JessicaMeixner-NOAA @DeniseWorthen We will keep tests to use both fast loop and slow loop for wave at this time. The ufs.configure.s2sw.IN has wave in slow loop and use traditional threads due to issues in GFSv3 HR3b (c1152). Please let me know if you have further questions.

[junwang-noaa]

The code changes in Rahul's PR #2074, which was committed on Jan 9.

[junwang-noaa]

The PR was committed in PR#2172, close issue.