Add efcs and epos to ufs_hybatm xml (#3192)

[RussTreadon-NOAA]

Description

g-w PR #3185 inadvertently removed metatasks enkfgdas_fcst and enkfgdas_epmn from the ufs_hybatmDA xml. This PR modifies workflow/applications/gfs_cycled.py so that these metatasks are once again in the ufs_hybatmDA xml.

Resolves: #3192

Type of change

• Bug fix (fixes something broken)

Change characteristics

• Does this change require a documentation update? NO
• Does this change require an update to any of the following submodules? NO

How has this been tested?

1. clone and install bugfix/ufs_hybatmDA on Dogwood
2. set up g-w CI C96C48_ufs_hybatmDA by executing $HOMEgfs/workflow/create_experiment.py
3. open C96C48_ufs_hybatmDA xml
4. confirm that metatasks enkfgdas_fcst and enkfgdas_epmn are defined

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• My changes generate no new warnings
• New and existing tests pass with my changes
• This change is covered by an existing CI test or a new one has been added

[aerorahul]

looks good to me.

[WalterKolczynski-NOAA]

I think the earc dependencies will need to be added back as well.

[RussTreadon-NOAA]

@WalterKolczynski-NOAA , feel free add the earc changes to my branch. Not sure if this will work, but give it a try. If it fails, point me at your earc changes and I'll add them. Alternatively, you can open a new issue and PR and pull my changes into your PR.

[WalterKolczynski-NOAA]

I'm on leave until Jan 2, but you can see what #3185 took out of gfs_tasks.py and put it back in.

[RussTreadon-NOAA]

No rush. We can wait until after everyone returns from leave.

[RussTreadon-NOAA]

Limited g-w CI on Hera

Ran g-w CI for C48mx500_3DVarAOWCDA, C96C48_hybatmaerosnowDA, and C96C48_ufs_hybatmDA on Hera from an installation of g-w bugfix/ufs_hybatmDA at 5acc46c.

All jobs in C48mx500_3DVarAOWCDA and C96C48_ufs_hybatmDA ran to completion.

The 20211220 18Z C96C48_hybatmaerosnowDA enkfgdas_earc01 failed with the error message

  File "/scratch1/NCEPDEV/da/Russ.Treadon/git/global-workflow/test/parm/archive/enkf_restartb_grp.yaml.j2", line 41, in template
    {% endfor %}
^^^^^^^^^^^^^^^^^
jinja2.exceptions.TemplateSyntaxError: Encountered unknown tag 'endfor'. You probably made a nesting mistake. Jinja is expecting this tag, but currently looking for 'elif' or 'else' or 'endif'. The innermost block that needs to be closed is 'if'.
+ JGDAS_ENKF_ARCHIVE[1]: postamble JGDAS_ENKF_ARCHIVE 1735397434 1

The line 41 {% endfor %} is one line before the end of enkf_restartb_grp.yaml.j2

        {% endfor %}
        {% endfor %}
        {% endif %}

This is not correct. The template needs to end with {% endfor %} to close the top-level loop over members

nkf_restartb_grp:
    name: "ENKF_RESTARTB_GRP"
    target: "{{ ATARDIR }}/{{ cycle_YMDH }}/{{ RUN }}_restartb_grp{{ ENSGRP }}.tar"
    required:
        {% for mem in range(first_group_mem, last_group_mem + 1) %}

Make the following change to parm/archive/enkf_restartb_grp.yaml.j2

@@ -38,5 +38,5 @@ enkf_restartb_grp:
         - "{{ COMIN_ATMOS_RESTART_MEM | relpath(ROTDIR) }}/{{ r_prefix }}.coupler.res"
         - "{{ COMIN_ATMOS_RESTART_MEM | relpath(ROTDIR) }}/{{ r_prefix }}.fv_core.res.nc"
         {% endfor %}
-        {% endfor %}
         {% endif %}
+        {% endfor %}

With this change in place, the 20211220 18Z enkfgdas_earc01 successfully ran to completion.

[TravisElless-NOAA]

@RussTreadon-NOAA Apologies for the inconvenience my PR errors caused and thank you for your efforts to correct them.

One suggestion: instead of adding the lines to the top half of the do_jediatmens if statement, we can just pull the lines #3185 added in the bottom half of the if statement outside of the if statement all together.

The earc dependencies are still fine and won't have to be updated. Changes to the dependencies in #3185 focused on the requirements between the early- and late-cycle. This PR focuses on restoring only late-cycle jobs for the UFS DA configuration and maintains that the UFS DA will still have the fcst and pos jobs turned off for the early-cycle.

[RussTreadon-NOAA]

Requested changes have been made. Once everyone is OK with the changes in this PR, we should run run g-w CI on Hera to ensure no unintended consequences.

[RussTreadon-NOAA]

Thank you @TravisElless-NOAA . I'll start g-w CI on Hera once I receive confirmation (approval) from the g-w team that the changes in this PR are OK with them. We don't want to run CI multiple times. Hence my waiting.

[aerorahul]

lgtm

[RussTreadon-NOAA]

Thank you @aerorahul for the approval and triggering Hera and Hercules CI.

[emcbot]

CI Passed on Hercules in Build# 1
Built and ran in directory /work2/noaa/global/CI/HERCULES/3193

Experiment C48_ATM_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 12:46:27 CST 2025
Experiment C48mx500_3DVarAOWCDA_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 12:52:30 CST 2025
Experiment C48mx500_hybAOWCDA_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 12:58:39 CST 2025
Experiment C96C48_hybatmDA_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 13:53:20 CST 2025
Experiment C96_S2SWA_gefs_replay_ics_e93036d4 Completed 1 Cycles: *SUCCESS* at Thu Jan  2 13:53:42 CST 2025
Experiment C96_atm3DVar_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 13:59:12 CST 2025
Experiment C48_S2SW_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 14:18:09 CST 2025
Experiment C48_S2SWA_gefs_e93036d4 Completed 1 Cycles: *SUCCESS* at Thu Jan  2 16:19:42 CST 2025

[emcbot]

CI Passed on Hera in Build# 2
Built and ran in directory /scratch1/NCEPDEV/global/CI/3193

Experiment C48mx500_3DVarAOWCDA_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 19:19:31 UTC 2025
Experiment C48mx500_hybAOWCDA_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 19:19:31 UTC 2025
Experiment C48_ATM_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 19:25:46 UTC 2025
Experiment C96_S2SWA_gefs_replay_ics_e93036d4 Completed 1 Cycles: *SUCCESS* at Thu Jan  2 19:33:19 UTC 2025
Experiment C96C48_hybatmaerosnowDA_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 20:42:56 UTC 2025
Experiment C96C48_hybatmDA_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 20:42:57 UTC 2025
Experiment C96_atm3DVar_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 20:52:16 UTC 2025
Experiment C96C48_ufs_hybatmDA_e93036d4 Completed 3 Cycles: *SUCCESS* at Thu Jan  2 21:29:12 UTC 2025
Experiment C48_S2SW_e93036d4 Completed 2 Cycles: *SUCCESS* at Thu Jan  2 21:36:54 UTC 2025
Experiment C48_S2SWA_gefs_e93036d4 Completed 1 Cycles: *SUCCESS* at Thu Jan  2 22:14:57 UTC 2025

[RussTreadon-NOAA]

@WalterKolczynski-NOAA : Hera and Hercules g-w CI pass. Is this sufficient or do we also need to run g-w CI on WCOSS2 (Dogwood)?

[RussTreadon-NOAA]

Thank you @aerorahul