[develop] Point source pre-combine

[zmoon]

DESCRIPTION OF CHANGES:

Update the workflow to use the pre-combined point source data files (NOAA-EMC/AQM-utils#4) in order to reduce runtime for Online-CMAQ with explicit point source on. This is a breaking change (if using explicit point source) since the invocation of the point source data merge tool has changed slightly.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

TESTS CONDUCTED:

• hera.intel
• orion.intel
• cheyenne.intel
• cheyenne.gnu
• gaea.intel
• jet.intel
• wcoss2.intel
• NOAA Cloud (indicate which platform)
• Jenkins
• fundamental test suite
• comprehensive tests (specify which if a subset was used)

DEPENDENCIES:

DOCUMENTATION:

ISSUE:

CHECKLIST

• My code follows the style guidelines in the Contributor's Guide
• I have performed a self-review of my own code using the Code Reviewer's Guide
• I have commented my code, particularly in hard-to-understand areas
• My changes need updates to the documentation. I have made corresponding changes to the documentation
• My changes do not require updates to the documentation (explain).
• My changes generate no new warnings
• New and existing tests pass with my changes
• Any dependent changes have been merged and published

LABELS (optional):

A Code Manager needs to add the following labels to this PR:

• Work In Progress
• bug
• enhancement
• documentation
• release
• high priority
• run_ci
• run_we2e_fundamental_tests
• run_we2e_comprehensive_tests
• Needs Cheyenne test
• Needs Jet test
• Needs Hera test
• Needs Orion test
• help wanted

CONTRIBUTORS (optional):

@ytangnoaa @chan-hoo

[zmoon]

Note that (AFAIK) the new data aren't currently available on WCOSS2.

[chan-hoo]

@zmoon, please update the hash of AQM-utils with "0a86f73".

[chan-hoo]

@bbakernoaa, @ytangnoaa, @zmoon, please copy the necessary (new data) files to WCOSS2.

[ytangnoaa]

@bbakernoaa, @ytangnoaa, @zmoon, please copy the necessary (new data) files to WCOSS2.

I copied to /lfs/h2/emc/physics/noscrub/Youhua.Tang/nei2016v1-pt/v2023-01-PT

[chan-hoo]

@zmoon, this change looks working well on Hera and WCOSS2. Please merge the latest develop branch to resolve the conflicts, then I'll approve this PR.

[zmoon]

@chan-hoo I resolved the merge conflict, was just the AQM-utils hash

[MichaelLueken]

@zmoon @chan-hoo Since this work will be going into develop, I have gone ahead and ran the fundamental WE2E tests on Jet. All tests successfully passed. I will now launch the Jenkins tests for Cheyenne, Gaea, and Orion. Being unable to run AQM/CMAQ tests on WCOSS2 and Hera, I'm not sure if you would prefer someone else to review and test these changes on one of those machines. The changes look straightforward, but I will wait to approve until given the go ahead. Thanks!

[panll]

Looks good to me!

[chan-hoo]

@panll, thank you for your approval! @MichaelLueken, I tested this PR on both Hera and WCOSS2, and I confirmed that it worked well on both machines. I think you can go ahead.

[zmoon]

@chan-hoo do you have a measure of how long the step in the workflow took in your tests on WCOSS2 with this update? Just curious.

[chan-hoo]

@zmoon, Cactus was switched with Dogwood this morning, so I don't have an exact answer now but it took less than before. @JianpingHuang-NOAA, do you have an answer to this?

[JianpingHuang-NOAA]

@zmoon @chan-hoo It took 512 s (8.5 minutes) and 506 s (8.4 minutes) to complete the task at 06z and 12z cycles on 20230405, respectively. It is much faster than the previous runtime (>43 minutes). This was tested on Cactus. I am setting up the NRT runs on Dogwood now.

[chan-hoo]

@zmoon @ytangnoaa @drnimbusrain, can you copy the new data files to Cheyenne too?

[bbakernoaa]

@chan-hoo I don't believe we have accounts on cheyenne to copy it there.

[MichaelLueken]

Manual testing on Jet and the automated Jenkins testing on the rest of the machines have all successfully passed. I will now approve these changes.

[MichaelLueken]

@chan-hoo Just want to double check, is this PR is ready to merge? If so, I can go ahead and do so. Also, for copying data to Cheyenne, please reach out to @natalie-perlin to see if she can add the new data to the EPIC maintained fixed file location. Also, @padhrigmccarthy is working on AQM on Cheyenne, so it would be a good idea to let him know about this update as well. Please make sure that they know where they can pull the data from. Thanks!

[chan-hoo]

@MichaelLueken, you can merge this PR, and I agree with you. @drnimbusrain, can you let @padhrigmccarthy know how to copy them to Cheyenne. For Cheyenne, I'll be able to update the script when I open my PR for fixing the AQM issues soon. Currently, AQM does not work in the develop branch (#709). Once it is fixed, @padhrigmccarthy can test this update on Cheyenne.