PBL, Convection and Microphysics Update for HR2

[Qingfu-Liu]

Description

This PR is created because the ccpp-physics is updated for HR2 (PR#65). The ccpp-physics update in PR#65 includes update for PBL scheme, shallow convection scheme, deep convection scheme and microphysics scheme. The changes of the physics improve hurricane forecast, and CAPE forecast values.

Top of commit queue on: TBD

Input data additions/changes

• No changes are expected to input data.
• There will be new input data.
• Input data will be updated.

Anticipated changes to regression tests:

• No changes are expected to any regression test.
• Changes are expected to the following tests:

The physics update in PR#65 will improve forecast results

Subcomponents involved:

• AQM
• CDEPS
• CICE
• CMEPS
• CMakeModules
• FV3
• GOCART
• HYCOM
• MOM6
• NOAHMP
• WW3
• stochastic_physics
• none

Combined with PR's (If Applicable):

Commit Queue Checklist:

• Link PR's from all sub-components involved
  PBL and Convection and Microphysics update for HR2 ccpp-physics#65
  Physics (PBL, Convection and Microphysics) Update for HR2 NOAA-EMC/fv3atm#652
  PBL, Convection and Microphysics Update for HR2 #1723
• Confirm reviews completed in sub-component PR's
• Add all appropriate labels to this PR.
• Run full RT suite on either Hera/Cheyenne with both Intel/GNU compilers
• Add list of any failed regression tests to "Anticipated changes to regression tests" section.

Linked PR's and Issues:

None

Testing Day Checklist:

• This PR is up-to-date with the top of all sub-component repositories except for those sub-components which are the subject of this PR.
• Move new/updated input data on RDHPCS Hera and propagate input data changes to all supported systems.

Testing Log (for CM's):

• RDHPCS
  • Intel
    • Hera
    • Orion
    • Jet
    • Gaea
    • Cheyenne
  • GNU
    • Hera
    • Cheyenne
• WCOSS2
  • Dogwood/Cactus
  • Acorn
• CI
  • Completed
• opnReqTest
  • N/A
  • Log attached to comment

[jkbk2004]

@Qingfu-Liu can you sync up branch? then we will try to combine in #1715

[grantfirl]

@Qingfu-Liu Also, we still need a list of which RTs are expected to fail. If you run all of the tests in rt.conf on Hera and copy/paste the list of failed tests found at the end of your tests/RegressionTest_hera.intel/gnu.log files, that is what we need.

[Qingfu-Liu]

Hi Jong, I saw that If I sync up the branch, I will discard "1 commit", is this what you want? Qingfu

…

On Fri, Apr 28, 2023 at 3:15 PM JONG KIM ***@***.***> wrote: @Qingfu-Liu <https://github.com/Qingfu-Liu> can you sync up branch? then we will try to combine in #1715 <#1715> — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTS6UST2EA3E4OXP5OWV5DXDQJLLANCNFSM6AAAAAAXKFB7H4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[grantfirl]

Hi Jong, I saw that If I sync up the branch, I will discard "1 commit", is this what you want? Qingfu
…
On Fri, Apr 28, 2023 at 3:15 PM JONG KIM @.> wrote: @Qingfu-Liu https://github.com/Qingfu-Liu can you sync up branch? then we will try to combine in #1715 <#1715> — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGTS6UST2EA3E4OXP5OWV5DXDQJLLANCNFSM6AAAAAAXKFB7H4 . You are receiving this because you were mentioned.Message ID: @.>

@Qingfu-Liu There was a PR merged into ufs-weather-model and fv3atm this morning. This branch and your fv3atm branch needs to pull in the latest develop commits.

[Qingfu-Liu]

@grant Firl - NOAA Affiliate ***@***.***> , all the runs will be failed since we need to create a baseline data, but the runs do produce the output data. If I rerun the tests, it will take very long time (might be 10+ hours) Qingfu

…

On Fri, Apr 28, 2023 at 3:22 PM Grant Firl ***@***.***> wrote: @Qingfu-Liu <https://github.com/Qingfu-Liu> Also, we still need a list of which RTs are expected to fail. If you run all of the tests in rt.conf on Hera and copy/paste the list of failed tests found at the end of your tests/RegressionTest_hera.intel/gnu.log files, that is what we need. — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTS6UXK43UKC2GWU6LU34DXDQKFNANCNFSM6AAAAAAXKFB7H4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[grantfirl]

@grant Firl - NOAA Affiliate @.> , all the runs will be failed since we need to create a baseline data, but the runs do produce the output data. If I rerun the tests, it will take very long time (might be 10+ hours) Qingfu
…
On Fri, Apr 28, 2023 at 3:22 PM Grant Firl @.> wrote: @Qingfu-Liu https://github.com/Qingfu-Liu Also, we still need a list of which RTs are expected to fail. If you run all of the tests in rt.conf on Hera and copy/paste the list of failed tests found at the end of your tests/RegressionTest_hera.intel/gnu.log files, that is what we need. — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGTS6UXK43UKC2GWU6LU34DXDQKFNANCNFSM6AAAAAAXKFB7H4 . You are receiving this because you were mentioned.Message ID: @.***>

If you've already run all the tests in rt.conf, then we just need the list of failed tests from the logs that were already produced. We want the list of failed runs to know whether they're all expected. When I go to repeat this procedure for the NCAR:main branch, the list of failed tests is crucial to know if the PR is behaving as expected.

[Qingfu-Liu]

@grant Firl - NOAA Affiliate ***@***.***> , I will start the regression tests now since I did not finish the test last time. I cancelled the run after 5 hours, and late I removed all the data. I hope the tests will be completed sometime tomorrow. Qingfu

…

On Fri, Apr 28, 2023 at 3:28 PM Grant Firl ***@***.***> wrote: @grant <https://github.com/grant> Firl - NOAA Affiliate *@*. *> , all the runs will be failed since we need to create a baseline data, but the runs do produce the output data. If I rerun the tests, it will take very long time (might be 10+ hours) Qingfu … <#m_930262221224032925_> On Fri, Apr 28, 2023 at 3:22 PM Grant Firl @.*> wrote: @Qingfu-Liu <https://github.com/Qingfu-Liu> https://github.com/Qingfu-Liu Also, we still need a list of which RTs are expected to fail. If you run all of the tests in rt.conf on Hera and copy/paste the list of failed tests found at the end of your tests/RegressionTest_hera.intel/gnu.log files, that is what we need. — Reply to this email directly, view it on GitHub <#1723 (comment) <#1723 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGTS6UXK43UKC2GWU6LU34DXDQKFNANCNFSM6AAAAAAXKFB7H4 . You are receiving this because you were mentioned.Message ID: *@*.***> If you've already run all the tests in rt.conf, then we just need the list of failed tests from the logs that were already produced. We *wan* the list of failed runs to know whether they're all expected. When I go to repeat this procedure for the NCAR:main branch, the list of failed tests is crucial to know if the PR is behaving as expected. — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTS6USLUD7UV747VQHUVGTXDQK33ANCNFSM6AAAAAAXKFB7H4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[grantfirl]

@grant Firl - NOAA Affiliate @.> , I will start the regression tests now since I did not finish the test last time. I cancelled the run after 5 hours, and late I removed all the data. I hope the tests will be completed sometime tomorrow. Qingfu
…
On Fri, Apr 28, 2023 at 3:28 PM Grant Firl @.> wrote: @grant https://github.com/grant Firl - NOAA Affiliate @. > , all the runs will be failed since we need to create a baseline data, but the runs do produce the output data. If I rerun the tests, it will take very long time (might be 10+ hours) Qingfu … <#m_930262221224032925_> On Fri, Apr 28, 2023 at 3:22 PM Grant Firl @.> wrote: @Qingfu-Liu https://github.com/Qingfu-Liu https://github.com/Qingfu-Liu Also, we still need a list of which RTs are expected to fail. If you run all of the tests in rt.conf on Hera and copy/paste the list of failed tests found at the end of your tests/RegressionTest_hera.intel/gnu.log files, that is what we need. — Reply to this email directly, view it on GitHub <#1723 (comment) <#1723 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGTS6UXK43UKC2GWU6LU34DXDQKFNANCNFSM6AAAAAAXKFB7H4 . You are receiving this because you were mentioned.Message ID: @.> If you've already run all the tests in rt.conf, then we just need the list of failed tests from the logs that were already produced. We wan the list of failed runs to know whether they're all expected. When I go to repeat this procedure for the NCAR:main branch, the list of failed tests is crucial to know if the PR is behaving as expected. — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGTS6USLUD7UV747VQHUVGTXDQK33ANCNFSM6AAAAAAXKFB7H4 . You are receiving this because you were mentioned.Message ID: @.>

Are you running with ecflow? It should be much quicker with ecflow.

[Qingfu-Liu]

@grant Firl - NOAA Affiliate ***@***.***> I am running the tests using ./rt.sh -l rt.conf I think I have a problem compiling the code now. Some network has problem to run the *py scripts Qingfu

…

On Fri, Apr 28, 2023 at 3:35 PM Grant Firl ***@***.***> wrote: @grant <https://github.com/grant> Firl - NOAA Affiliate *@*. *> , I will start the regression tests now since I did not finish the test last time. I cancelled the run after 5 hours, and late I removed all the data. I hope the tests will be completed sometime tomorrow. Qingfu … <#m_-5431920392816038775_> On Fri, Apr 28, 2023 at 3:28 PM Grant Firl @.*> wrote: @grant <https://github.com/grant> https://github.com/grant Firl - NOAA Affiliate *@*. *> , all the runs will be failed since we need to create a baseline data, but the runs do produce the output data. If I rerun the tests, it will take very long time (might be 10+ hours) Qingfu … <#m_930262221224032925_> On Fri, Apr 28, 2023 at 3:22 PM Grant Firl @.*> wrote: @Qingfu-Liu <https://github.com/Qingfu-Liu> https://github.com/Qingfu-Liu https://github.com/Qingfu-Liu Also, we still need a list of which RTs are expected to fail. If you run all of the tests in rt.conf on Hera and copy/paste the list of failed tests found at the end of your tests/RegressionTest_hera.intel/gnu.log files, that is what we need. — Reply to this email directly, view it on GitHub <#1723 <#1723> (comment) <#1723 (comment) <#1723 (comment)>>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGTS6UXK43UKC2GWU6LU34DXDQKFNANCNFSM6AAAAAAXKFB7H4 . You are receiving this because you were mentioned.Message ID: *@*.*> If you've already run all the tests in rt.conf, then we just need the list of failed tests from the logs that were already produced. We wan the list of failed runs to know whether they're all expected. When I go to repeat this procedure for the NCAR:main branch, the list of failed tests is crucial to know if the PR is behaving as expected. — Reply to this email directly, view it on GitHub <#1723 (comment) <#1723 (comment)>>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGTS6USLUD7UV747VQHUVGTXDQK33ANCNFSM6AAAAAAXKFB7H4 <https://github.com/notifications/unsubscribe-auth/AGTS6USLUD7UV747VQHUVGTXDQK33ANCNFSM6AAAAAAXKFB7H4> . You are receiving this because you were mentioned.Message ID: @.*> Are you running with ecflow? It should be much quicker with ecflow. — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTS6UUBLUHKGLJ36YV3BFTXDQLWTANCNFSM6AAAAAAXKFB7H4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Qingfu-Liu]

I got the error message like this (never see this before): --- Logging error --- Traceback (most recent call last): File "/apps/spack/linux-centos7-x86_64/gcc-9.2.0/python-3.7.6-gi3efxgcxqilpjehkqnxrriedsuedoqu/lib/python3.7/logging/handlers.py", line 940, in emit self.socket.sendto(msg, self.address) OSError: [Errno 101] Network is unreachable Call stack: File "<frozen importlib._bootstrap>", line 983, in _find_and_load File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 677, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 728, in exec_module File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed File "/apps/spack/linux-centos7-x86_64/gcc-9.2.0/python-3.7.6-gi3efxgcxqilpjehkqnxrriedsuedoqu/lib/python3.7/site.py", line 579, in <module> main() .... On Fri, Apr 28, 2023 at 3:41 PM Qingfu Liu - NOAA Federal < ***@***.***> wrote:

…

@grant Firl - NOAA Affiliate ***@***.***> I am running the tests using ./rt.sh -l rt.conf I think I have a problem compiling the code now. Some network has problem to run the *py scripts Qingfu On Fri, Apr 28, 2023 at 3:35 PM Grant Firl ***@***.***> wrote: > @grant <https://github.com/grant> Firl - NOAA Affiliate *@*. > > *> , I will start the regression tests now since I did not finish the > test last time. I cancelled the run after 5 hours, and late I removed all > the data. I hope the tests will be completed sometime tomorrow. Qingfu … > <#m_-2302659029809548283_m_-5431920392816038775_> On Fri, Apr 28, 2023 at > 3:28 PM Grant Firl @.*> wrote: @grant <https://github.com/grant> > https://github.com/grant Firl - NOAA Affiliate *@*. *> , all the runs > will be failed since we need to create a baseline data, but the runs do > produce the output data. If I rerun the tests, it will take very long time > (might be 10+ hours) Qingfu … <#m_930262221224032925_> On Fri, Apr 28, 2023 > at 3:22 PM Grant Firl @.*> wrote: @Qingfu-Liu > <https://github.com/Qingfu-Liu> https://github.com/Qingfu-Liu > https://github.com/Qingfu-Liu Also, we still need a list of which RTs > are expected to fail. If you run all of the tests in rt.conf on Hera and > copy/paste the list of failed tests found at the end of your > tests/RegressionTest_hera.intel/gnu.log files, that is what we need. — > Reply to this email directly, view it on GitHub <#1723 > <#1723> (comment) > <#1723 (comment) > <#1723 (comment)>>>, > or unsubscribe > https://github.com/notifications/unsubscribe-auth/AGTS6UXK43UKC2GWU6LU34DXDQKFNANCNFSM6AAAAAAXKFB7H4 > . You are receiving this because you were mentioned.Message ID: *@*.*> > If you've already run all the tests in rt.conf, then we just need the list > of failed tests from the logs that were already produced. We wan the list > of failed runs to know whether they're all expected. When I go to repeat > this procedure for the NCAR:main branch, the list of failed tests is > crucial to know if the PR is behaving as expected. — Reply to this email > directly, view it on GitHub <#1723 (comment) > <#1723 (comment)>>, > or unsubscribe > https://github.com/notifications/unsubscribe-auth/AGTS6USLUD7UV747VQHUVGTXDQK33ANCNFSM6AAAAAAXKFB7H4 > <https://github.com/notifications/unsubscribe-auth/AGTS6USLUD7UV747VQHUVGTXDQK33ANCNFSM6AAAAAAXKFB7H4> > . You are receiving this because you were mentioned.Message ID: @.*> > > Are you running with ecflow? It should be much quicker with ecflow. > > — > Reply to this email directly, view it on GitHub > <#1723 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AGTS6UUBLUHKGLJ36YV3BFTXDQLWTANCNFSM6AAAAAAXKFB7H4> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

[DeniseWorthen]

@Qingfu-Liu I know you mentioned earlier that the RTs were taking many many hours. If you were using ./rt.sh -l rt.conf, then you were not running with ecflow---meaning each job ran sequentially. That would explain why your tests were taking so long.

On Hera, you'll need to specifically log into the ecflow nodes using ssh hecflow01. Then do

./rt.sh -e >output 2>&1 &

Your tests should go much faster.

[jkbk2004]

@Qingfu-Liu Do you intend to close pr? fv3 dependency was closed as well. You force pushed, right? , For the this pr, you need to point to your ccpp branch, right? can you fix and re-open?

[Qingfu-Liu]

Hi Jong, I just synchronized the fork from the website as you requested. I did not push anything. Qingfu

…

On Fri, Apr 28, 2023 at 3:49 PM JONG KIM ***@***.***> wrote: @Qingfu-Liu <https://github.com/Qingfu-Liu> Do you intend to close pr? fv3 dependency was closed as well. You force pushed, right? , For the this pr, you need to point to your ccpp branch, right? can you fix and re-open? — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTS6UVM74H2ED6SOMDGVADXDQNNZANCNFSM6AAAAAAXKFB7H4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Qingfu-Liu]

@jong Kim - NOAA Affiliate ***@***.***> I use the sync button from the website to synchronize the branch update_HR2. On Fri, Apr 28, 2023 at 3:51 PM Qingfu Liu - NOAA Federal < ***@***.***> wrote:

…

Hi Jong, I just synchronized the fork from the website as you requested. I did not push anything. Qingfu On Fri, Apr 28, 2023 at 3:49 PM JONG KIM ***@***.***> wrote: > @Qingfu-Liu <https://github.com/Qingfu-Liu> Do you intend to close pr? > fv3 dependency was closed as well. You force pushed, right? , For the this > pr, you need to point to your ccpp branch, right? can you fix and re-open? > > — > Reply to this email directly, view it on GitHub > <#1723 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AGTS6UVM74H2ED6SOMDGVADXDQNNZANCNFSM6AAAAAAXKFB7H4> > . > You are receiving this because you were mentioned.Message ID: > ***@***.***> >

[grantfirl]

@Qingfu-Liu By "sync", Jong means to pull in the latest commits from the develop branches of ufs-weather-model and fv3atm to your PR branches. You still want your PR branch of fv3atm to point to your PR branch of ccpp-physics and your PR branch of ufs-weather-model to point to your PR branch of fv3atm after you've pulled in the latest develop commits.

[Qingfu-Liu]

@grant Firl - NOAA Affiliate ***@***.***> I think I did not quite understand the button "sync". If I click the button, it seems that all the files in branch update_HR2 are replaced. I

…

On Fri, Apr 28, 2023 at 3:59 PM Grant Firl ***@***.***> wrote: @Qingfu-Liu <https://github.com/Qingfu-Liu> By "sync", Jong means to pull in the latest commits from the develop branches of ufs-weather-model and fv3atm to your PR branches. You still want your PR branch of fv3atm to point to your PR branch of ccpp-physics and your PR branch of ufs-weather-model to point to your PR branch of fv3atm after you've pulled in the latest develop commits. — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTS6UXTTCRCC5RVLJ2ONPLXDQOQJANCNFSM6AAAAAAXKFB7H4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[jkbk2004]

@Qingfu-Liu we may let #1715 got first and come back for this pr next week. would that be ok?

[Qingfu-Liu]

@denise Worthen - NOAA Affiliate ***@***.***> Thank you very much. I have not logged into the ecflow nodes before. I will try it now Qingfu

…

On Fri, Apr 28, 2023 at 3:48 PM Denise Worthen ***@***.***> wrote: @Qingfu-Liu <https://github.com/Qingfu-Liu> I know you mentioned earlier that the RTs were taking many many hours. If you were using ./rt.sh -l rt.conf, then you were not running with ecflow---meaning each job ran sequentially. That would explain why your tests were taking so long. On Hera, you'll need to specifically log into the ecflow nodes using ssh hecflow01. Then do ./rt.sh -e >output 2>&1 & Your tests should go much faster. — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTS6UW547CBXGO75SOH7WLXDQNGZANCNFSM6AAAAAAXKFB7H4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Qingfu-Liu]

@Jong.Kim Yes. It is good. So I need to create 2 new PRs. It seems to me that the closed 2 PRs can't be reopened. Qingfu

…

On Fri, Apr 28, 2023 at 4:29 PM JONG KIM ***@***.***> wrote: @Qingfu-Liu <https://github.com/Qingfu-Liu> we may let #1715 <#1715> got first and come back for this pr next week. would that be ok? — Reply to this email directly, view it on GitHub <#1723 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGTS6UVX5MHHABI5AVH333DXDQSCJANCNFSM6AAAAAAXKFB7H4> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[grant]

@Qingfu-Liu: I think you mean to ask @grantfirl, not @grant.