Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch up 1 deg configs with dev-025deg_jra55do_ryf #181

Merged
merged 9 commits into from
Feb 20, 2025

Conversation

dougiesquire
Copy link
Collaborator

@dougiesquire dougiesquire commented Feb 19, 2025

This PR implements a few changes that were made to the dev-025deg_jra55do_ryf config but were not included in the 1 deg configs:

Update MOM_parameter_docs
@dougiesquire
Copy link
Collaborator Author

!test repro

Update MOM_parameter_docs
@dougiesquire dougiesquire changed the title dev-1deg_jra55do_ryf: disable MOM tidal momentum forcing dev-1deg_jra55do_ryf: catch up with 0.25 configuration parameters Feb 19, 2025
@dougiesquire dougiesquire changed the title dev-1deg_jra55do_ryf: catch up with 0.25 configuration parameters Catch up 1 deg configs with dev-025deg_jra55do_ryf Feb 19, 2025
Copy link

❌ The Bitwise Reproducibility Check Failed ❌

When comparing:

  • 276-1deg_jra55do_ryf (checksums created using commit 0707fa2), against
  • dev-1deg_jra55do_ryf (checksums in commit a706052)
Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-om3-configs/0707fa2af73bef36cc1c19b7db75a9e36e6f081d, and the test results at https://github.com/ACCESS-NRI/access-om3-configs/runs/37443386464.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-om3-configs/actions/runs/13404832476/artifacts/2613753269.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-om3-configs/tree/a706052774b76dffd0071b07c0029bc53889e994/testing/checksum

@dougiesquire
Copy link
Collaborator Author

!test repro

Copy link

❌ The Bitwise Reproducibility Check Failed ❌

When comparing:

  • 276-1deg_jra55do_ryf (checksums created using commit 76dd7cb), against
  • dev-1deg_jra55do_ryf (checksums in commit a706052)
Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-om3-configs/76dd7cb7698990314eb4b5225feaa71a90e9272e, and the test results at https://github.com/ACCESS-NRI/access-om3-configs/runs/37444638152.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-om3-configs/actions/runs/13405333294/artifacts/2613886425.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-om3-configs/tree/a706052774b76dffd0071b07c0029bc53889e994/testing/checksum

@dougiesquire
Copy link
Collaborator Author

Note the reproducibility tests reported in the commit above failed because MOM aborted due to velocity truncations

@dougiesquire
Copy link
Collaborator Author

!test repro

Copy link

❌ The Bitwise Reproducibility Check Failed ❌

When comparing:

  • 276-1deg_jra55do_ryf (checksums created using commit ae52db9), against
  • dev-1deg_jra55do_ryf (checksums in commit a706052)
Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-om3-configs/ae52db93c337aa94708ca6758ed3e3541d916508, and the test results at https://github.com/ACCESS-NRI/access-om3-configs/runs/37445654486.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-om3-configs/actions/runs/13405639079/artifacts/2613993546.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-om3-configs/tree/a706052774b76dffd0071b07c0029bc53889e994/testing/checksum

@dougiesquire
Copy link
Collaborator Author

!test repro

Copy link

❌ The Bitwise Reproducibility Check Failed ❌

When comparing:

  • 276-1deg_jra55do_ryf (checksums created using commit 178ad47), against
  • dev-1deg_jra55do_ryf (checksums in commit a706052)
Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-om3-configs/178ad471b474723ba6e98162b8255f3b86b00695, and the test results at https://github.com/ACCESS-NRI/access-om3-configs/runs/37461381561.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-om3-configs/actions/runs/13410170947/artifacts/2615560893.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-om3-configs/tree/a706052774b76dffd0071b07c0029bc53889e994/testing/checksum

@dougiesquire
Copy link
Collaborator Author

!test repro

Copy link

❌ The Bitwise Reproducibility Check Failed ❌

When comparing:

  • 276-1deg_jra55do_ryf (checksums created using commit d07b907), against
  • dev-1deg_jra55do_ryf (checksums in commit a706052)
Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-om3-configs/d07b9072d65916045397e26d6a697e1e5d6426eb, and the test results at https://github.com/ACCESS-NRI/access-om3-configs/runs/37506769581.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-om3-configs/actions/runs/13424941184/artifacts/2620310239.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-om3-configs/tree/a706052774b76dffd0071b07c0029bc53889e994/testing/checksum

@dougiesquire
Copy link
Collaborator Author

!test repro

Copy link

❌ The Bitwise Reproducibility Check Failed ❌

When comparing:

  • 276-1deg_jra55do_ryf (checksums created using commit 4977df8), against
  • dev-1deg_jra55do_ryf (checksums in commit a706052)
Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-om3-configs/4977df8d9dfe11a3b8abe628cdbe7a7561949dee, and the test results at https://github.com/ACCESS-NRI/access-om3-configs/runs/37507726313.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-om3-configs/actions/runs/13425291173/artifacts/2620408384.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-om3-configs/tree/a706052774b76dffd0071b07c0029bc53889e994/testing/checksum

@dougiesquire
Copy link
Collaborator Author

I'm a bit surprised that setting MINIMUM_DEPTH = 0.0 changed answers (very slightly) since the minimum depth of TOPO_FILE is 11.8 m...

Screenshot 2025-02-20 at 12 09 32 pm

@dougiesquire
Copy link
Collaborator Author

dougiesquire commented Feb 20, 2025

@minghangli-uni, would you be able to take a look at this? Once you're happy with it, I'll update the checksums

Copy link

@minghangli-uni minghangli-uni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dougiesquire. I was able to run it successfully for a month. Just one question: while DIABATIC_FIRST is agreed to be False, another issue is for DT and DT_THERM. I haven’t investigated this for the 1deg config yet, but I believe that the coupling timestep and cice therm timestep should at least match the MOM DT. This ensures that both components exchange fluxes at the same time, regardless of the final choice for dt_therm. This is what we do for the 0.25deg at the moment. So DT=DT_CPL=3600s? 1800s would be a bit low for the 1deg config.

@dougiesquire
Copy link
Collaborator Author

Thanks @minghangli-uni. I'm not sure I understand all of your point sorry. The coupling timestep in this configuration is 3600s and both MOM and CICE exchange fluxes at that timestep, right? With DT = 1800, MOM just does two dynamic timesteps for every coupling (and thermodynamic) timestep.

I agree with you that the MOM dynamic timestep should probably be increased. But I think this probably requires some basic testing and should probably be done in a subsequent PR. @anton-seaice already opened an issue about this last week - see COSIMA/access-om3#268

@minghangli-uni
Copy link

My thinking is if dt is half of dt_cpl, would this delay feedback between ocean and cice? E.g., fluxes remain constant for two ocean baroclinic steps before being updated.

@dougiesquire
Copy link
Collaborator Author

I tried running with DT = 3600.0. I got a lot of warnings like this:

WARNING from PE     3: btstep: eta has dropped below bathyT:  -1.4331200363952876E+01 vs.  -1.1805749893188477E+01 at  -1.4450E+02 -7.5568E+01    136     10

and then MOM aborted due to velocity truncations.

@dougiesquire
Copy link
Collaborator Author

@minghangli-uni can we address this in a subsequent PR?

@minghangli-uni minghangli-uni self-requested a review February 20, 2025 04:48
Copy link

@minghangli-uni minghangli-uni left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure lets finish this first.

@dougiesquire
Copy link
Collaborator Author

!test repro commit

Copy link

❌ The Bitwise Reproducibility Check Failed ❌

When comparing:

  • 276-1deg_jra55do_ryf (checksums created using commit a35facd), against
  • dev-1deg_jra55do_ryf (checksums in commit a706052)

🔧 The checksums will be committed to this PR, as they differ.

Further information

The experiment can be found on Gadi at /scratch/tm70/repro-ci/experiments/access-om3-configs/a35facd9988392efc943bce2f3541402fd3557ab, and the test results at https://github.com/ACCESS-NRI/access-om3-configs/runs/37515478116.

The checksums generated by this !test command are found in the testing/checksum directory of https://github.com/ACCESS-NRI/access-om3-configs/actions/runs/13428161262/artifacts/2621222784.

The checksums compared against are found here https://github.com/ACCESS-NRI/access-om3-configs/tree/a706052774b76dffd0071b07c0029bc53889e994/testing/checksum

@dougiesquire dougiesquire merged commit d7873ba into dev-1deg_jra55do_ryf Feb 20, 2025
10 checks passed
@dougiesquire dougiesquire deleted the 276-1deg_jra55do_ryf branch February 20, 2025 05:23
@dougiesquire
Copy link
Collaborator Author

!cherry-pick 0707fa2 76dd7cb ee678b9 63f98c2 11e544d 6d239ca c0aea88 into dev-1deg_jra55do_iaf

Copy link

!cherry-pick 0707fa2 76dd7cb ee678b9 63f98c2 11e544d 6d239ca c0aea88 into dev-1deg_jra55do_iaf

Automatic cherry-pick failed. Could not find commit(s) 0707fa2af73bef36cc1c19b7db75a9e36e6f081d 76dd7cb7698990314eb4b5225feaa71a90e9272e ee678b95e197a2951eb9c272999c48474c4af3b5 63f98c2628e32761c14449df203b96980ef9c146 11e544d5a4ea50eeeb7f298c15dbb441bb638b80 6d239ca13f3fb3e2449529c4ada332a632c35b60 c0aea88afa6e3da5a7a297c30dc3e20522ce3c65 in dev-1deg_jra55do_ryf.

@dougiesquire
Copy link
Collaborator Author

Duh!

@dougiesquire
Copy link
Collaborator Author

!cherry-pick b7947cf d39270f 2d0a6cd 903774e 8a2fcab f711d80 d0278aa into dev-1deg_jra55do_iaf

Copy link

Automatic Git cherry-picking of commit(s) b7947cf d39270f 2d0a6cd 903774e 8a2fcab f711d80 d0278aa into dev-1deg_jra55do_iaf failed. This usually happens when cherry-picking results in a conflic or an empty commit. To manually cherry-pick the commits and open a pull request, please follow these instructions:

  1. Create new branch from target branch:
git checkout dev-1deg_jra55do_iaf
git pull
git checkout -b cherry_pick_from_pr181_into_dev-1deg_jra55do_iaf
  1. Cherry-pick commits:
git cherry-pick b7947cf1123d784c942547300d6fc6c8a4674631 d39270f261178d993dacbc125cf39dfaf0822cf8 2d0a6cdcdf06d885c623c384d109bbce713f664c 903774e71af678272ed8173bae5d8eb1b41a1b60 8a2fcab4c0d5e6a9a8dfabb5dd698af2061d1c7c f711d80fd8109aa85d83d01e228a526e700e8b60 d0278aa85adfe6f338e677e60cf20b06526c1a70
  1. Fix any conflicts and/or empty commits by following the instructions provided by Git.
  2. Push the new branch:
git push --set-upstream origin cherry_pick_from_pr181_into_dev-1deg_jra55do_iaf
  1. Open a new pull request on github making sure the target branch is set to dev-1deg_jra55do_iaf.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants