Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Model dies with segmentation fault if depth greater than 9250 m #491

Closed
pittwolfe opened this issue Oct 3, 2023 · 5 comments
Closed

Model dies with segmentation fault if depth greater than 9250 m #491

pittwolfe opened this issue Oct 3, 2023 · 5 comments
Labels
bug Something isn't working

Comments

@pittwolfe
Copy link

The model dies with a segmentation fault if the ocean depth is anywhere greater than 9250 m unless non-obvious, non-default settings are used.

Reproduction

Run the "double_gyre" example out of the box, except set TOPO_CONFIG = "flat" and MAXIMUM_DEPTH to anything greater than 9250.

Error message

Depends on fortran runtime and optimization level. Optimized Intel is unintelligible. gfortran helpfully produces

Error termination. Backtrace:
At line 465 of file ../../../../src/MOM6/src/ALE/MOM_regridding.F90
Fortran runtime error: Index '41' of dimension 1 of array 'woa09_dz' above upper bound of 40

Expected behavior

No segmentation faults. If the parameters are inconsistent, model should post an error message and exit gracefully.

Origin

initialize_regridding, called by the diagnostics mediator, will default (for convenience) to WOA09 vertical levels if MAXIMUM_DEPTH > 3000. The offending lines then look for an entry in the list of WOA09 vertical levels that exceeds the maximum depth of the domain. After 9250 m, the list ends but initialize_regridding keeps looking until it runs off the end of the list.

Workaround

Set DIAG_COORD_DEF_Z explicitly. If you don't need diagnostics on anything other than the native grid, DIAG_COORD_DEF_Z = "UNIFORM" works fine.

@marshallward
Copy link
Member

I get a slightly different error (a MOM error, not a SIGSEGV), but otherwise the same result:

FATAL: MOM_diag_remap, initialize_regridding: For "WOA05:N" N must 0<N<41 but got "WOA09".

As you mentioned, the woa09_dz thicknesses end at 9250, so the lookup table for dz construction fails.

The segmentation fault has been "fixed", but this new error does not do a great job of directing the user to fix the problem, if all they want to do is run double gyre at a very large depth.

@herrwang0
Copy link

herrwang0 commented Oct 4, 2023

Just to add a data point, I encountered the same error before with 1-layer barotropic runs. And I have also been using DIAG_COORD_DEF_Z = "UNIFORM" to fix the issue.

It is a bit counter-intuitive that subroutine initialize_regridding is called even with USE_REGRIDDING = False

@pittwolfe
Copy link
Author

Agreed. It took me a while to track down the problem because I had USE_REGRIDDING = False and couldn't figure out why and when initialize_regridding was being called.

@marshallward
Copy link
Member

According to @adcroft it's the diagnostics which need initialize_regridding, not the model fields. I agree that none of this feels very intuitive.

@Hallberg-NOAA Hallberg-NOAA added the bug Something isn't working label Dec 14, 2023
Hallberg-NOAA added a commit to Hallberg-NOAA/MOM6 that referenced this issue May 16, 2024
  It was noted in NOAA-GFDL/issues/491 that the call to
initialize_regridding() for diagnostics gives a segmentation fault when the
maximum depth of the ocean is greater than 9250 m unless DIAG_COORD_DEF_Z is
explicitly set.  This commit fixes this problem by (1) adding an extra layer to
the bottom of the hard-coded WOA09 list of diagnostic depths when the ocean is
deeper than the 9250 m maximum depth in that file, and (2) changing the default
behavior for diagnostic grids to be be uniform when the maximum ocean depth is
deeper than 9250 m, for which WOA09 is no longer likely to be a convenient
choice.  With this change, MOM6 no longer has segmentation faults for the
configuration described in issues/491.  All solutions are bitwise identical, but
some z-space diagnostic grids will be modified in cases that previously had
segmentation faults.
marshallward pushed a commit that referenced this issue Jun 18, 2024
  It was noted in /issues/491 that the call to
initialize_regridding() for diagnostics gives a segmentation fault when the
maximum depth of the ocean is greater than 9250 m unless DIAG_COORD_DEF_Z is
explicitly set.  This commit fixes this problem by (1) adding an extra layer to
the bottom of the hard-coded WOA09 list of diagnostic depths when the ocean is
deeper than the 9250 m maximum depth in that file, and (2) changing the default
behavior for diagnostic grids to be be uniform when the maximum ocean depth is
deeper than 9250 m, for which WOA09 is no longer likely to be a convenient
choice.  With this change, MOM6 no longer has segmentation faults for the
configuration described in issues/491.  All solutions are bitwise identical, but
some z-space diagnostic grids will be modified in cases that previously had
segmentation faults.
@Hallberg-NOAA
Copy link
Member

This issue has been addressed and corrected by PR #636.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants