iris=3.8.1 ignores bounds at loading a hybrid height coordinate

[valeriupredoi]

Hey folks, one from me please:

Problem in a 🥥 shell

New iris=3.8.1 loads a field (cl, cloud_area_fraction_in_atmosphere_layer) but fails to correctly load its atmosphere_hybrid_height_coordinate ie its bounds. It spits a few warnings but then proceeds to call it a good day. Attached offending netCDF4 file and minimal test reproducible code, please rename the netCDF4 file with correct extension 😁
problem-cube.nc.txt

Working iris

iris=3.7.0

iris                      3.7.0              pyha770c72_0    conda-forge

Not working iris

iris=3.8.1

iris                      3.8.1              pyha770c72_0    conda-forge

Minimal reproducible code

import iris
from iris import NameConstraint


short_name = "cl"
cubes = iris.load("problem-cube.nc")
cube = cubes.extract_cube(NameConstraint(var_name=short_name))
print(cube)
height_coord = cube.coord('altitude')
print(height_coord.bounds)
assert height_coord.bounds is not None

Expected correct output

cube cloud_area_fraction_in_atmosphere_layer / (%) (time: 1; atmosphere_hybrid_height_coordinate: 2; latitude: 1; longitude: 2)
    Dimension coordinates:
        time                                       x                                       -            -             -
        atmosphere_hybrid_height_coordinate        -                                       x            -             -
        latitude                                   -                                       -            x             -
        longitude                                  -                                       -            -             x
    Auxiliary coordinates:
        b                                          -                                       x            -             -
        surface_altitude                           -                                       -            x             x
    Derived coordinates:
        altitude                                   -                                       x            x             x
[[[[0.5 1.5]
   [0.5 1.5]]]


 [[[1.5 3. ]
   [2.5 4. ]]]]

Resulting problem output

/home/valeriu/miniconda3/envs/esmvaltool/lib/python3.11/site-packages/iris/fileformats/cf.py:1271: IrisCfNonSpanningVarWarning: Ignoring formula terms variable 'orog' referenced by data variable 'b_bnds' via variable 'lev': Dimensions ('lat', 'lon') do not span ('lev', 'bnds')
  warnings.warn(
/home/valeriu/miniconda3/envs/esmvaltool/lib/python3.11/site-packages/iris/fileformats/_nc_load_rules/helpers.py:1126: IrisCfLoadWarning: Ignoring bounds on NetCDF variable 'lev_bnds'. Expected units compatible with 'm', got '1'.
  bounds_data = _normalise_bounds_units(attr_units, cf_bounds_var, bounds_data)
/home/valeriu/miniconda3/envs/esmvaltool/lib/python3.11/site-packages/iris/fileformats/netcdf/loader.py:416: IrisFactoryCoordNotFoundWarning: Unable to find coordinate for variable 'orog'
  warnings.warn(
cloud_area_fraction_in_atmosphere_layer / (%) (time: 1; atmosphere_hybrid_height_coordinate: 2; latitude: 1; longitude: 2)
    Dimension coordinates:
        time                                       x                                       -            -             -
        atmosphere_hybrid_height_coordinate        -                                       x            -             -
        latitude                                   -                                       -            x             -
        longitude                                  -                                       -            -             x
    Auxiliary coordinates:
        b                                          -                                       x            -             -
        surface_altitude                           -                                       -            x             x
    Derived coordinates:
        altitude                                   -                                       x            x             x
None
Traceback (most recent call last):
  File "/home/valeriu/ESMValCore/test-problem-cube.py", line 11, in <module>
    assert height_coord.bounds is not None
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

[trexfeathers]

Problem likely a result of #5746. It would be good to confirm this, I'm going to investigate this with @HGWright

[trexfeathers]

If this DOES turn out to be a malformed file, there is debate about what Iris should/should-not load - see #5165. Could be a job for ncdata? 🦸‍♂️

[valeriupredoi]

cheers @trexfeathers - I take it this is not solved in any way in 3.9.0 as well? Not pressuring you folk in any way - pure engineering reasons for me, we marked a test as xfailed, and was wondering if we should un-xfail it? 🍻

[HGWright]

@valeriupredoi What a weird coincidence we were in the middle of writing this reply when you commented that.

We have confirmed that this is an intended change in #5746 as suggested above. Obviously the problem in this case is that lev_bnds has the old default unit rather than a different specified unit. So this is probably a safe case to fix before loading. Here is a demonstration of doing this with ncdata:

import iris
from iris import NameConstraint

from ncdata.netcdf4 import from_nc4
from ncdata.iris import to_iris
ncdata = from_nc4("problem-cube.nc")
ncdata.variables["lev_bnds"].attributes["units"] = ncdata.variables["lev"].attributes["units"]
cubes = to_iris(ncdata)


short_name = "cl"
cube = cubes.extract_cube(NameConstraint(var_name=short_name))
print(cube)
height_coord = cube.coord('altitude')
print(height_coord.units)
assert height_coord.bounds is not None

[valeriupredoi]

@HGWright brill, many thanks! We'll do the fix dance then 😁 🍺