unify_time_units() prevents concatenation due to inconsistent dtype

[trexfeathers]

🐛 Bug Report

Thanks to @HelenRJohnson for spotting this.

Using iris.util.unify_time_units() on a CubeList where the time units have an integer dtype prevents the CubeList from being concatenated:

iris.exceptions.ConcatenateError: failed to concatenate into a single cube.
  Dimension coordinates metadata differ: time != time

Explanation

This is because unify_time_units() converts all time units to match the first one. cf-units will not convert something if it is already equal, so the first time unit does not undergo conversion. Meanwhile all the other time units are coerced to dtype="float64", to avoid problems with partial days etc. So the end result is that the first time unit remains an integer dtype while all the others are floats, and this is something that is checked when preparing for concatenation.

Proposed solution

Force all time coordinates to be dtype=float64 at the start of the unify_time_units() process, and advertise this behaviour in the docstring. We should include tests for the new lines, too. I tested this with the below and it fixed my example.

    for cube in cubes:
        for time_coord in cube.coords():
            if time_coord.units.is_time_reference():
+               time_coord.points = time_coord.core_points().astype("float64")
+               time_coord.bounds = time_coord.core_bounds().astype("float64")
                epoch = epochs.setdefault(
                    time_coord.units.calendar, time_coord.units.origin
                )

How To Reproduce

Steps to reproduce the behaviour:

1. Define a CubeList with multiple Cubes, which can be concatenated together.
   Each Cube should include a time coordinate, using dtype=int for the points.
   Make sure the final Cube's time coordinate has a different epoch to the others (prevents successful concatenation).
2. Apply iris.util.unify_time_units() to the CubeList.
3. Attempt CubeList.concatenate_cube() on this CubeList.

Expected behaviour

Should produce a single concatenated Cube.

Environment

• OS & Version: Red Hat 7.9
• Iris Version: v3.6.1

[rcomer]

This seems familiar… #3213

[trexfeathers]

Closed by #5412

[am-thyst]

Thanks for the fix. It's unfortunate that this issue has been reported for over 10 years.

[trexfeathers]

Thanks for the fix. It's unfortunate that this issue has been reported for over 10 years.

Hi @am-thyst, I'm afraid this is the unfortunate reality. The SciTools packages have around 10 regular maintainers (as usual balancing with other responsibilities too), and we prioritise as best we can; this leaves many bugs and good ideas that cannot be worked on with the available resource. I expect we have enough work for 20 maintainers working near full-time, but realistically the current situation is here to stay and we set our expectations accordingly.

I can understand that this might not meet yours and others' needs, and we will always strive to do better, but I want to be honest with you about the situation.