KIOST-ESM-tasmax.yaml fails clean step again, in yet more colorful ways

[brews]

KIOST-ESM tasmax fails cmip6-cleaning with a few new errors.

Workflow in archive is here: https://argo.cildc6.org/archived-workflows/default/1d14e081-92a4-44b2-8710-818e4a6fdf5d.

First, ssp585 clears cleaning and passes validation.

In contrast, ssp126 fails cleaning validation. Here is the logged error:

Validating gs://clean-b1dbca25/cmip6/ScenarioMIP/KIOST/KIOST-ESM/ssp126/r1i1p1f1/day/tasmax/gr1/v20191202.zarr
Traceback (most recent call last):
  File "/argo/staging/script", line 18, in <module>
    _test_timesteps(ds, data_type, time_period)
  File "/opt/dodola/dodola/core.py", line 646, in _test_timesteps
    assert (
AssertionError: projection cmip6 file is missing timesteps, has 4380

Cleaning for ssp245 and historical (which has ssp245 appended) both fail when they try to slice ssp245 in time. Both have logged errors:

Read gs://raw-305d04da/cmip6/ScenarioMIP/KIOST/KIOST-ESM/ssp245/r1i1p1f1/day/tasmax/gr1/v20191202.zarr
slicing 2015 : 2110
Traceback (most recent call last):
  File "/opt/conda/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3081, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 101, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 4554, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 4562, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: cftime.DatetimeNoLeap(2110, 12, 31, 23, 59, 59, 999999, has_year_zero=True)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/argo/staging/script", line 17, in <module>
    ds = ds.sel(time=slice(from_time, to_time))
  File "/opt/conda/lib/python3.9/site-packages/xarray/core/dataset.py", line 2474, in sel
    pos_indexers, new_indexes = remap_label_indexers(
  File "/opt/conda/lib/python3.9/site-packages/xarray/core/coordinates.py", line 421, in remap_label_indexers
    pos_indexers, new_indexes = indexing.remap_label_indexers(
  File "/opt/conda/lib/python3.9/site-packages/xarray/core/indexing.py", line 117, in remap_label_indexers
    idxr, new_idx = index.query(labels, method=method, tolerance=tolerance)
  File "/opt/conda/lib/python3.9/site-packages/xarray/core/indexes.py", line 198, in query
    indexer = _query_slice(index, label, coord_name, method, tolerance)
  File "/opt/conda/lib/python3.9/site-packages/xarray/core/indexes.py", line 91, in _query_slice
    indexer = index.slice_indexer(
  File "/opt/conda/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 5278, in slice_indexer
    start_slice, end_slice = self.slice_locs(start, end, step=step, kind=kind)
  File "/opt/conda/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 5483, in slice_locs
    end_slice = self.get_slice_bound(end, "right", kind)
  File "/opt/conda/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 5397, in get_slice_bound
    raise err
  File "/opt/conda/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 5391, in get_slice_bound
    slc = self.get_loc(label)
  File "/opt/conda/lib/python3.9/site-packages/xarray/coding/cftimeindex.py", line 466, in get_loc
    return pd.Index.get_loc(self, key, method=method, tolerance=tolerance)
  File "/opt/conda/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3083, in get_loc
    raise KeyError(key) from err
KeyError: cftime.DatetimeNoLeap(2110, 12, 31, 23, 59, 59, 999999, has_year_zero=True)

Only ssp245 has this issue, it would appear.

This blocks progress on run #315, #395.

[brews]

@dgergel This is a 'second deliverable' product, but the errors are updated from #347.

[dgergel]

@brews this looks to be a bit more involved with the ssp245 errors, and given that 370 isn't available for this model, I'm going to hold off on dealing with this since it's a second round deliverable. More pressing errors to address for now.

[emileten]

Not surprising, this is still failing for the exact same reasons : https://argo.cildc6.org/archived-workflows/default/c79801eb-9a1d-49d1-9971-4fc65024ca88.

[dgergel]

eliminated this model due to QC issues