Need to pin (temporarily) to an older version of xarray

[JimCircadian]

• IceNet version: 0.2.7
• Python version: 3.8.16

Description

icenet_data_era5 is failing with

[14-03-24 22:42:18 :ERROR   ] - Thread failure: __init__() got an unexpected keyword argument 'base'
Traceback (most recent call last):
  File "/data/hpcdata/users/jambyr/miniconda3/envs/icenet-new/lib/python3.8/site-packages/icenet/data/interfaces/downloader.py", line 226, in download
    future.result()
  File "/data/hpcdata/users/jambyr/miniconda3/envs/icenet-new/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/data/hpcdata/users/jambyr/miniconda3/envs/icenet-new/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/data/hpcdata/users/jambyr/miniconda3/envs/icenet-new/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/data/hpcdata/users/jambyr/miniconda3/envs/icenet-new/lib/python3.8/site-packages/icenet/data/interfaces/downloader.py", line 302, in _single_download
    self.postprocess(var, latlon_path)
  File "/data/hpcdata/users/jambyr/miniconda3/envs/icenet-new/lib/python3.8/site-packages/icenet/data/interfaces/cds.py", line 235, in postprocess
    da = da.sortby("time").resample(time='1D').mean()
  File "/data/hpcdata/users/jambyr/miniconda3/envs/icenet-new/lib/python3.8/site-packages/xarray/core/dataarray.py", line 6666, in resample
    return self._resample(
  File "/data/hpcdata/users/jambyr/miniconda3/envs/icenet-new/lib/python3.8/site-packages/xarray/core/common.py", line 993, in _resample
    grouper = pd.Grouper(
  File "/data/hpcdata/users/jambyr/miniconda3/envs/icenet-new/lib/python3.8/site-packages/pandas/core/resample.py", line 1663, in __init__
    super().__init__(freq=freq, axis=axis, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'base'

pydata/xarray#2385 is the cause, v2023.01.0 dropped support for python3.8 which is likely the underlying issue

What I Did

Running run_era5_forecast.sh from the pipeline

[bnubald]

Minimum working example of issue from xarray docs:

# Example from xarray docs
# https://docs.xarray.dev/en/v2024.02.0/generated/xarray.DataArray.resample.html
import numpy as np
import pandas as pd
import xarray as xr

da = xr.DataArray(
    np.linspace(0, 11, num=12),
    coords=[
        pd.date_range(
            "1999-12-15",
            periods=12,
            freq=pd.DateOffset(months=1),
        )
    ],
    dims="time",
)
da
da.resample(time="QS-DEC").mean()

Gives following error:

Traceback (most recent call last):
  File "resample_test.py", line 19, in <module>
    da.resample(time="QS-DEC").mean()
  File "/data/hpcdata/users/username/miniconda3/envs/icenet3.8/lib/python3.8/site-packages/xarray/core/dataarray.py", line 6666, in resample
    return self._resample(
  File "/data/hpcdata/users/username/miniconda3/envs/icenet3.8/lib/python3.8/site-packages/xarray/core/common.py", line 993, in _resample
    grouper = pd.Grouper(
  File "/data/hpcdata/users/username/miniconda3/envs/icenet3.8/lib/python3.8/site-packages/pandas/core/resample.py", line 1663, in __init__
    super().__init__(freq=freq, axis=axis, **kwargs)
TypeError: __init__() got an unexpected keyword argument 'base'

---

Pinning xarray version still causes same error.

Error caused by Pandas >=2.0.0 (Even though 3.8 was supported till 2.0.3)

Xarray support for 3.8 was dropped in version 2023.2.0, and before Pandas 2.0 release (Ref: pydata/xarray#7873 (comment)).

---

Solution: Should pin to Pandas to < 2.0.0 and Xarray to < 2023.2.0 for Python 3.8.

Updating requirements.txt (has been tested) with:

pandas<2; python_version == '3.8'
xarray[io]<2023.2.0; python_version == '3.8'
xarray[io]; python_version > '3.8'

[JimCircadian]

Hi @bnubald there's already a PR for this EDIT: just saw the content on the PR, apologies.