Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix 360 day calendar conversion chunk errors from dodola.services.clean_cmip6 #151

Merged
merged 5 commits into from
Dec 9, 2021
Merged

Fix 360 day calendar conversion chunk errors from dodola.services.clean_cmip6 #151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4254756
Fix 360 day calendar chunks
brews Dec 9, 2021
b915c00
Update HISTORY with PR reference
brews Dec 9, 2021
e3c90c8
lint
brews Dec 9, 2021
f2dac12
Merge branch 'main' into fix_360calendar_chunk
brews Dec 9, 2021
57da5aa
Correct history after upstream release
brews Dec 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion HISTORY.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ History

0.XX.X (XXXX-XX-XX)
-------------------
*
* Fix rechunk error when converting 360 days calendars. (#149, PR #151, @brews)


0.12.0 (2021-12-09)
Expand Down
3 changes: 3 additions & 0 deletions dodola/core.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -472,6 +472,9 @@ def xesmf_regrid(
def standardize_gcm(ds, leapday_removal=True):
"""

360 calendar conversion requires that there are no chunks in
the 'time' dimension of `ds`.

Parameters
----------
ds : xr.Dataset
Expand Down
8 changes: 8 additions & 0 deletions dodola/services.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -708,6 +708,9 @@ def regrid(
def clean_cmip6(x, out, leapday_removal):
"""Cleans and standardizes CMIP6 GCM

This loads the entire `x` Dataset into memory for speed
and to avoid chunking errors.

Parameters
----------
x : str
Expand All @@ -718,6 +721,11 @@ def clean_cmip6(x, out, leapday_removal):
Whether or not to remove leap days.
"""
ds = storage.read(x)

# Cannot have chunks in time dimension for 360 day calendar conversion so loading
# data into memory.
ds.load()

cleaned_ds = standardize_gcm(ds, leapday_removal)
storage.write(out, cleaned_ds)

Expand Down