Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dask: Data collapse methods and functions #356

Merged
merged 43 commits into from
Apr 20, 2022
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
20562ad
dev
davidhassell Mar 11, 2022
4eca601
dev
davidhassell Mar 11, 2022
5609805
dev
davidhassell Mar 12, 2022
b65b63e
dev
davidhassell Mar 13, 2022
8678fa3
dev
davidhassell Mar 14, 2022
d40544b
dev
davidhassell Mar 14, 2022
f6ffd6d
dev
davidhassell Mar 14, 2022
1e4389e
dev
davidhassell Mar 15, 2022
9ff92ca
dev
davidhassell Mar 16, 2022
ec3adae
upstream merge
davidhassell Mar 16, 2022
30540aa
dev
davidhassell Mar 16, 2022
ade6277
docs tidy
davidhassell Mar 17, 2022
d5f884d
docs tidy
davidhassell Mar 17, 2022
b132139
getting there
davidhassell Mar 17, 2022
41306d0
collapse docs
davidhassell Mar 17, 2022
9a5624c
dev
davidhassell Mar 17, 2022
c2cf274
Fix bug in cf_percentile mtol inequality
davidhassell Mar 18, 2022
f59feb7
dev
davidhassell Mar 18, 2022
7320e04
dask v2022.03.0
davidhassell Mar 18, 2022
a096103
Merge branch 'lama-to-dask' into dask-collapse
davidhassell Mar 18, 2022
72b29df
collapse dtype
davidhassell Mar 18, 2022
3df9388
Merge branch 'lama-to-dask' into dask-collapse
davidhassell Mar 22, 2022
fe128c4
Merge branch 'lama-to-dask' into dask-collapse
davidhassell Apr 5, 2022
728a47a
Typo
davidhassell Apr 12, 2022
dc7bf53
Typo
davidhassell Apr 12, 2022
b47fc2d
Typo
davidhassell Apr 12, 2022
816925a
Typos
davidhassell Apr 12, 2022
5ecdf08
Merge branch 'dask-collapse' of ssh://github.com/davidhassell/cf-pyth…
davidhassell Apr 12, 2022
3d14610
Merge branch 'lama-to-dask' into dask-collapse
davidhassell Apr 12, 2022
61567b2
keep source in harden/soften mask
davidhassell Apr 12, 2022
6f6ad33
Typo
davidhassell Apr 13, 2022
788fbae
Typo
davidhassell Apr 13, 2022
c87d1d3
Corrected docs
davidhassell Apr 13, 2022
3c65cbd
Typo
davidhassell Apr 13, 2022
f7fc780
Clarity
davidhassell Apr 13, 2022
ff48ef8
Correct return statement in alias
davidhassell Apr 13, 2022
fe2f882
Typo
davidhassell Apr 13, 2022
9520190
Typo
davidhassell Apr 13, 2022
69ed1e8
Typo
davidhassell Apr 13, 2022
441f74b
collapse datea type comments
davidhassell Apr 13, 2022
0d87f44
deprecate _HDF_chunks
davidhassell Apr 13, 2022
82d3508
Data.sqrt: incompatible units
davidhassell Apr 19, 2022
60fcb17
cell methods clarification
davidhassell Apr 20, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions cf/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@
__date__ = "2022-01-18"
__version__ = "4.0.0b0"

_requires = ("numpy", "netCDF4", "cftime", "cfunits", "cfdm", "psutil")
_requires = ("numpy", "netCDF4", "cftime", "cfunits", "cfdm", "psutil", "dask")

x = ", ".join(_requires)
_error0 = f"cf v{ __version__} requires the modules {x}. "
Expand All @@ -104,10 +104,6 @@

_found_ESMF = bool(importlib.util.find_spec("ESMF"))

# TODODASK - Remove the next 2 lines when the move to dask is complete
mpi_on = False
mpi_size = 1

try:
import netCDF4
except ImportError as error1:
Expand All @@ -133,6 +129,11 @@
except ImportError as error1:
raise ImportError(_error0 + str(error1))

try:
import dask
except ImportError as error1:
raise ImportError(_error0 + str(error1))

# Check the version of psutil
_minimum_vn = "0.6.0"
if LooseVersion(psutil.__version__) < LooseVersion(_minimum_vn):
Expand Down Expand Up @@ -183,6 +184,14 @@
f"Got {_cfdm_version} at {cfdm.__file__}"
)

# Check the version of dask
_minimum_vn = "2022.03.0"
if LooseVersion(dask.__version__) < LooseVersion(_minimum_vn):
raise RuntimeError(
f"Bad dask version: cf requires dask>={_minimum_vn}. "
f"Got {dask.__version__} at {dask.__file__}"
)

from .constructs import Constructs

from .mixin import Coordinate
Expand Down
Loading