Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Zarr-Python 3 #1082

Merged
merged 6 commits into from
Jan 29, 2025
Merged

Support Zarr-Python 3 #1082

merged 6 commits into from
Jan 29, 2025

Conversation

maxrjones
Copy link
Member

Adds support for both opening Zarr stores using either Zarr-Python 2 or Zarr-Python 3.

Fixed #1068

ds = xarray.open_zarr(store, **xr_open_args)
else:
fs = fsspec.filesystem(protocol)
ds = xarray.open_zarr(fs.get_mapper(src_path), **xr_open_args)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the bulk of the changes to account for Zarr no longer accepting fsspec's mutable mapping wrappers and expecting asynchronous file systems.

"h5netcdf",
"fsspec",
"s3fs",
"aiohttp",
"gcsfs",
]
minimal = [
"zarr>=2,<3",
"zarr",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will this work for zarr<3 ? Should that be specified?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I accounted for both the new and old syntax in

if module_available("zarr", minversion="3.0"):
if protocol == "file":
store = zarr.storage.LocalStore(parsed.path, read_only=True)
else:
fs = fsspec.filesystem(protocol, storage_options={"asynchronous": True})
store = zarr.storage.FsspecStore(fs, path=src_path, read_only=True)
else:
store = fsspec.filesystem(protocol).get_mapper(src_path)
and tested locally for zarr<3.

@vincentsarago vincentsarago merged commit ea71607 into main Jan 29, 2025
10 checks passed
@vincentsarago vincentsarago deleted the support-zarr-v3 branch January 29, 2025 11:17
@maxrjones maxrjones self-assigned this Feb 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[titiler.xarray] update for Zarr 3.0
3 participants