You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #18 , we integrated fsspec to add support for reading metadata over http. And GDAL already supports reading over http.
Since much of our data is on GCS and GDrive, it might make sense to add more explicit support for those backends too. http support allows reading data with public URLs from those locations, but if we add support for gcs:// and gdrivefs then we start supporting reading from private stores, and even writing metadata directly to those stores.
Here's some things to consider first, from @phargogh
I think we should explore this option a little more before making a decision on being able to read from private filesystems and write to these filesystems. Open questions on my mind include:
Can we read all fsspec-required auth parameters from GDAL's auth configuration?
How will we read a GDAL-supported dataset from Google Drive (which is not supported by GDAL's VSIs) without downloading the whole dataset?
Are there any other cases where GDAL will handle a URI with the same protocol prefix (e.g. gs://) as fsspec, or will we have to pass GDAL VSI-style path strings?
The text was updated successfully, but these errors were encountered:
davemfish
changed the title
more complete support for remote filesystems
more complete support for remote filesystems like GCS and GDrive
Mar 8, 2024
In #18 , we integrated
fsspec
to add support for reading metadata overhttp
. And GDAL already supports reading overhttp
.Since much of our data is on GCS and GDrive, it might make sense to add more explicit support for those backends too.
http
support allows reading data with public URLs from those locations, but if we add support forgcs://
andgdrivefs
then we start supporting reading from private stores, and even writing metadata directly to those stores.Here's some things to consider first, from @phargogh
The text was updated successfully, but these errors were encountered: