Add progress bars for blob fs

[jenhagg]

🚀

• Is your feature request essential for your project?
  No, but it would be pretty nice.

Describe the workflow you want to enable

I want progress bars for upload/download.

Describe your proposed implementation

Write a custom download method instead of delegating to the base FS implementation. The download method allows for additional kwargs, one of which should be a callback, so that the exact progress bar implementation is left up to consumers.

Note: the BlobFile implementation first downloads the blob to a temp file before any read operations occur. We should consider providing some context to the user since the progress bar function below is delayed until the initial download is complete. Alternatively, the behavior could be optimized, but this might be difficult to do while preserving the file-like-object api.

Describe alternatives you've considered, if relevant

Not having progress bars.

Additional context

Refer to the similar customization done for the ssh fs

[jenhagg]

Snippet to add progress bar to download operation. Could be provided as documentation and copy/paste as needed, or integrated into the package somehow. TBD which approach is best..

def download(bfs, path, dst_file):
    size = bfs.getinfo(path, namespaces=["details"]).size
    with bfs.openbin(path) as src_file:
        read = src_file.read
        write = dst_file.write
        with tqdm(
            unit="B",
            unit_scale=True,
            unit_divisor=1024,
            miniters=1,
            total=size,
        ) as pbar:
            for chunk in iter(lambda: read(4096) or None, None):
                write(chunk)
                pbar.update(len(chunk))


with open("hydro.csv", "wb") as f:
    download(bfs, "raw/usa_tamu/hydro_vJan2021.csv", f)