You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now, all of the blob storage operations require passing Bytes, which translates to having everything in memory to make a request. This makes uploading large files exceedingly difficult, even uploading the file using multiple PutBlock calls.
Right now, Blob storage allows blobs to have up to 50,000 blocks, with each block up to 4000 MiB, for a total of approximately 190.7TiB. 1
Right now, in order to upload such a file, each 4000MiB block must be entirely read into memory before uploading. Both reqwest2 and hyper3 support a mechanism to wrap a Stream, which would enable streaming the input, rather than reading it entirely in memory first.
Work added in #230 started the work to support this by adding azure_core::SeekableStream.
Right now, all of the blob storage operations require passing Bytes, which translates to having everything in memory to make a request. This makes uploading large files exceedingly difficult, even uploading the file using multiple PutBlock calls.
Right now, Blob storage allows blobs to have up to 50,000 blocks, with each block up to 4000 MiB, for a total of approximately 190.7TiB. 1
Right now, in order to upload such a file, each 4000MiB block must be entirely read into memory before uploading. Both
reqwest
2 andhyper
3 support a mechanism to wrap a Stream, which would enable streaming the input, rather than reading it entirely in memory first.Work added in #230 started the work to support this by adding azure_core::SeekableStream.
Footnotes
https://docs.microsoft.com/en-us/azure/storage/blobs/scalability-targets#scale-targets-for-blob-storage ↩
https://docs.rs/reqwest/0.11.9/reqwest/struct.Body.html#method.wrap_stream ↩
https://docs.rs/hyper/latest/hyper/body/struct.Body.html#method.wrap_stream ↩
The text was updated successfully, but these errors were encountered: