Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add configurable parallelism to objstore.DownloadDir and objstore.UploadDir #5414

Closed
gburek-fastly opened this issue Jun 9, 2022 · 1 comment
Closed

Add configurable parallelism to objstore.DownloadDir and objstore.UploadDir #5414

gburek-fastly opened this issue Jun 9, 2022 · 1 comment
Labels
feature request/improvement

Comments

@gburek-fastly
Copy link

Our environment has large blocks that are comprised of many chunks:

512 MiB      gs://-/01G53KK11WRQTSM7M2W1217H5H/chunks/002371
512 MiB      gs://-/01G53KK11WRQTSM7M2W1217H5H/chunks/002372
512 MiB      gs://-/01G53KK11WRQTSM7M2W1217H5H/chunks/002373
187.95 MiB   gs://-/01G53KK11WRQTSM7M2W1217H5H/chunks/002374
1.16 TiB     gs://-/01G53KK11WRQTSM7M2W1217H5H/chunks/
1.18 TiB     gs://-/01G53KK11WRQTSM7M2W1217H5H/

They are slow to upload and download:

"duration":"5h40m44.011902037s","msg":"downloaded and verified blocks; compacting blocks"
...
"duration":"8h9m20.598265105s","msg":"uploaded block"

It appears that thanos iterates through a bucket and downloads these files sequentially and it does the same for uploading them.

It would be a huge performance boost for compact nodes to be able to concurrently upload and download the individual objects that a block is comprised of.
@yeya24 yeya24 mentioned this issue Jul 7, 2022
Merged
@kakkoyun
Copy link
Member

This is fixed by #5475

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature request/improvement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@kakkoyun @matej-g @gburek-fastly