-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Switch to s2 for bulk loader and backup/restore #9315
Switch to s2 for bulk loader and backup/restore #9315
Conversation
Thank you for the PR @RJKeevil. I don't see a direct benefit of doing this. It doesn't change dependency tree much. Performance for snappy is not clear that it improves it significantly either. Am I missing something? |
Yeh its a minor one, you had both dependencies anyway so I just thought it might be best to default to the one that claims to be faster and seems to receive more bug fixes and improvements. Where it becomes interesting is it would allow you to consider more compression modes in future, such as s2, which is a more significantly improved snappy. |
For Badger, we already use s2 instead of snappy. I think we can do that here too https://github.com/hypermodeinc/badger/blob/main/table/builder.go#L151 For connections, not sure what the best option is. I wouldn't change that unless we have a solid reasoning for it. |
fbcd553
to
9fc68d9
Compare
9fc68d9
to
ce7f1f1
Compare
@mangalaman93 made those changes |
@RJKeevil could you rebase on latest main? We merged a fix so that the failing tests pass. |
Head branch was pushed to by a user without write access
ce7f1f1
to
be163f3
Compare
Description
A small change, but i noticed https://github.com/klauspost/compress was already an indirect dependency of the repo, so this uses that version over the golang version as a drop in replacement. According to the repo it offers "better compression and concurrent streams".
Checklist