Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(storage/dataflux): run worksteal listing parallel to sequential listing #10966

Merged
merged 62 commits into from
Nov 25, 2024
Merged

feat(storage/dataflux): run worksteal listing parallel to sequential listing #10966

Changes from 1 commit
Commits
Show all changes
62 commits
Select commit Hold shift + click to select a range
3ab3831
test case for fast-list
akansha1812 Sep 25, 2024
559ae40
resolve comments
akansha1812 Sep 27, 2024
1cdc956
add object_lister and worksteal to fast_list
akansha1812 Oct 7, 2024
7eb0471
Merge branch 'googleapis:main' into main
akansha1812 Oct 7, 2024
1f05177
add unit tests with emulator
akansha1812 Oct 8, 2024
05b1515
Merge branch 'googleapis:main' into main
akansha1812 Oct 8, 2024
5bed09c
resolve PR errors
akansha1812 Oct 8, 2024
80054bf
reduce numobjects to resolve timeout error
akansha1812 Oct 8, 2024
e33adf5
reduce objects created for timeout error
akansha1812 Oct 8, 2024
b1b69b9
remove env variables for grpc and http
akansha1812 Oct 9, 2024
c2e56f8
run dataflux emulator tests
akansha1812 Oct 9, 2024
c78aaba
Merge branch 'main' into main
akansha1812 Oct 14, 2024
b714222
Merge branch 'main' into main
akansha1812 Oct 21, 2024
4028cb6
resolve comments
akansha1812 Oct 21, 2024
4851cc8
update ranges to nil when sequential listing is faster
akansha1812 Oct 21, 2024
0d494e0
default page size for seq listing is 5000
akansha1812 Oct 21, 2024
722e961
remove version enabled from TestDoSeqListingEmulated
akansha1812 Oct 21, 2024
f3c1571
increase emulator time
akansha1812 Oct 21, 2024
db1a757
make next_page more readable
akansha1812 Oct 22, 2024
56ab509
Merge branch 'main' into main
akansha1812 Oct 22, 2024
2b72b0e
to resolve race conditions
akansha1812 Oct 22, 2024
692e74e
rename goroutineID to id
akansha1812 Oct 22, 2024
a6cb0f4
Merge branch 'main' into main
akansha1812 Oct 23, 2024
1998873
move counter from beginning of the loop
akansha1812 Oct 25, 2024
7583e16
add mutex to error counter
akansha1812 Oct 25, 2024
4b1dcf0
emulator test for error counter and remove worker to track error from…
akansha1812 Oct 25, 2024
34f0840
Merge branch 'main' into main
akansha1812 Oct 25, 2024
564bbd3
test emulator error
akansha1812 Oct 25, 2024
ab830f6
grpc client returns context eror in desc
akansha1812 Oct 28, 2024
e14c31a
Merge branch 'main' into main
akansha1812 Oct 28, 2024
5af30c7
Merge branch 'main' into main
akansha1812 Oct 29, 2024
328f2b7
Merge branch 'main' into main
akansha1812 Oct 31, 2024
d539c41
Merge branch 'googleapis:main' into main
akansha1812 Nov 14, 2024
ba845cc
rename example test
akansha1812 Nov 14, 2024
c245987
rearrage lister methods
akansha1812 Nov 14, 2024
71d12c0
change close comment
akansha1812 Nov 14, 2024
8a9e007
rename context err counter
akansha1812 Nov 14, 2024
4a15b2b
make var cc more descriptive
akansha1812 Nov 14, 2024
8c80263
reuse ctx var
akansha1812 Nov 14, 2024
25ae3cc
update prefixAdjustedOffsets as per comment
akansha1812 Nov 14, 2024
53fb97d
update prefixAdjustedOffsets as per comment
akansha1812 Nov 14, 2024
510cd17
update comments
akansha1812 Nov 15, 2024
539f54b
switch case for open,worksteal and listing method
akansha1812 Nov 15, 2024
11c45c4
typo
akansha1812 Nov 15, 2024
2ffcca1
dynamic split
akansha1812 Nov 15, 2024
de35c84
updated comments
akansha1812 Nov 15, 2024
ab435ec
update next_page to be more readable
akansha1812 Nov 18, 2024
0be446b
update worksteal comments
akansha1812 Nov 18, 2024
88c88b0
Merge branch 'googleapis:main' into main
akansha1812 Nov 19, 2024
a662121
resolve comments
akansha1812 Nov 19, 2024
1436512
change nextpage name
akansha1812 Nov 20, 2024
7cc5fc7
complete nextPage] comment
akansha1812 Nov 20, 2024
72bb590
change writer wc to w
akansha1812 Nov 20, 2024
2dbe02e
change listNextPageSequentially comment
akansha1812 Nov 20, 2024
111852d
update comments
akansha1812 Nov 20, 2024
e4e81f1
add range splitter test
akansha1812 Nov 20, 2024
84b6d50
PageInfo().MaxSize to wsDefaultPageSize so not objects are discarded
akansha1812 Nov 20, 2024
ab10ba2
remove space
akansha1812 Nov 20, 2024
25ee9d9
update example-test comment
akansha1812 Nov 20, 2024
8e5eeec
update comment addCharsToAlphabet
akansha1812 Nov 21, 2024
a219001
Merge branch 'main' into main
akansha1812 Nov 23, 2024
9e60f5e
Merge branch 'main' into main
BrennaEpp Nov 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'main' into main
  • Loading branch information
@akansha1812
akansha1812 authored Oct 25, 2024
commit 34f0840ead69841ac387429b06085cf57938d9be

This merge commit was added into this branch cleanly.

There are no new changes to show, but you can still view the diff.