gemini: partition key generation improved #159
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A dedictated source concept is introduced and producing and consuming partition keys are both coordinated through this object. The size of the buffers governing how many partition keys are buffered can be controlled through the CLI args `partition-key-buffer-size` and `partition-key-buffer-reuse-size` respectively. The `concurrency` concept is somewhat changed into meaning that each `job` is running with this number of goroutines. So for a `mixed` mode it means that `2*concurrency` number of goroutines will be running at the same time. You may need to rescale your command line arg `concurrency` accordingly.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A dedictated source concept is introduced and producing and consuming
partition keys are both coordinated through this object. The size of
the buffers governing how many partition keys are buffered can be
controlled through the CLI args
partition-key-buffer-sizeandpartition-key-buffer-reuse-sizerespectively.The
concurrencyconcept is somewhat changed into meaning thateach
jobis running with this number of goroutines. So fora
mixedmode it means that2*concurrencynumber of goroutineswill be running at the same time. You may need to rescale your
command line arg
concurrencyaccordingly.Relates to: #158