Support overflow_action in v14 api
#943
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tagomoris
force-pushed
the
support-overflow-action-in-v14-api
branch
from
5b23cc0 to
d3154f1
Compare
tagomoris
changed the title
overflow_action in v14 api
|
This fix changes more code than expected: details are in commit comment here: |
tagomoris
force-pushed
the
support-overflow-action-in-v14-api
branch
from
24cd94c to
9d7ae8c
Compare
* without this fix, some events will be written paritally if any chunks raise error in #append/#concat
* it makes duplicated events
Buffer#write will do: (method name was changed: #emit is method names for event routers)
* this method receives pairs of metadata and data to be written at once
* append/concat these data to chunks (not committed)
* commit first chunk
* if succeeded, then commit all (even if any following chunk raises error)
* if failed, rollback all
In memory/file buffer, #commit is very lightweight operation and will NOT fail in most cases.
This change requires some additional internal APIs for buffers/chunks
* chunk status in general: this is required to make #write thread safe
* keyword argument of #write: bulk and enqueue
* #write method becomes much more complex, so bulk operation should be merged into an implementation
(we can't maintain two different/similar methods)
* #write method enqueues chunks if needed, so :immediate mode should be implemented in same level
* chunk_full_threshold configuration parameter to control "chunk_size_full?"
* bulk and non-bulk #write were merged
* In non-bulk mode, it's too rare that written chunk content bytesize is same with limitation
tagomoris
force-pushed
the
support-overflow-action-in-v14-api
branch
from
9d7ae8c to
6de12b0
Compare
|
I confirmed that CI is green now. |
This was referenced May 17, 2016
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I found that implementing
buffer_queue_full_actionin v0.14 requires change of internal API of Buffer.This pull-request does:
Buffer#emitandBuffer#emit_bulktoBuffer#writeBuffer#writeto get a set of metadata and its data (as a hash) to control commit/rollback as a transactionThis change is based on #912