Add a coalescing writer #1537
Merged
Add a coalescing writer #1537
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Motivation: Creating frames in HTTP/2 isn't free: they incur one allocation for the frame payload and another for the data buffer. In addition to this HTTP/2 currently creates a promise per write, so we also have an additional write promise. On top of the allocations we have the repeated work for processing extra frames. We can recover much of this cost by coalescing writes before a flush. For streaming RPCs this can have a fairly significant impact. Modifications: - Add a coalescing length prefixed writer: serialized messages are appended and length prefixing (and compression) is done lazily on calls to `next()`. - Add an internal 'one-or-many' queue which is a FIFO queue backed by either a single element on the stack or a deque. This avoids allocating a queue on the heap for the length prefixed writer when there is only one message appended before flushing. Result: Nothing changes yet, but we have a writer in place which we can adopt later.
glbrntt
force-pushed
the
gb-coalescing-writer
branch
from
0ff2fe9 to
fbf33b4
Compare
FranzBusch
approved these changes
Dec 21, 2022
WendellXY
pushed a commit
to sundayfun/grpc-swift
that referenced
this pull request
Aug 24, 2023
Motivation: Creating frames in HTTP/2 isn't free: they incur one allocation for the frame payload and another for the data buffer. In addition to this HTTP/2 currently creates a promise per write, so we also have an additional write promise. On top of the allocations we have the repeated work for processing extra frames. We can recover much of this cost by coalescing writes before a flush. For streaming RPCs this can have a fairly significant impact. Modifications: - Add a coalescing length prefixed writer: serialized messages are appended and length prefixing (and compression) is done lazily on calls to `next()`. - Add an internal 'one-or-many' queue which is a FIFO queue backed by either a single element on the stack or a deque. This avoids allocating a queue on the heap for the length prefixed writer when there is only one message appended before flushing. Result: Nothing changes yet, but we have a writer in place which we can adopt later.
pinlin168
pushed a commit
to sundayfun/grpc-swift
that referenced
this pull request
Aug 24, 2023
Motivation: Creating frames in HTTP/2 isn't free: they incur one allocation for the frame payload and another for the data buffer. In addition to this HTTP/2 currently creates a promise per write, so we also have an additional write promise. On top of the allocations we have the repeated work for processing extra frames. We can recover much of this cost by coalescing writes before a flush. For streaming RPCs this can have a fairly significant impact. Modifications: - Add a coalescing length prefixed writer: serialized messages are appended and length prefixing (and compression) is done lazily on calls to `next()`. - Add an internal 'one-or-many' queue which is a FIFO queue backed by either a single element on the stack or a deque. This avoids allocating a queue on the heap for the length prefixed writer when there is only one message appended before flushing. Result: Nothing changes yet, but we have a writer in place which we can adopt later.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation:
Creating frames in HTTP/2 isn't free: they incur one allocation for the frame payload and another for the data buffer. In addition to this HTTP/2 currently creates a promise per write, so we also have an additional write promise. On top of the allocations we have the repeated work for processing extra frames.
We can recover much of this cost by coalescing writes before a flush. For streaming RPCs this can have a fairly significant impact.
Modifications:
next().Result:
Nothing changes yet, but we have a writer in place which we can adopt later.