Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kv: reduce gRPC window size to avoid OOM #2699

Merged
merged 8 commits into from
Sep 2, 2021
Merged

kv: reduce gRPC window size to avoid OOM #2699

Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
dbc5848
kv: reduce gRPC window size to avoid OOM
overvenus Sep 1, 2021
08bffa8
Merge branch 'master' into shrink-grpc-window
overvenus Sep 1, 2021
e83449a
go fmt
overvenus Sep 1, 2021
99ec05b
Merge branch 'master' into shrink-grpc-window
ti-chi-bot Sep 2, 2021
e7de250
address comments
overvenus Sep 2, 2021
2ff79b1
Merge branch 'shrink-grpc-window' of github.com:overvenus/ticdc into …
overvenus Sep 2, 2021
c68caa0
Merge branch 'master' into shrink-grpc-window
ti-chi-bot Sep 2, 2021
a5a48f8
Merge branch 'master' into shrink-grpc-window
ti-chi-bot Sep 2, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions cdc/kv/client.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,10 @@ import (
const (
dialTimeout = 10 * time.Second
tikvRequestMaxBackoff = 20000 // Maximum total sleep time(in ms)
// TODO find optimal values and test extensively before releasing
// The old values cause the gRPC stream to stall for some unknown reason.
grpcInitialWindowSize = 1 << 26 // 64 MB The value for initial window size on a stream
grpcInitialConnWindowSize = 1 << 27 // 128 MB The value for initial window size on a connection
// TiCDC may open numerous gRPC streams,
// with 64KB window size, 10K streams takes about 27GB memory.
grpcInitialWindowSize = 65535 // 64 KB The value for initial window size on a stream
grpcInitialConnWindowSize = 1 << 23 // 8 MB The value for initial window size on a connection
grpcMaxCallRecvMsgSize = 1 << 28 // 256 MB The maximum message size the client can receive

// The threshold of warning a message is too large. TiKV split events into 6MB per-message.
Expand Down