Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase memory allocation for JDBC buffers in source DB connectors #20939

Merged
merged 23 commits into from
Jan 27, 2023
Merged
Changes from 2 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
6484a0b
Bump total allocation of the source's available memory for the JDBC b…
akashkulk Dec 30, 2022
9419e2e
Merge branch 'master' into larger-db-buffer
akashkulk Dec 30, 2022
e7abdf1
Merge branch 'master' into larger-db-buffer
akashkulk Jan 4, 2023
021b578
Merge branch 'master' into larger-db-buffer
akashkulk Jan 17, 2023
9c47fd5
Profiling setup
akashkulk Jan 18, 2023
945f700
Merge branch 'master' into larger-db-buffer
akashkulk Jan 18, 2023
63fd8ee
Merge branch 'master' into larger-db-buffer
akashkulk Jan 19, 2023
3d32a3e
Merge branch 'master' into larger-db-buffer
akashkulk Jan 24, 2023
62e1fa4
Merge branch 'master' into larger-db-buffer
akashkulk Jan 25, 2023
ea7abe1
Update Dockerfile
akashkulk Jan 18, 2023
9b875a5
Fixes
akashkulk Jan 25, 2023
0b77512
Merge branch 'master' into larger-db-buffer
akashkulk Jan 25, 2023
dd0292a
Merge branch 'master' into larger-db-buffer
akashkulk Jan 26, 2023
c207ee5
Update FetchSizeConstants.java
akashkulk Jan 26, 2023
ca1c6a4
Update dockerfiles
akashkulk Jan 26, 2023
3936822
Merge branch 'master' into larger-db-buffer
akashkulk Jan 26, 2023
397e1c1
Merge branch 'master' into larger-db-buffer
akashkulk Jan 26, 2023
4dd3576
Merge branch 'master' into larger-db-buffer
akashkulk Jan 26, 2023
59ed2a5
Merge branch 'master' into larger-db-buffer
akashkulk Jan 26, 2023
23265b2
Merge branch 'master' into larger-db-buffer
akashkulk Jan 27, 2023
58809bd
Update JdbcStreamingQueryConfig.java
akashkulk Jan 27, 2023
94925df
auto-bump connector version
octavia-squidington-iii Jan 27, 2023
6b4f90f
Manually bump source-mysql
akashkulk Jan 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ public final class FetchSizeConstants {
// This size is not enforced. It is only used to calculate a proper
// fetch size. The max row size the connector can handle is actually
// limited by the heap size.
public static final double TARGET_BUFFER_SIZE_RATIO = 0.6;
public static final double TARGET_BUFFER_SIZE_RATIO = 0.7;
public static final long MIN_BUFFER_BYTE_SIZE = 250L * 1024L * 1024L; // 250 MB
public static final long MAX_BUFFER_BYTE_SIZE = 1024L * 1024L * 1024L; // 1GB
// sample size for making the first estimation of the row size
Expand Down