Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cdc: Enable follower reads for changefeed #114611

Closed
miretskiy opened this issue Nov 16, 2023 · 2 comments · Fixed by #120077
Closed

cdc: Enable follower reads for changefeed #114611

miretskiy opened this issue Nov 16, 2023 · 2 comments · Fixed by #120077
Assignees
@rharding6373
Labels
A-cdc Change Data Capture C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-cdc

Comments

@miretskiy
Copy link
Contributor

miretskiy commented Nov 16, 2023 •
edited by cockroach-jira-scripts
Loading

Enable the useof followers when running changefeeds once #114528 merges.

Jira issue: CRDB-33595
@miretskiy miretskiy added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-cdc Change Data Capture T-cdc labels Nov 16, 2023
@blathers-crl blathers[crl]
Copy link

blathers-crl bot commented Nov 16, 2023

cc @cockroachdb/cdc

@jayshrivastava jayshrivastava mentioned this issue Nov 27, 2023
Closed
@nicktrav nicktrav assigned jayshrivastava and unassigned miretskiy Dec 20, 2023
@rharding6373 rharding6373 self-assigned this Mar 21, 2024
@rharding6373
Copy link
Collaborator

This is the same as #119777 which I'm currently working on.

@rharding6373 rharding6373 mentioned this issue Mar 21, 2024
Merged
craig bot pushed a commit that referenced this issue Mar 21, 2024
@rharding6373
Merge #120077
5242dcb 
120077: changefeedccl: use new bulk oracle for changefeed planning r=rharding6373 a=rharding6373

This change uses the BulkOracle by default as part of changefeed planning, instead of the bin packing oracle. This will allow changefeeds to have plans that randomly assign spans to any replica, including followers if enabled, following locality filter constraints.

A new cluster setting, `changefeed.balanced_distribution.enabled`, protects this change. When enabled (by default), changefeeds will use the new BulkOracle for planning. If disabled, changefeeds will use the previous bin packing oracle.

Epic: none
Fixes: #119777
Fixes: #114611

Release note (enterprise change): Changefeeds now use the BulkOracle for planning, which distributes work evenly across all replica in the locality filter, including followers if enabled. This is enabled by default with the cluster setting
`changefeed.balanced_distribution.enabled`. If disabled, changefeed planning reverts to its previous bin packing oracle.

Co-authored-by: rharding6373 <[email protected]>
@craig craig bot closed this as completed in 84e0e51 Mar 22, 2024
@rharding6373 rharding6373 mentioned this issue May 31, 2024
Merged
rharding6373 added a commit to rharding6373/cockroach that referenced this issue May 31, 2024
@rharding6373
changefeedccl: use new bulk oracle for changefeed planning
71c5185 
This change uses the BulkOracle by default as part of changefeed
planning, instead of the bin packing oracle. This will allow changefeeds
to have plans that randomly assign spans to any replica, including
followers if enabled, following locality filter constraints.

A new cluster setting, `changefeed.balanced_distribution.enabled`,
protects this change. When enabled (by default), changefeeds will use
the new BulkOracle for planning. If disabled, changefeeds will use the
previous bin packing oracle.

Epic: none
Fixes: cockroachdb#119777
Fixes: cockroachdb#114611

Release note (enterprise change): Changefeeds now use the BulkOracle for
planning, which distributes work evenly across all replica in the
locality filter, including followers if enabled. This is enabled by
default with the cluster setting
`changefeed.balanced_distribution.enabled`. If disabled, changefeed
planning reverts to its previous bin packing oracle.
@rharding6373 rharding6373 mentioned this issue May 31, 2024
Merged
rharding6373 added a commit to rharding6373/cockroach that referenced this issue May 31, 2024
@rharding6373
changefeedccl: use new bulk oracle for changefeed planning
19175a9 
This change uses the BulkOracle by default as part of changefeed
planning, instead of the bin packing oracle. This will allow changefeeds
to have plans that randomly assign spans to any replica, including
followers if enabled, following locality filter constraints.

A new cluster setting, `changefeed.balanced_distribution.enabled`,
protects this change. When enabled (by default), changefeeds will use
the new BulkOracle for planning. If disabled, changefeeds will use the
previous bin packing oracle.

Epic: none
Fixes: cockroachdb#119777
Fixes: cockroachdb#114611

Release note (enterprise change): Changefeeds now use the BulkOracle for
planning, which distributes work evenly across all replica in the
locality filter, including followers if enabled. This is enabled by
default with the cluster setting
`changefeed.balanced_distribution.enabled`. If disabled, changefeed
planning reverts to its previous bin packing oracle.
rharding6373 added a commit to rharding6373/cockroach that referenced this issue Jun 3, 2024
@rharding6373
changefeedccl: use new bulk oracle for changefeed planning
740d057 
This change uses the BulkOracle by default as part of changefeed
planning, instead of the bin packing oracle. This will allow changefeeds
to have plans that randomly assign spans to any replica, including
followers if enabled, following locality filter constraints.

A new cluster setting, `changefeed.balanced_distribution.enabled`,
protects this change. When enabled (by default), changefeeds will use
the new BulkOracle for planning. If disabled, changefeeds will use the
previous bin packing oracle.

Epic: none
Fixes: cockroachdb#119777
Fixes: cockroachdb#114611

Release note (enterprise change): Changefeeds now use the BulkOracle for
planning, which distributes work evenly across all replica in the
locality filter, including followers if enabled. This is enabled by
default with the cluster setting
`changefeed.balanced_distribution.enabled`. If disabled, changefeed
planning reverts to its previous bin packing oracle.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rharding6373 rharding6373

Labels
A-cdc Change Data Capture C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-cdc
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

changefeedccl: use new bulk oracle for changefeed planning rharding6373/cockroach
3 participants
@rharding6373 @jayshrivastava @miretskiy