Skip to content

Commit

Permalink
[Dataflow Streaming] Enable Windmill GetWork Response Batching by def…
Browse files Browse the repository at this point in the history
…ault (#33847)

With this change user workers will request batched responses from
backend and backend will send multiple WorkItems in the same response
proto. Backend changes are rollbacksafe.

The feature can be disabled by passing `--windmillRequestBatchedGetWorkResponse=false`
  • Loading branch information
arunpandianp authored Feb 4, 2025
1 parent 8805ab2 commit 908a50b
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 0 deletions.
4 changes: 4 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,10 @@
* Support the AnyOf Environment for execution in Prism ([#33705](https://github.com/apache/beam/pull/33705))
* This improves support for developing Xlang pipelines, when using a compatible cross language service.
* Partitions are now configurable for the DaskRunner in the Python SDK ([#33805](https://github.com/apache/beam/pull/33805)).
* [Dataflow Streaming] Enable Windmill GetWork Response Batching by default ([#33847](https://github.com/apache/beam/pull/33847)).
* With this change user workers will request batched GetWork responses from backend and backend will send multiple WorkItems in the same response proto.
* The feature can be disabled by passing `--windmillRequestBatchedGetWorkResponse=false`


## Breaking Changes

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -137,6 +137,7 @@ public interface DataflowStreamingPipelineOptions extends PipelineOptions {
void setUseSeparateWindmillHeartbeatStreams(Boolean value);

@Description("If true, GetWorkStreams will request multiple work items in a response chunk.")
@Default.Boolean(true)
boolean getWindmillRequestBatchedGetWorkResponse();

void setWindmillRequestBatchedGetWorkResponse(boolean value);
Expand Down

0 comments on commit 908a50b

Please sign in to comment.