[Forward main] Enable message time-to-live checker configuration #11956
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Test Results1 009 files + 1 1 009 suites +1 1h 54m 6s ⏱️ + 5m 54s Results for commit c2f562b. ± Comparison against base commit 8bd8399. This pull request removes 518 and adds 739 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
I want to add a parameter to the constructor, but discovered there was this unused one as well. To reduce the effort, I just deleted this one. The unused constructor was keeping the typedRecordProcessorFactory from being final. So, now it can be made final as well. (cherry picked from commit 76e13ec)
The EngineCfg will be used for users to configure the engine at a system level. It does not allow to configure the different engines per partition. All configurations will be applied to the engine of each partition. This requires changes in both the broker and the engine. The engine does not have a dependency on the broker. So the EngineCfg needs to be mapped to an EngineConfig that is specified by the workflow engine. (cherry picked from commit 63dcfb9) Conflicts: broker/src/main/java/io/camunda/zeebe/broker/system/partitions/impl/steps/StreamProcessorTransitionStep.java engine/src/test/java/io/camunda/zeebe/engine/util/TestStreams.java
Adds the the `broker.engine.messages` configuration, allowing to configure the: - `ttlCheckerBatchLimit` - `ttlCheckerInterval` (cherry picked from commit 271c221)
We're not yet sure whether we want to expose these configuration options going forward. We may also move the recent changes into feature flags, which would make those changes experimental as well. It makes only sense to move the entire config settings for this into the experimental section. (cherry picked from commit 31cbbf8) Conflicts: broker/src/main/java/io/camunda/zeebe/broker/system/configuration/ExperimentalCfg.java broker/src/main/java/io/camunda/zeebe/broker/system/partitions/impl/steps/StreamProcessorTransitionStep.java
This enables the experimental configuration settings: - zeebe.broker.experimental.engine.messages.ttlCheckerBatchLimit - zeebe.broker.experimental.engine.messages.ttlCheckerInterval These settings are now used by the MessageTimeToLiveChecker. (cherry picked from commit b5d4d4a) Conflicts: engine/src/main/java/io/camunda/zeebe/engine/processing/EngineProcessors.java engine/src/main/java/io/camunda/zeebe/engine/processing/streamprocessor/TypedRecordProcessorContext.java engine/src/main/java/io/camunda/zeebe/engine/processing/streamprocessor/TypedRecordProcessorContextImpl.java engine/src/test/java/io/camunda/zeebe/engine/processing/message/MessageStreamProcessorTest.java
The checker is run async, it is no longer running on the same actor/thread (cherry picked from commit bc3a7e6)
Running the message TTL Checker asynchronous is mostly untested. To avoid unforseen situations (especially as we are releasing this in a patch release), it should be disabled by default and can be enabled if users want to try it out. We can choose whether or not we want to enable this feature by default with the upcoming 8.2 release. (cherry picked from commit 7a2576a) Conflicts: broker/src/main/java/io/camunda/zeebe/broker/system/configuration/FeatureFlagsCfg.java dist/src/main/config/broker.standalone.yaml.template dist/src/main/config/broker.yaml.template engine/src/main/java/io/camunda/zeebe/engine/processing/message/MessageEventProcessors.java engine/src/test/java/io/camunda/zeebe/engine/processing/message/MessageStreamProcessorTest.java util/src/main/java/io/camunda/zeebe/util/FeatureFlags.java util/src/test/java/io/camunda/zeebe/util/FeatureFlagsTest.java
This means the behavior is effectively equivalent and user space is unaffected (unless there are more than max int messages to expire). We can adjust the default value in minor releases. (cherry picked from commit 78b3349)
korthout
force-pushed
the
forwardport-11947-to-main
branch
from
b773400 to
c2f562b
Compare
|
Discussed with @oleschoenburg and looks good to merge bors merge |
ghost
pushed a commit
that referenced
this pull request
Mar 8, 2023
11956: [Forward main] Enable message time-to-live checker configuration r=korthout a=korthout ## Description <!-- Please explain the changes you made here. --> Forwards #11947 to `main`. Conflicts were resolved by hand in multiple commits. Please review the commits with the `Conflicts:` section in the commit message with additional care. ## Related issues <!-- Which issues are closed by this PR or are related --> closes #11922 Co-authored-by: Nico Korthout <[email protected]>
|
Build failed: |
|
Infra problems with GH. bors merge |
|
Build succeeded: |
ghost
merged commit 
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Forwards #11947 to
main.Conflicts were resolved by hand in multiple commits. Please review the commits with the
Conflicts:section in the commit message with additional care.Related issues
closes #11922
Definition of Done
Not all items need to be done depending on the issue and the pull request.
Code changes:
backport stable/1.3) to the PR, in case that fails you need to create backports manually.Testing:
Documentation:
Other teams:
If the change impacts another team an issue has been created for this team, explaining what they need to do to support this change.
Please refer to our review guidelines.