Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify production code to perform ingest job tracker updates based on a state store change sync/asynchronously #4282

Closed
patchwork01 opened this issue Feb 18, 2025 · 0 comments · Fixed by #4322

Comments

@patchwork01
Copy link
Collaborator

patchwork01 commented Feb 18, 2025

Background

Split from:

Follows on from:

Description

When we add a transaction that requires a job tracker update, we may need to provide a job tracker to apply the update synchronously. In that case we'd like the job tracker update to be done directly.

We'd like there to be a record in the transaction log that this has happened so that anything listening to the log will not duplicate this.

Analysis

Note that whether updates are done synchronously or not is configured in separate properties for each update type. Whether it should be done synchronously or not will need to be decided by the code that calls the state store.

We could add a method on the transaction object to apply it synchronously to the state store, which will also perform the job tracker updates and throw any failures.

@patchwork01 patchwork01 added this to the 0.29.0 milestone Feb 18, 2025
@patchwork01 patchwork01 changed the title Option on state store to perform job tracker updates sync/asynchronously Simplify production code to perform job tracker updates based on a state store change sync/asynchronously Feb 21, 2025
@patchwork01 patchwork01 changed the title Simplify production code to perform job tracker updates based on a state store change sync/asynchronously Simplify production code to perform ingest job tracker updates based on a state store change sync/asynchronously Feb 26, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants