Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

4282 - Handle sync/asynchronous commit of ingest job tracker updates #4322

Merged
merged 18 commits into from
Feb 27, 2025
Merged

4282 - Handle sync/asynchronous commit of ingest job tracker updates #4322

merged 18 commits into from
Feb 27, 2025

Conversation

rtjd6554
Copy link
Collaborator

@rtjd6554 rtjd6554 commented Feb 26, 2025 •
edited by patchwork01
Loading

Make sure you have checked all steps below.

Issue

Tests

  • My PR adds the following tests OR does not need testing for this extremely good reason:
    • New tests in BulkImportJobDriverTest, AddFilesToStateStoreTest
    • Run existing test suites

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it, or I have linked to a
    separate issue for that below.
  • If I have added or removed any dependencies from the project, I have updated the NOTICES file.

rtjd6554 and others added 18 commits February 26, 2025 11:16
@rtjd6554
Inclue updateTrackerFromLog as part of bulk import and new test
8470e1b
@patchwork01
Add test shouldCommitSynchronouslyWithJob
2cc0b1d
@patchwork01
Test ingest synchronous commit with job
59e291f
@patchwork01
Add new asynchronous commit setup in AddFilesToStateStore
97f7db2
@patchwork01
Remove old asynchronous signature from AddFilesToStateStore
2acc220
@patchwork01
Reformat AddFilesToStateStore
1cbf7f8
@patchwork01
Remove AddFilesTransaction.updateTrackerFromLog
97bc092
@patchwork01
Adjust naming in AddFilesToStateStoreTest
1721d6d
@patchwork01
Track failure of synchronous standard ingest commit
16b6271
@patchwork01
Use IngestJobRunIds in asynchronous commit
31dcd9f
@patchwork01
Add missing Javadoc in IngestJobRunIds
19c20f9
@patchwork01
Add more information on AddFilesTransaction.Builder.jobRunIds
fc50ec9
@patchwork01
Use IngestJobRunIds in BulkImportJobDriver
3fc4809
@patchwork01
Fix Javadoc in IngestJobRunIds
1613785
@patchwork01
Set table ID from table properties in IngestJobRunner
360443a
@patchwork01
Log job run IDs in asynchronous ingest commit
72329a8
@patchwork01
Log job run IDs in BulkImportJobDriver
0cd2098
@patchwork01
Improve logging/failure tracking
47450e9
@patchwork01 patchwork01 changed the title 4282 - Simplify production code to perform job tracker updates based on a state store change sync/asynchronously 4282 - Handle sync/asynchronous commit of ingest job tracker updates Feb 26, 2025
@patchwork01 patchwork01 marked this pull request as ready for review February 26, 2025 13:23
@patchwork01 patchwork01 added the pr-base-for-stacking Base for stacked pull requests (a dependency for others, where this PR's branch will be the base) label Feb 26, 2025
@gaffer01 gaffer01 assigned rtjd6554 and unassigned gaffer01 Feb 26, 2025
@rtjd6554 rtjd6554 merged commit ad0ae08 into develop Feb 27, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gaffer01 gaffer01 gaffer01 approved these changes

@patchwork01 patchwork01 patchwork01 approved these changes

Assignees

@rtjd6554 rtjd6554

Labels
pr-base-for-stacking Base for stacked pull requests (a dependency for others, where this PR's branch will be the base)
Projects
None yet
Milestone
No milestone
3 participants
@rtjd6554 @gaffer01 @patchwork01