Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

improves: Error in onPartitionsAssigned in parallel consumer #537

Merged
merged 5 commits into from
Jul 17, 2023
Merged

improves: Error in onPartitionsAssigned in parallel consumer #537

merged 5 commits into from
Jul 17, 2023

Conversation

nachomdo
Copy link
Contributor

@nachomdo nachomdo commented Jan 13, 2023 •
edited by eddyv
Loading

Improves: #326

Adds an option to pass an invalid offset metadata error policy in case of accidentally or deliberately reusing a Kafka Streams consumer group. The default policy is to fail and stop the execution.

Checklist

  • Documentation (if applicable)
  • Changelog

@what-the-diff
Copy link

what-the-diff bot commented Jan 13, 2023

  • Added a new enum InvalidOffsetMetadataHandlingPolicy to ParallelConsumerOptions.java
  • Updated EncodedOffsetPair and OffsetMapCodecManager classes with the following changes:
    • added an optional parameter errorPolicy of type InvalidOffsetMetadataHandlingPolicy in getDecodedIncompletes() method, which is set to FAIL by default
    • updated all callsites of getDecodedIncompletes() method accordingly (added this new parameter)
  • In WorkManagerOffSetMapCodecMangerTest class, two tests were added for testing deserialiseKafkaStreamsV1WithIgnoreErrorPolicy(): one test uses IGNORE policy while another uses DEFAULT(FAIL) policy as expected behaviour when Kafka Streams V1 magic numbers are found in offset metadata

@nachomdo nachomdo marked this pull request as ready for review January 16, 2023 18:49
eddyv
eddyv requested changes Jun 20, 2023
View reviewed changes
Copy link
Member

@eddyv eddyv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add one more test and LGTM.

...rc/test/java/io/confluent/parallelconsumer/offsets/WorkManagerOffsetMapCodecManagerTest.java Outdated Show resolved Hide resolved
@eddyv eddyv added the wait for info Waiting for additional info from user label Jun 20, 2023
@nachomdo nachomdo force-pushed the improvements/reuse-existing-offsets branch from 30052f3 to 60f089f Compare July 10, 2023 10:40
@nachomdo nachomdo requested a review from eddyv July 10, 2023 10:40
eddyv
eddyv approved these changes Jul 17, 2023
View reviewed changes
Copy link
Member

@eddyv eddyv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@eddyv eddyv merged commit 8417498 into confluentinc:master Jul 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eddyv eddyv eddyv approved these changes

Assignees
No one assigned
Labels
wait for info Waiting for additional info from user
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@nachomdo @eddyv