Skip to content

NIFI-14470 - PublishKafka; compatibility with PublishKafka26 keys #9897

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

greyp9
Copy link
Contributor

@greyp9 greyp9 commented Apr 23, 2025

The JIRA notes a difference in the handling of the kafka.key setting in the new PublishKafka processor, when
compared to the deprecated PublishKafka_2_6 processor.

  • as expression language is noted as supported in the component property, the code should honor that
  • the expression language resolved value should be used as is, rather than as a lookup into the FlowFile attribute set

There were a couple of other component properties that employ the expression language evaluation, so those
PropertyDescriptors have been updated to reflect the code usage.

As NiFi 2.0 was released in October 2024, there might be significant uptake of the processor (with its incompatible behavior). Weighing this against the compatibility expectations of the community, this seems like the best path forward, but it is also a reasonable point of discussion.

Summary

NIFI-14470

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using mvn clean install -P contrib-check
    • JDK 21

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

Copy link
Contributor

@exceptionfactory exceptionfactory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank for proposing a solution to this issue @greyp9.

On the one hand, the current implementation in arguably broken, since the Kafka Key property indicates support for Expression Language.

On the other hand, supporting EL for the property value introduces a level of indirection that could also be confusing.

Since this change aligns with the earlier implementation, it seems like the best way forward, but I will leave this open for a bit longer to see if there is any additional feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants