Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Schemas] Use configured schema service in Avro processors #1743

Merged
merged 14 commits into from
Aug 6, 2024
Merged

[Schemas] Use configured schema service in Avro processors #1743

merged 14 commits into from
Aug 6, 2024

Conversation

hariso
Copy link
Contributor

@hariso hariso commented Aug 5, 2024 •
edited
Loading

Description

Closes #1653.

Depends on #1742.

Tested with Confluent's Schema Registry and two pipelines (one that encodes data from a file, and another that reads the encoded and decodes it back).

Quick checks

  • I have followed the Code Guidelines.
  • There is no other pull request for the same update/change.
  • I have written unit tests.
  • I have made sure that the PR is of reasonable size and can be easily reviewed.

@hariso hariso changed the title Haris/avro schema service [Schemas] Use configured schema service in Avro processors Aug 5, 2024
@hariso hariso changed the base branch from main to lovro/processor-middleware August 5, 2024 16:43
Base automatically changed from lovro/processor-middleware to main August 5, 2024 16:53
lovromazgon and others added 9 commits August 6, 2024 10:49
@lovromazgon @hariso
update processor SDK
e695eb0 
- apply processor middleware to builtin processors
- use config.Config for configuration map
- take into account middleware parameters in tests
@lovromazgon @hariso
organize imports
6246c22
@hariso
done with decode
5da9a36
@hariso
Use Conduit's schema registry in avro.encode
d827950
@hariso
fix test
e931df0
@hariso
fix
b87d063
@hariso
attach sr to processor
27d01f1
@hariso
linter
4ca5e96
@hariso
error msg
955d9c2
@hariso hariso force-pushed the haris/avro-schema-service branch from 93b4835 to 955d9c2 Compare August 6, 2024 08:58
@hariso hariso marked this pull request as ready for review August 6, 2024 09:19
@hariso hariso requested a review from a team as a code owner August 6, 2024 09:19
@hariso hariso enabled auto-merge (squash) August 6, 2024 09:28
lovromazgon
lovromazgon approved these changes Aug 6, 2024
View reviewed changes
Copy link
Member

@lovromazgon lovromazgon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just that one thing about the disabled middleware, approving in advance 👍

pkg/plugin/processor/builtin/impl/avro/encode.go Outdated Show resolved Hide resolved
pkg/plugin/processor/builtin/registry.go Show resolved Hide resolved
@lovromazgon lovromazgon disabled auto-merge August 6, 2024 11:00
@lovromazgon lovromazgon enabled auto-merge (squash) August 6, 2024 11:46
@lovromazgon lovromazgon merged commit 817d52c into main Aug 6, 2024
3 checks passed
@lovromazgon lovromazgon deleted the haris/avro-schema-service branch August 6, 2024 11:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lovromazgon lovromazgon lovromazgon approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Schemas] Use configured schema service in Avro processors
2 participants
@hariso @lovromazgon