Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update links about referencing fields in processors #1424

Merged
merged 4 commits into from
Mar 12, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion pkg/plugin/processor/builtin/impl/avro/decode.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@ type decoder interface {
}

type decodeConfig struct {
// The field that will be encoded.
// The field that will be decoded.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" default:".Payload.After"`

// URL of the schema registry (e.g. http://localhost:8085)
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions pkg/plugin/processor/builtin/impl/avro/encode.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ type encoder interface {

type encodeConfig struct {
// The field that will be encoded.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" default:".Payload.After"`

// URL of the schema registry (e.g. http://localhost:8085)
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions pkg/plugin/processor/builtin/impl/base64/decode.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,10 @@ type decodeProcessor struct {
}

type decodeConfig struct {
// Field is the target field, as it would be addressed in a Go template (e.g. `.Payload.After.foo`).
// Note that it is not allowed to base64 decode the `.Position` field.
// Field is the reference to target field. Note that it is not allowed to
// base64 decode the `.Position` field.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" validate:"required,exclusion=.Position"`
}

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions pkg/plugin/processor/builtin/impl/base64/encode.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,10 @@ type encodeProcessor struct {
}

type encodeConfig struct {
// Field is the target field, as it would be addressed in a Go template (e.g. `.Payload.After.foo`).
// Note that it is not allowed to base64 encode the `.Position` field.
// Field is a reference to the target field. Note that it is not allowed to
// base64 encode the `.Position` field.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" validate:"required,exclusion=.Position"`
}

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 5 additions & 2 deletions pkg/plugin/processor/builtin/impl/field/convert.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,11 @@ func NewConvertProcessor(log.CtxLogger) sdk.Processor {
}

type convertConfig struct {
// Field is the target field, as it would be addressed in a Go template (e.g. `.Payload.After.foo`).
// you can only convert fields that are under `.Key` and `.Payload`, and said fields should contain structured data.
// Field is the target field that should be converted.
// Note that you can only convert fields in structured data under `.Key` and
// `.Payload`.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" validate:"required,regex=^\\.(Payload|Key).*"`
// Type is the target field type after conversion, available options are: string, int, float, bool.
Type string `json:"type" validate:"required,inclusion=string|int|float|bool"`
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion pkg/plugin/processor/builtin/impl/field/exclude.go
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,9 @@ func NewExcludeProcessor(log.CtxLogger) sdk.Processor {
}

type excludeConfig struct {
// Fields is a comma separated list of target fields, as they would be addressed in a Go template (e.g. `.Metadata,.Payload.After.foo`).
// Fields is a comma separated list of target fields which should be excluded.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Fields []string `json:"fields" validate:"required"`
}

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 6 additions & 2 deletions pkg/plugin/processor/builtin/impl/field/rename.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,12 @@ func NewRenameProcessor(log.CtxLogger) sdk.Processor {
}

type renameConfig struct {
// Mapping is a comma separated list of keys and values for fields and their new names (keys and values
// are separated by colons ":"). For example: `.Metadata.key:id,.Payload.After.foo:bar`.
// Mapping is a comma separated list of keys and values for fields and their
// new names (keys and values are separated by colons ":").
//
// For example: `.Metadata.key:id,.Payload.After.foo:bar`.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Mapping []string `json:"mapping" validate:"required"`
}

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions pkg/plugin/processor/builtin/impl/field/set.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,10 @@ func NewSetProcessor(log.CtxLogger) sdk.Processor {
}

type setConfig struct {
// Field is the target field, as it would be addressed in a Go template (e.g. `.Payload.After.foo`).
// Note that it is not allowed to set the `.Position` field.
// Field is the target field that will be set. Note that it is not allowed
// to set the `.Position` field.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" validate:"required,exclusion=.Position"`
// Value is a Go template expression which will be evaluated and stored in `field` (e.g. `{{ .Payload.After }}`).
Value string `json:"value" validate:"required"`
Expand Down
2 changes: 1 addition & 1 deletion pkg/plugin/processor/builtin/impl/field/set_paramgen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 5 additions & 3 deletions pkg/plugin/processor/builtin/impl/filter.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,12 @@ func (p *filterProcessor) Specification() (sdk.Specification, error) {
return sdk.Specification{
Name: "filter",
Summary: "Acknowledges all records that get passed to the filter.",
Description: `Acknowledges all records that get passed to the filter, so the records will be filtered out if
the condition provided to the processor is evaluated to "true".
Description: `Acknowledges all records that get passed to the filter, so
the records will be filtered out if the condition provided to the processor is
evaluated to ` + "`true`" + `.

**Important:** Make sure to add a condition to this processor, otherwise all records will be filtered out.`,
**Important:** Make sure to add a [condition](https://conduit.io/docs/processors/conditions)
lovromazgon marked this conversation as resolved.
Show resolved Hide resolved
to this processor, otherwise all records will be filtered out.`,
Version: "v0.1.0",
Author: "Meroxa, Inc.",
Parameters: map[string]config.Parameter{},
Expand Down
6 changes: 4 additions & 2 deletions pkg/plugin/processor/builtin/impl/json/decode.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,10 @@ func NewDecodeProcessor(log.CtxLogger) sdk.Processor {
}

type decodeConfig struct {
// Field is the target field, as it would be addressed in a Go template (e.g. `.Payload.After.foo`).
// you can only decode fields that are under `.Key` and `.Payload`.
// Field is a reference to the target field. Only fields that are under
// `.Key` and `.Payload` can be decoded.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" validate:"required,regex=^\\.(Payload|Key).*,exclusion=.Payload"`
}

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 4 additions & 2 deletions pkg/plugin/processor/builtin/impl/json/encode.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,10 @@ func NewEncodeProcessor(log.CtxLogger) sdk.Processor {
}

type encodeConfig struct {
// Field is the target field, as it would be addressed in a Go template (e.g. `.Payload.After.foo`).
// you can only encode fields that are under `.Key`, `.Payload.Before` and `.Payload.After`.
// Field is a reference to the target field. Only fields that are under
// `.Key` and `.Payload` can be encoded.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" validate:"required,regex=^\\.(Payload|Key).*,exclusion=.Payload"`
}

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions pkg/plugin/processor/builtin/impl/unwrap/debezium.go
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,9 @@ const (
)

type debeziumConfig struct {
// Field is a reference to the field which contains the Debezium record
// (see [`sdk.NewReferenceResolver`](https://github.com/ConduitIO/conduit-processor-sdk/blob/02b60dc7acad11a2fcdf0a0188a13cf573648650/util.go#L53-L66)).
// Field is a reference to the field that contains the Debezium record.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" validate:"regex=^.Payload" default:".Payload.After"`
}

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions pkg/plugin/processor/builtin/impl/unwrap/kafka_connect.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,9 @@ import (
)

type kafkaConnectConfig struct {
// Field is a reference to the field which contains the Kafka Connect record
// (see [`sdk.NewReferenceResolver`](https://github.com/ConduitIO/conduit-processor-sdk/blob/02b60dc7acad11a2fcdf0a0188a13cf573648650/util.go#L53-L66)).
// Field is a reference to the field that contains the Kafka Connect record.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" validate:"regex=^.Payload" default:".Payload.After"`
}

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 3 additions & 2 deletions pkg/plugin/processor/builtin/impl/unwrap/opencdc.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,9 @@ import (
)

type openCDCConfig struct {
// Field is a reference to the field which contains the OpenCDC record
// (see [`sdk.NewReferenceResolver`](https://github.com/ConduitIO/conduit-processor-sdk/blob/02b60dc7acad11a2fcdf0a0188a13cf573648650/util.go#L53-L66)).
// Field is a reference to the field that contains the OpenCDC record.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
Field string `json:"field" default:".Payload.After"`
}

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 9 additions & 8 deletions pkg/plugin/processor/builtin/impl/webhook/http.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,17 +50,18 @@ type httpConfig struct {
BackoffRetryMax time.Duration `json:"backoffRetry.max" default:"5s"`

// Specifies which field from the input record should be used as the body in
// the HTTP request. The value of this parameter should be a valid record
// field reference (see [`sdk.NewReferenceResolver`](https://github.com/ConduitIO/conduit-processor-sdk/blob/02b60dc7acad11a2fcdf0a0188a13cf573648650/util.go#L53-L66)).
// the HTTP request.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
RequestBodyRef string `json:"request.body" default:"."`
// Specifies in which field should the response body be saved.
// The value of this parameter should be a valid record field reference
// (see [`sdk.NewReferenceResolver`](https://github.com/ConduitIO/conduit-processor-sdk/blob/02b60dc7acad11a2fcdf0a0188a13cf573648650/util.go#L53-L66)).
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
ResponseBodyRef string `json:"response.body" default:".Payload.After"`
// Specifies in which field should the response status be saved.
// The value of this parameter should be a valid record field reference
// (see [`sdk.NewReferenceResolver`](https://github.com/ConduitIO/conduit-processor-sdk/blob/02b60dc7acad11a2fcdf0a0188a13cf573648650/util.go#L53-L66)).
// If no value is set, then the response status will NOT be saved.
// Specifies in which field should the response status be saved. If no value
// is set, then the response status will NOT be saved.
//
// For more information about the format, see [Referencing fields](https://conduit.io/docs/processors/referencing-fields).
ResponseStatusRef string `json:"response.status"`
}

Expand Down
Loading