Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update processors docs #91

Merged
merged 6 commits into from
May 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 134 additions & 0 deletions docs/processors/builtin/error.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
---
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'error'
sidebar_position: 5
---

import ReactDiffViewer from 'react-diff-viewer';
import Chip from '@mui/material/Chip';
import Box from "@mui/system/Box";
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

# `error`

Returns an error for all records that get passed to the processor.

## Description

Any time a record is passed to this processor it returns an error,
which results in the record being sent to the DLQ if it's configured, or the pipeline stopping.

**Important:** Make sure to add a [condition](https://conduit.io/docs/processors/conditions)
to this processor, otherwise all records will trigger an error.

## Configuration parameters

<Tabs groupId="config-params">
<TabItem value="yaml" label="YAML">
```yaml
version: 2.2
pipelines:
- id: example
status: running
connectors:
# define source and destination ...
processors:
- id: example
plugin: "error"
settings:
# Error message to be returned. This can be a Go
# [template](https://pkg.go.dev/text/template) executed on each
# [`Record`](https://pkg.go.dev/github.com/conduitio/conduit-commons/opencdc#Record)
# being processed.
# Type: string
message: "error processor triggered"
```
</TabItem>
<TabItem value="table" label="Table">
<table class="no-margin-table">
<tr>
<th>Name</th>
<th>Type</th>
<th>Default</th>
<th>Description</th>
</tr>
<tr>
<td>`message`</td>
<td>string</td>
<td>`error processor triggered`</td>
<td>
Error message to be returned. This can be a Go [template](https://pkg.go.dev/text/template)
executed on each [`Record`](https://pkg.go.dev/github.com/conduitio/conduit-commons/opencdc#Record)
being processed.
</td>
</tr>
</table>
</TabItem>
</Tabs>

## Examples

### Error record with custom error message

This example shows how to configure the error processor to
return a custom error message for a record using a Go template.

#### Configuration parameters

<Tabs groupId="config-params">
<TabItem value="yaml" label="YAML">
```yaml
version: 2.2
pipelines:
- id: example
status: running
connectors:
# define source and destination ...
processors:
- id: example
plugin: "error"
settings:
message: "custom error message with data from record: {{.Metadata.foo}}"
```
</TabItem>
<TabItem value="table" label="Table">
<table class="no-margin-table">
<tr>
<th>Name</th>
<th>Value</th>
</tr>
<tr>
<td>`message`</td>
<td>`custom error message with data from record: {{.Metadata.foo}}`</td>
</tr>
</table>
</TabItem>
</Tabs>

#### Record difference

```mdx-code-block
<Box className='diff-viewer'>
<ReactDiffViewer
styles={{
diffContainer: {
overflowX: 'auto',
overflowY: 'hidden',
},
}}
leftTitle={'Before'}
rightTitle={'After'}
oldValue={"{\n \"position\": null,\n \"operation\": \"create\",\n \"metadata\": {\n \"foo\": \"bar\"\n },\n \"key\": null,\n \"payload\": {\n \"before\": {\n \"bar\": \"baz\"\n },\n \"after\": {\n \"foo\": \"bar\"\n }\n }\n}"}
newValue={"{\n \"error\": \"custom error message with data from record: bar\"\n}"}
hideLineNumbers={false}
showDiffOnly={false}
splitView={true}
/>
</Box>
```



![scarf pixel conduit-site-docs-processors](https://static.scarf.sh/a.png?x-pxid=33ce4d9e-1948-4cab-846a-7e6cd33b8cae)
2 changes: 1 addition & 1 deletion docs/processors/builtin/field.convert.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'field.convert'
sidebar_position: 5
sidebar_position: 6
---

import ReactDiffViewer from 'react-diff-viewer';
Expand Down
2 changes: 1 addition & 1 deletion docs/processors/builtin/field.exclude.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'field.exclude'
sidebar_position: 6
sidebar_position: 7
---

import ReactDiffViewer from 'react-diff-viewer';
Expand Down
2 changes: 1 addition & 1 deletion docs/processors/builtin/field.rename.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'field.rename'
sidebar_position: 7
sidebar_position: 8
---

import ReactDiffViewer from 'react-diff-viewer';
Expand Down
2 changes: 1 addition & 1 deletion docs/processors/builtin/field.set.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'field.set'
sidebar_position: 8
sidebar_position: 9
---

import ReactDiffViewer from 'react-diff-viewer';
Expand Down
2 changes: 1 addition & 1 deletion docs/processors/builtin/filter.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'filter'
sidebar_position: 9
sidebar_position: 10
---

import ReactDiffViewer from 'react-diff-viewer';
Expand Down
2 changes: 1 addition & 1 deletion docs/processors/builtin/json.decode.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'json.decode'
sidebar_position: 10
sidebar_position: 11
---

import ReactDiffViewer from 'react-diff-viewer';
Expand Down
2 changes: 1 addition & 1 deletion docs/processors/builtin/json.encode.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'json.encode'
sidebar_position: 11
sidebar_position: 12
---

import ReactDiffViewer from 'react-diff-viewer';
Expand Down
2 changes: 1 addition & 1 deletion docs/processors/builtin/unwrap.debezium.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'unwrap.debezium'
sidebar_position: 12
sidebar_position: 13
---

import ReactDiffViewer from 'react-diff-viewer';
Expand Down
2 changes: 1 addition & 1 deletion docs/processors/builtin/unwrap.kafkaconnect.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'unwrap.kafkaconnect'
sidebar_position: 13
sidebar_position: 14
---

import ReactDiffViewer from 'react-diff-viewer';
Expand Down
2 changes: 1 addition & 1 deletion docs/processors/builtin/unwrap.opencdc.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
IMPORTANT: This file was generated using src/processorgen/main.go. DO NOT EDIT.

title: 'unwrap.opencdc'
sidebar_position: 14
sidebar_position: 15
---

import ReactDiffViewer from 'react-diff-viewer';
Expand Down
Loading