Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improve][API] Add IGNORE savemode type #7443

Merged
merged 6 commits into from
Aug 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/en/connector-v2/sink/Doris.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,8 @@ Before the synchronous task is turned on, different treatment schemes are select
Option introduction:
`RECREATE_SCHEMA` :Will create when the table does not exist, delete and rebuild when the table is saved
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode[Enum]

Expand Down
13 changes: 7 additions & 6 deletions docs/en/connector-v2/sink/Elasticsearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,17 +109,18 @@ Sink plugin common parameters, please refer to [Sink Common Options](../sink-com

Before the synchronous task is turned on, different treatment schemes are selected for the existing surface structure of the target side.
Option introduction:
RECREATE_SCHEMA :Will create when the table does not exist, delete and rebuild when the table is saved
CREATE_SCHEMA_WHEN_NOT_EXIST :Will Created when the table does not exist, skipped when the table is saved
ERROR_WHEN_SCHEMA_NOT_EXIST :Error will be reported when the table does not exist
`RECREATE_SCHEMA` :Will create when the table does not exist, delete and rebuild when the table is saved
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode

Before the synchronous task is turned on, different processing schemes are selected for data existing data on the target side.
Option introduction:
DROP_DATA: Preserve database structure and delete data
APPEND_DATA:Preserve database structure, preserve data
ERROR_WHEN_DATA_EXISTS:When there is data, an error is reported
`DROP_DATA`: Preserve database structure and delete data
`APPEND_DATA`:Preserve database structure, preserve data
`ERROR_WHEN_DATA_EXISTS`:When there is data, an error is reported

## Examples

Expand Down
3 changes: 2 additions & 1 deletion docs/en/connector-v2/sink/Jdbc.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,8 @@ Before the synchronous task is turned on, different treatment schemes are select
Option introduction:
`RECREATE_SCHEMA` :Will create when the table does not exist, delete and rebuild when the table is saved
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode [Enum]

Expand Down
1 change: 1 addition & 0 deletions docs/en/connector-v2/sink/LocalFile.md
Original file line number Diff line number Diff line change
Expand Up @@ -214,6 +214,7 @@ Existing dir processing method.
- RECREATE_SCHEMA: will create when the dir does not exist, delete and recreate when the dir is exist
- CREATE_SCHEMA_WHEN_NOT_EXIST: will create when the dir does not exist, skipped when the dir is exist
- ERROR_WHEN_SCHEMA_NOT_EXIST: error will be reported when the dir does not exist
- IGNORE :Ignore the treatment of the table

### data_save_mode [string]

Expand Down
3 changes: 2 additions & 1 deletion docs/en/connector-v2/sink/PostgreSql.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,8 @@ Before the synchronous task is turned on, different treatment schemes are select
Option introduction:
`RECREATE_SCHEMA` :Will create when the table does not exist, delete and rebuild when the table is saved
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode[Enum]

Expand Down
3 changes: 2 additions & 1 deletion docs/en/connector-v2/sink/S3File.md
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,8 @@ Before turning on the synchronous task, do different treatment of the target pat
Option introduction:
`RECREATE_SCHEMA` :Will be created when the path does not exist. If the path already exists, delete the path and recreate it.
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the path does not exist, use the path when the path is existed.
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the path does not exist
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the path does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode[Enum]

Expand Down
3 changes: 2 additions & 1 deletion docs/en/connector-v2/sink/StarRocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,8 @@ Before the synchronous task is turned on, different treatment schemes are select
Option introduction:
`RECREATE_SCHEMA` :Will create when the table does not exist, delete and rebuild when the table is saved
`CREATE_SCHEMA_WHEN_NOT_EXIST` :Will Created when the table does not exist, skipped when the table is saved
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`ERROR_WHEN_SCHEMA_NOT_EXIST` :Error will be reported when the table does not exist
`IGNORE` :Ignore the treatment of the table

### data_save_mode[Enum]

Expand Down
15 changes: 8 additions & 7 deletions docs/zh/connector-v2/sink/Doris.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,18 +53,19 @@ Doris Sink连接器的内部实现是通过stream load批量缓存和导入的
### schema_save_mode[Enum]

在开启同步任务之前,针对现有的表结构选择不同的处理方案。
选项介绍:
选项介绍:
`RECREATE_SCHEMA` :表不存在时创建,表保存时删除并重建。
`CREATE_SCHEMA_WHEN_NOT_EXIST` :表不存在时会创建,表存在时跳过。
`ERROR_WHEN_SCHEMA_NOT_EXIST` :表不存在时会报错。
`CREATE_SCHEMA_WHEN_NOT_EXIST` :表不存在时会创建,表存在时跳过。
`ERROR_WHEN_SCHEMA_NOT_EXIST` :表不存在时会报错。
`IGNORE` :忽略对表的处理。

### data_save_mode[Enum]

在开启同步任务之前,针对目标端已有的数据选择不同的处理方案。
选项介绍:
`DROP_DATA`: 保留数据库结构并删除数据。
`APPEND_DATA`:保留数据库结构,保留数据。
`CUSTOM_PROCESSING`:用户自定义处理。
选项介绍:
`DROP_DATA`: 保留数据库结构并删除数据。
`APPEND_DATA`:保留数据库结构,保留数据。
`CUSTOM_PROCESSING`:用户自定义处理。
`ERROR_WHEN_DATA_EXISTS`:有数据时报错。

### save_mode_create_template
Expand Down
1 change: 1 addition & 0 deletions docs/zh/connector-v2/sink/Elasticsearch.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ Sink插件常用参数,请参考 [Sink常用选项](../sink-common-options.md)
`RECREATE_SCHEMA` :当表不存在时会创建,当表已存在时会删除并重建<br/>
`CREATE_SCHEMA_WHEN_NOT_EXIST` :当表不存在时会创建,当表已存在时则跳过创建<br/>
`ERROR_WHEN_SCHEMA_NOT_EXIST` :当表不存在时将抛出错误<br/>
`IGNORE` :忽略对表的处理<br/>

### data_save_mode

Expand Down
1 change: 1 addition & 0 deletions docs/zh/connector-v2/sink/Jdbc.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,7 @@ Sink插件常用参数,请参考 [Sink常用选项](../sink-common-options.md)
`RECREATE_SCHEMA`:当表不存在时会创建,当表已存在时会删除并重建<br/>
`CREATE_SCHEMA_WHEN_NOT_EXIST`:当表不存在时会创建,当表已存在时则跳过创建<br/>
`ERROR_WHEN_SCHEMA_NOT_EXIST`:当表不存在时将抛出错误<br/>
`IGNORE` :忽略对表的处理<br/>

### data_save_mode [Enum]

Expand Down
9 changes: 5 additions & 4 deletions docs/zh/connector-v2/sink/StarRocks.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,10 +99,11 @@ table选项参数可以填入一任意表名,这个名字最终会被用作目

### schema_save_mode[Enum]

在同步任务打开之前,针对目标端已存在的表结构选择不同的处理方法。可选值有:
`RECREATE_SCHEMA` :不存在的表会直接创建,已存在的表会删除并根据参数重新创建
`CREATE_SCHEMA_WHEN_NOT_EXIST` :忽略已存在的表,不存在的表会直接创建
`ERROR_WHEN_SCHEMA_NOT_EXIST` :当有不存在的表时会直接报错
在同步任务打开之前,针对目标端已存在的表结构选择不同的处理方法。可选值有:
`RECREATE_SCHEMA` :不存在的表会直接创建,已存在的表会删除并根据参数重新创建
`CREATE_SCHEMA_WHEN_NOT_EXIST` :忽略已存在的表,不存在的表会直接创建
`ERROR_WHEN_SCHEMA_NOT_EXIST` :当有不存在的表时会直接报错
`IGNORE` :忽略对表的处理

### data_save_mode[Enum]

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ public void handleSchemaSaveMode() {
case ERROR_WHEN_SCHEMA_NOT_EXIST:
errorWhenSchemaNotExist();
break;
case IGNORE:
break;
default:
throw new UnsupportedOperationException("Unsupported save mode: " + schemaSaveMode);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,7 @@ public enum SchemaSaveMode {

// Error will be reported when the table does not exist
ERROR_WHEN_SCHEMA_NOT_EXIST,

// Ignore creation
IGNORE
}
Loading