sink(ticdc): DDL event support for the Debezium protocol

[wk989898]

What problem does this PR solve?

Issue Number: close #11566

What is changed and how it works?

Debezium provides the SQL CREATE, ALTER, or DROP statement that results in the schema change. https://debezium.io/documentation/reference/2.7/connectors/mysql.html#mysql-schema-change-topic
This PR implements encoding for the DDL event and checkpoint event.

Difference between TiCDC and Debezium:

• Default length display
  • Debezium output is different when setting the default length. e.g. int length is nil but int(11) length is 11
• Unsupported data type
  • TiCDC converts DECIMAL to NUMERIC
  • TiCDC converts INTEGER to INT
  • TiCDC converts REAL to FLOAT when setting sql_mode='REAL_AS_FLOAT'
  • TiCDC converts BOOLEAN to TINYINT(1)
• Generated column
  • TiCDC does't output the virtual generated column. generate column doesn't show in DML event #11704
• TiCDC does not support some DDLs
  • Alter table add/drop foreign key
  • Adding a new column and setting it to the primary key
  • Dropping primary key columns or columns covered by the composite index
• Debezium does not support some DDLs
  • Alter/Remove TTL
  • Recover table
• Default value
  • In TiCDC, BLOB, TEXT, GEOMETRY, or JSON column 'binaryRepresentation' can't have a default value
  • BIT defaultValueExpression may not same when the length of the default value is not equal to the column length
• Incorrect FLOAT data output
  • Debezium value is incorrect. e.g. The value is "5.61", but Debezium output value is "5.610000133514404"
  • TiCDC incorrect length output. print wrong flen with the FLOAT tidb#57060
• Incorrect charset name
  • Debezium converts charsetName to "utf8mb4" when column COLLATE is "utf8_unicode_ci" and CHARACTER is null, but TiCDC doesn't
• Incorrect enum elements
  • Debezium doesn't escape character, but TiCDC does. e.g. ENUM('c,'d', 'g,''h')
• Incorrect default about TIME
  • TiCDC converts TIME default '1000-00-00 01:00:00.000' to "1000-00-00", but Debezium doesn't
  • Debezium produces the wrong result when the insert value is illegal like "-838:59:59.000000"

Check List

Tests

• Unit test
• Integration test

Questions

Will it cause performance regression or break compatibility?

Do you need to update user documentation, design documentation or monitoring documentation?

yes

Release note

DDL event support for the Debezium protocol

[wk989898]

/ok-to-test

[wk989898]

/retest

[hongyunyan]

LGTM

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CharlesCheung96, hongyunyan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [CharlesCheung96,hongyunyan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-11-12 01:40:00.695659864 +0000 UTC m=+320362.886528856: ☑️ agreed by hongyunyan.
• 2024-11-12 10:40:09.137144264 +0000 UTC m=+352771.328013260: ☑️ agreed by CharlesCheung96.