parser: support OPTIMIZE statement

[ub-3]

What problem does this PR solve?

Now it is possible to parse SQL with OPTIMIZE statement.

OPTIMIZE [NO_WRITE_TO_BINLOG | LOCAL]
    TABLE tbl_name [, tbl_name] ...

Issue Number: close #49204

Problem Summary: There is no support for the OPTIMIZE statement

What changed and how does it work?

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No need to test

  • I checked and no code files have been changed.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

Added support for the "OPTIMIZE" statement.

[sre-bot]

All committers have signed the CLA.

[ti-chi-bot]

Welcome @ub-3!

It looks like this is your first PR to pingcap/tidb 🎉.

I'm the bot to help you request reviewers, add labels and more, See available commands.

We want to make sure your contribution gets all the attention it needs!



Thank you, and welcome to pingcap/tidb. 😃

[ti-chi-bot]

Hi @ub-3. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tiprow]

Hi @ub-3. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ti-chi-bot]

@ub-3:
I Skip it since the diff size(10765509 bytes > 80000 bytes) is too large

In response to this:

/review default

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[codecov]

Codecov Report

Merging #49205 (3795fd4) into master (8dac756) will increase coverage by 0.9983%.
The diff coverage is 0.0000%.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #49205        +/-   ##
================================================
+ Coverage   71.0504%   72.0487%   +0.9983%     
================================================
  Files          1368       1406        +38     
  Lines        402168     413226     +11058     
================================================
+ Hits         285742     297724     +11982     
- Misses        96553      96714       +161     
+ Partials      19873      18788      -1085     

Flag	Coverage Δ
integration	43.6759% <0.0000%> (?)
unit	71.0504% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	53.9663% <ø> (ø)
parser	∅ <ø> (∅)
br	48.2512% <ø> (-4.6804%)	⬇️

[bb7133]

It would be confusing to TiDB users when 'OPTIMIZE TABLE' is accepted but does nothing. Do you have any specific requirement to have this supported in syntax only?

[lance6716]

It would be confusing to TiDB users when 'OPTIMIZE TABLE' is accepted but does nothing. Do you have any specific requirement to have this supported in syntax only?

I agree that since this PR changes the behaviour for user, we need to discuss what's the expected behaviour for TiDB of OPTIMIZE statement. For example, should we invoke TiKV's functions? Don't want to change it back and forth in future.

Maybe If you only care about the parser's golang API, you can still let TiDB report error for OPTIMIZE.

[ub-3]

@bb7133 To be honest, I'm only interested in parser support. I should read MySQL slow-logs and analyzes queries, and your package is just what I need. I'm not so familiar with the project, but it seems to me, given that the parser used to be in a separate project, adding this support is not such an issue.

[ub-3]

@lance6716

I agree that since this PR changes the behaviour for user, we need to discuss what's the expected behaviour for TiDB of OPTIMIZE statement. For example, should we invoke TiKV's functions? Don't want to change it back and forth in future.
Maybe If you only care about the parser's golang API, you can still let TiDB report error for OPTIMIZE.

Can you explain what it means? I won't like to add edits that break the API, is there an example of how this can be avoided?

[mjonss]

LGTM, but probably something is needed in TiDB code to either implement it or to block/fail it. With current patch it will just say OK, without any checks for table etc.

tidb> drop table if exists t;
Query OK, 0 rows affected, 1 warning (0.00 sec)

tidb> show warnings;
+-------+------+------------------------+
| Level | Code | Message                |
+-------+------+------------------------+
| Note  | 1051 | Unknown table 'test.t' |
+-------+------+------------------------+
1 row in set (0.00 sec)

tidb> optimize table t;
Query OK, 0 rows affected (0.01 sec)

[ub-3]

@mjonss What about this kind of message ?

mysql> optimize table t;
ERROR 1235 (42000): This version of TiDB doesn't yet support 'OPTIMIZE'

This can be done by handling the statement as follows.

pkg/planner/core/planbuilder.go:819
...
	case *ast.OptimizeTableStmt:
		return nil, ErrNotSupportedYet.GenWithStackByArgs("OPTIMIZE")
...

[mjonss]

@mjonss What about this kind of message ?

mysql> optimize table t;
ERROR 1235 (42000): This version of TiDB doesn't yet support 'OPTIMIZE'

This can be done by handling the statement as follows.

pkg/planner/core/planbuilder.go:819
...
	case *ast.OptimizeTableStmt:
		return nil, ErrNotSupportedYet.GenWithStackByArgs("OPTIMIZE")
...

Yes, something like that. Since we moved the parser into the TiDB repository, we also need to update the rest of TiDB code to handle new changes in the parser (which is the normal case).

I also think that generating the error in buildDDL() is better, as well as having a test case for the TiDB server, like this:

diff --git a/pkg/ddl/db_integration_test.go b/pkg/ddl/db_integration_test.go
index 672d04df30..66cd9e9f9c 100644
--- a/pkg/ddl/db_integration_test.go
+++ b/pkg/ddl/db_integration_test.go
@@ -2955,3 +2955,9 @@ func TestDefaultCollationForUTF8MB4(t *testing.T) {
        tk.MustQuery("show create database dby").Check(testkit.Rows(
                "dby CREATE DATABASE `dby` /*!40100 DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci */"))
 }
+
+func TestOptimizeTable(t *testing.T) {
+       store := testkit.CreateMockStore(t, mockstore.WithDDLChecker())
+       tk := testkit.NewTestKit(t, store)
+       tk.MustGetErrMsg("optimize table t", "[ddl:8200]OPTIMIZE TABLE is not supported")
+}
diff --git a/pkg/planner/core/planbuilder.go b/pkg/planner/core/planbuilder.go
index febc2dcfb9..3064841f2e 100644
--- a/pkg/planner/core/planbuilder.go
+++ b/pkg/planner/core/planbuilder.go
@@ -5147,6 +5147,8 @@ func (b *PlanBuilder) buildDDL(ctx context.Context, node ast.DDLNode) (Plan, err
        case *ast.CreateResourceGroupStmt, *ast.DropResourceGroupStmt, *ast.AlterResourceGroupStmt:
                err := ErrSpecificAccessDenied.GenWithStackByArgs("SUPER or RESOURCE_GROUP_ADMIN")
                b.visitInfo = appendDynamicVisitInfo(b.visitInfo, "RESOURCE_GROUP_ADMIN", false, err)
+       case *ast.OptimizeTableStmt:
+               return nil, dbterror.ErrGeneralUnsupportedDDL.GenWithStack("OPTIMIZE TABLE is not supported")
        }
        p := &DDL{Statement: node}
        return p, nil

[ub-3]

@mjonss Thanks for the good explanation!

[Defined2014]

LGTM
Need update a private repo to pass mysql-tester, which I have done. https://github.com/PingCAP-QE/tidb-test/pull/2261

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2023-12-07 04:39:18.851884098 +0000 UTC m=+1675187.517110277: ☑️ agreed by Defined2014.
• 2023-12-07 11:39:13.539458196 +0000 UTC m=+1700382.204684391: ☑️ agreed by mjonss.

[mjonss]

I holding this PR, so we can merge it with the matching PR for passing the mysql-tester CI test.

[mjonss]

@yudongusa or @easonn7 can you approve this PR, it adds OPTIMIZE TABLE to the parser, and also blocks it in the TiDB code, so the statement will show 'OPTIMIZE TABLE is not supported' instead of a parser error.

[dveeden]

This looks good. However it should have set some of the items in the description as this affects mysql compatibility, user behavior and syntax (all of them in a positive way).

[easonn7]

/approve

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Defined2014, easonn7, mjonss, tangenta

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [Defined2014,mjonss,tangenta]
• pkg/ddl/OWNERS [tangenta]
• pkg/parser/OWNERS [easonn7,tangenta]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[tangenta]

/retest

[Defined2014]

/test mysql-test

[tiprow]

@Defined2014: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

• /test tiprow_fast_test

Use /test all to run all jobs.

In response to this:

/test mysql-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Defined2014]

/test mysql-test

[tiprow]

@Defined2014: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

• /test tiprow_fast_test

Use /test all to run all jobs.

In response to this:

/test mysql-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Defined2014]

/test mysql-test

[tiprow]

@Defined2014: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

• /test tiprow_fast_test

Use /test all to run all jobs.

In response to this:

/test mysql-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mjonss]

/retest

[mjonss]

/retest