Can I get the original text of a parser node?

[h3n4l]

package main

import (
	"fmt"

	"github.com/pingcap/tidb/pkg/parser"
	"github.com/pingcap/tidb/pkg/parser/ast"
	_ "github.com/pingcap/tidb/pkg/parser/test_driver"
)

func main() {
	statement := `
CREATE VIEW v1 AS SELECT * FROM t;
CREATE VIEW v2 AS SELECT 123123123123123;
`

	p := parser.New()

	// To support MySQL8 window function syntax.
	// See https://github.com/bytebase/bytebase/issues/175.
	p.EnableWindowFunc(true)

	nodes, _, err := p.Parse(statement, "", "")
	if err != nil {
		panic(err)
	}

	for _, node := range nodes {
		if node, ok := node.(*ast.CreateViewStmt); ok {
			viewName := node.ViewName.Name.O
			selectText := node.Select.Text()
			fmt.Printf("%s: %s\n\n\n\n", viewName, selectText)
		}
	}
}

I get the output:

v1: SELECT * FROM t;
CREATE VIEW v2 AS SELECT 123123123123123;



v2: SELECT 123123123123123;

IMO, the original text should return SELECT * FROM t; of v1.

[Defined2014]

Try use Restore function like code below.

func TestA(t *testing.T) {
	statement := `
CREATE VIEW v1 AS SELECT * FROM t;
CREATE VIEW v2 AS SELECT 123123123123123;
`

	p := parser.New()

	// To support MySQL8 window function syntax.
	// See https://github.com/bytebase/bytebase/issues/175.
	p.EnableWindowFunc(true)

	nodes, _, err := p.Parse(statement, "", "")
	if err != nil {
		panic(err)
	}

	var sb strings.Builder
	for _, node := range nodes {
		if node, ok := node.(*ast.CreateViewStmt); ok {
			sb.Reset()
			viewName := node.ViewName.Name.O
			// selectText := node.Select.Text()
			node.Select.Restore(format.NewRestoreCtx(format.DefaultRestoreFlags, &sb))
			fmt.Printf("%s: %s\n\n\n\n", viewName, sb.String())
		}
	}
}

[h3n4l]

Hi @Defined2014 , thanks for your response. These are two requirements, when I need to get normalized SQL, I should use Restore, and I do. But when I need to get the original input for this statement from the input stream, I need OriginalText().

[Defined2014]

Ok, I think we didn't call SetText for every nodes correctly, so maybe we need support it later.

[h3n4l]

Related docs:

tidb/docs/design/2018-11-29-ast-to-sql-text.md

Lines 11 to 25 in f272707

	## Background
	Some new features in TiDB need all the `ast.Node` to be restorable to SQL text.
	For example, we parse the `create view v as select * from t;` SQL statement to an AST,
	and then expand `select` of the AST to `select test.t.col0, test.t.col1 from test.t;`.
	The structure of an `ast.Node` is shown in the following picture (taking `ast.CreateUserStmt` as an example).
	
	We know there is a `Text()` method of `ast.Node`,
	in which the parser calls the `SetText()` method during the parsing process,
	and then we can call `Text()` to get the original input SQL statement.
	But the `Text()` method is incomplete, and only `Text()` of root nodes can work well.
	We should not use it when implementing this proposal.