Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate most enginetests for new name resolution #1869

Merged
merged 44 commits into from
Jul 28, 2023
Merged

Conversation

max-hoffman
Copy link
Contributor

@max-hoffman max-hoffman commented Jul 7, 2023

This doubles most of the enginetests to add versions with new name resolution. As a result testing takes ~2x as long, temporarily. Gets majority of those enginetests working with a couple bigger exceptions:

  • plans projections are subtly different in a way that should be optimized but is probably not priority
  • stored procedures need their custom resolution logic ported
  • on duplicate update expressions are buggy, going to rewrite those for new format
  • skipping one derived table alias test, where we do not have expression memoization or lateral joins to help us execute a resolved plan (related Alias references in subquery expressions dolt#6407)
  • many tests throw "column not found" instead of "table not found" errors. I tried to bookkeep those with Skips, but the skipped suites may accumulate other differences in the meantime.
  • I'll need to revert our prepared statement logic before the final switch
  • Various validators work a bit differently, might end up skipping some error tests to get the final switch in sooner

Other suites:

  • TestJSONTableScripts_Experimental -- json_table still broken
  • TestRenameColumn_Exp -- error test has different error

A couple other discoveries:

  • We have to hallucinate unknown table, column, and procedure names in trigger bodies on CREATE, and only fail on execution
  • Column default expressions appear to be resolved at execution time
  • Alter statements are only resolved at execution time
  • The precedence for ASOF in nested views and procedure calls is a bit hairy
  • json_array w/ and w/o distinct appears to be untested
  • ORDER BY in UNION seems pretty flaky and lightly tested, we resolve names from the left

Dolt PR: dolthub/dolt#6414

@max-hoffman max-hoffman marked this pull request as ready for review July 28, 2023 15:15
@max-hoffman max-hoffman requested a review from zachmu July 28, 2023 15:54
@max-hoffman max-hoffman changed the title More name resolution enginetests Migrate most enginetests for new name resolution Jul 28, 2023
Copy link
Member

@zachmu zachmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good, although I can't promise I thoroughly reviewed all the new parts

a.aliases[alias.Name()] = alias.Child
}

func (a *aliasScope) isOuterRef(name string) (bool, sql.Expression) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment about lower casing (or use method doc to make clear args need to be lower cased before calling this)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we let aliases play as they lie, after name resolution. We have a few tests that hit that case.


l.Destination = children[0]
return l, nil
ret := *l
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You an avoid this rigmarole by declaring the receiver to a non-pointer type

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the bugs i run into are still usually because we were using a value instead of pointer type, if we overcorrect at some point we can always mass codegen all nodes as value types

if len(exprs) != 1 {
return nil, sql.ErrInvalidChildrenNumber.New(sv, len(exprs), 1)
}
ret := *sv
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment about receiver type

@@ -42,6 +41,8 @@ type scope struct {
windowDefs map[string]*sql.WindowDefinition
// exprs collects unique expression ids for reference
exprs map[string]columnId
// subqueries is a workaround for execution limitations
subqueries map[string]scopeColumn
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Name is not great, unclear what a field called subqueries with this type could be. Missing a noun or two.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deleting, this was from a discarded prototype

// getExpr returns a columnId if the given expression has
// been built.
func (s *scope) getExpr(name string) (columnId, bool) {
func (s *scope) getExpr(name string, checkCte bool) (columnId, bool) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably better to have a getExprInCte method than add this extra param

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There can be CTEs in higher scopes whose naming precedence is interleaved

@max-hoffman max-hoffman merged commit 9b9301f into main Jul 28, 2023
@Hydrocharged Hydrocharged deleted the max/name-res-3 branch August 24, 2023 00:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants