API for gogrep matching engine

[benjaminjkraft]

In writing linters (in my case, go/analysis passes to be run as golangci-lint plugins) I find myself wanting some sort of AST-matching engine. This tool seems like a really great one! But I want to use it inside my own tool, rather than as a simple global search, like this package or go-ruleguard is designed to do. To do that, I need to use the gogrep matcher, but then refer back to the AST or types in an arbitrary way.

A sample API that I think would be sufficient, and which looks to my quick glance similar to what you're already using internally:

type Matcher

// Compile compiles a gogrep pattern into a Matcher object.
func Compile(pattern string) (*Matcher, error)

// FindAll returns all matches to the given pattern within the given AST node.
func (m *Matcher) FindAll(node ast.Node) []Match

// A single match of a pattern.
type Match struct {
  // The top-level node that matched the pattern.
  Node ast.Node
  // The nodes that matched each variable; for example if $x + $_ matched 2 + 3,
  // captures would be {"$x": <node for 2>}
  Captures map[string]ast.Node
}

// optionally other regexp-style APIs like MustCompile, Matcher.Find, Matcher.Match.

Related to #32, but it seems like that won't actually do what we would want, for the same reason go-ruleguard isn't quite enough for us: we would still have no direct access to the underlying AST/types.

[benjaminjkraft]

As an example usage, I have a linter that wants to flag cases where methods of certain types don't call out to their "super" -- that is, where you have a declaration

func (recv *SomeType) SomeMethod(...) ...

which does not call

recv.<something>.SomeMethod(...)

So one part of that is to look for expressions of the form $x.$y.SomeMethod(...), but then we want to get back $x so we can check if it's a reverence to recv (easy, using go/types); and more importantly we only want to do the entire search within some function-declaration. Ideally, we might even vary look for a specific $y depending on the type we're looking at. (Right now we're just doing manual ast traversal -- call, ok := node.(*ast.CallExpr); if !ok { return false } and repeat -- and it's kind of a mess, which is why I'm looking at gogrep.)

The former constraint we could do with some loss of fidelity (due to shadowing) with gogrep alone, something like func ($x $_) $y($*_) $*_ { $*_; $x.$_.$y($*_); $*_ }. But the latter two we really just need types to figure out. I guess in principle we could call out to gogrep to search for each func ($x <specific type>) <specific method-name>($*_) ..., but that seems at best very slow. Of course, we could ask for a way to search " within ", but that feels like a kludge on both sides: you have to add a bunch of syntax, and we are still very constrained in what we can do.

[mvdan]

Sorry that I never replied here. To be honest, I've gotten tired of working on the gogrep project for now, and I don't know when I'll actively get back to it. I assume you already saw the refactor branch linked in #32; that was my latest thinking when it comes to a reusable API.

If you want anything other than that, I would suggest to fork or copy code, as that will be the fastest and simplest way. I will probably resurrect/redesign this project at some point in the future, but it's not going to be in the immediate future.

[benjaminjkraft]

Totally understandable and appreciate the guidance! I'll post here if I end up forking.

[ole108]

@benjaminjkraft I would love to have an API, too.
We might be able to work together on it.