Megular Expressions

[MarcoPolo]

A very simple regular expression matcher for Multiaddr components. Supports capturing values. Matches in linear time (no back tracking).

The core logic is about 100 LOC. The sugar to make it nicer to use is about another 100 LOC.

Motivation

If we are going to treat Multiaddrs as encoding, then we need to make it more ergonomic to parse multiaddrs. Right now we have a lot of somewhat wrong manual parsers using ForEach. This should be able to replace those, make it cleaner, and most importantly make them obviously correct.

In draft while I try this out in go-libp2p.

Example

Parsing a WebTransport Multiaddr m.

var dnsName string
var ip4Addr string
var ip6Addr string
var udpPort string
var certHashesStr []string
matched, err := m.Match(
  meg.Or(
    meg.CaptureVal(ma.P_IP4, &ip4Addr),
    meg.CaptureVal(ma.P_IP6, &ip6Addr),
    meg.CaptureVal(ma.P_DNS4, &dnsName),
    meg.CaptureVal(ma.P_DNS6, &dnsName),
    meg.CaptureVal(ma.P_DNS, &dnsName),
  ),
  meg.CaptureVal(ma.P_UDP, &udpPort),
  meg.Val(ma.P_QUIC_V1),
  meg.Optional(
    meg.CaptureVal(ma.P_SNI, &wtAddr.sni),
  ),
  meg.Val(ma.P_WEBTRANSPORT),
  meg.CaptureZeroOrMore(ma.P_CERTHASH, &certHashesStr),
)
if err != nil {
  return webtransportAddr{}, err
}
if !matched {
  return webtransportAddr{}, errNotQUICAddr
}

[sukunrt]

This looks very useful to me modulo comments. See how nice the IsWebRTCDirectMultiaddr method is now vs compared to master.

I think we should make this package Experimental / Alpha and start using this in go-libp2p. After some experience we can remove the Experimental tag, till then we can keep iterating on a satisfactory api.

[sukunrt]

I like the API. Some other things I'd like are:

Any which matches any protocol

Match the pattern somewhere in the middle, not necessarily at the start.

Any()* which will match everything from that point onwards, so we can match IP, TCP, Port, Rest,

[MarcoPolo]

We have any now. Here's an example that captures a ipport and any*

found, err := m.Match(
    CaptureAddrPort(&network, &addrPort),
    meg.ZeroOrMore(meg.Any),
)

[sukunrt]

I think it's more powerful to capture the entire Component and not just Value. That way for some uses you don't have to move back from the string from to the component form.

[MarcoPolo]

This is done in #269

[sukunrt]

Is there a reason to make this a method as opposed to:

MatchPattern(m Multiaddr, p ...meg.Pattern)

[MarcoPolo]

No reason. I think it's a bit more ergonomic for it to be a method, but I don't have a strong preference

[sukunrt]

I expect most users to not want to run this everytime. s := meg.PatternToMatchState(p...)

[MarcoPolo]

I don't follow. They would just do:

var m Multiaddr
// ...
m.Match(<some pattern>)

No use of meg.PatternToMatchState directly

[sukunrt]

Do we want an Optional or do we want a MatchIfExists? By MatchIfExists I mean if the component is there we always want to match it. While an Optional if I understand correctly, may be ignored if it is a prefix of the following match components.This might simplify the implementation.

[MarcoPolo]

What's the regex equivalent of MatchIfExists?

[sukunrt]

Not sure if there is. I think the expectation is to rewrite such regexs.

Implementation wise, if the next character doesn't match we still move the state ahead without moving the character index ahead.

[MarcoPolo]

Implementation wise, if the next character doesn't match we still move the state ahead without moving the character index ahead.

This sounds a bit like Optional.

Overall, I think we do want Optional. For example, SNI is an optional field in many multiaddrs

[sukunrt]

Just one unaddresses comment:
#263 (comment)