New ore filter implementation

[Tictim]

What

This PR replaces previous ore dictionary filter system with a brand new implementation of ore filter expression parser and interpreter. This PR features a newly designed language based on previous ore filter expression, with a cheeky name "oreglob". (ore + glob)

Implementation Details

Language Specification

Grammar of the lauguage in EBNF is as follows.

oreGlob = [ FLAG ], [ or ], EOF

or  = and, { '|', { '|' }, and }
and = xor, { '&', { '&' }, xor }
xor = not, { '^', not }

not = '!', { '!' }, '(', [ or ], [ ')', [ not ] ]
    | { '!' }, primary, [ not ]

primary = LITERAL
        | '(', [ or ], [ ')' ]
        | ( '*' | '?' ), { '*' | '?' }

FLAG = '$', CHARACTER - WHITESPACE, { CHARACTER - WHITESPACE }
LITERAL = CHARACTER - WHITESPACE, { CHARACTER - WHITESPACE }

WHITESPACE = ' ' | '\t' | '\n' | '\r'

CHARACTER = ? every character with codepoint of <= 0xFFFF ?

This grammar set is very similar to previous ore filter expression, for example:

• | denotes logical OR.
• & denotes logical AND.
• ^ denotes logical XOR.
• ! denotes logical negation.
• * denotes wildcard; i.e. 0 or more characters.

A number of changes and additions are also present, namely:

• Optional enclosing parenthesis and lenient checking of redundant operators is now a part of the language spec.
• The binary operators have precedence; for example, in a | b & c and a & b | c, logical AND is computed first in both scenario.
• ! has an additional rule to prevent possible case of confusion.
  • If two or more matches follow ! token, it will be grouped as one match before negation; for example, !*plate* is equivalent to !(*plate*).
  • If ( directly follows after !, the following matches after enclosing parenthesis are not grouped. For example, !(ore)Gold does not mean !(oreGold); it is a syntactic sugar to treat negation on a group as same as negation inside a group. (!(a) == (!a))
• New keyword, ?, matches any one character. Using this, if you want input with 1 or more characters, then you could write this: ?*
• Empty group () denotes nothing, i.e. zero characters. This is logically equivalent to !(?*).
• Escape character (\) allows keyword characters to be included in literals.

Outside of grammars, the language contains some behavior changes, including:

• OreGlob is, by default, case insensitive. This means that the expression ingotiron can match ingotIron and so on. Users can opt-out from this and make the match case sensitive again, by inserting $c (compilation flag) at the start of expression.
• Though not in the form of language spec, ore filter are now able to match items with no ore dictionary if it can match empty string. This change has a possiblity of breaking certain setups, for example setups with * as expression will start to match items with no oredict.

A simple parser, OreGlobParser, was created using top-down approach.

Interpreter Design

The interpreter, which inspired by this article, uses set of states to track the match result. Match is considered as success if any of the final output state is equal to input string's length, indicating the state chewed through entire string while evaluating each OreGlob nodes.

More detailed explanation is written on NodeInterpreter.

User Convenience Feature.

The goal of OreGlob is not just to clean up quirks of old implementation, but generally provide much nicer usability for users including those without extensive knowledge on programming languages. One way of achieving such goal is literally telling players the compiled result of expressions and errors.

The compiler may output error and warnings along with OreGlob instance, which can be displayed by text. The OreGlob instance itself can be translated to formatted string, which looks like below.

Input: dust*Gold | (plate* & !*Double*)
Output: 
one of...
> 'dust'
  ... followed by anything
  ... followed by 'Gold'
> or anything that is...
  > 'plate'
    ... followed by anything
  > and not:
      anything
      ... followed by 'Double'
      ... followed by anything

API layer

I have tried to hide implementation detail behind abstraction layer as much as possible, to account for future rewrite potential. For now, only 7 classes are defined in API package, with 3 of them being inner classes.

UI changes

Ore filter UI has been redesigned to accommodate new features available with OreGlob, as well as to provide better quality-of-life features.

Potential Compatibility Issues

As the new language spec contains numerous behavior changes, it is possible that some of the setups will behave differently. Users should be notified about this potential issue before updating.

[LAGIdiot]

Good work on this PR! Just now I am testing it so I will leave few comments on what I think needs addressing.

[LAGIdiot]

When there is no string in text field there is no indication what will happened.

It would be good to have something like question mark on slot top right corner. And 3 dot slot could mention something like expression is empty nothing will be filtered.

[LAGIdiot]

Thanks for updating this PR! I feel that now it is completed. Implementation looks reasonable and new behavior is great. From ingame test no problem with upgrading old filters found and adding new filters now is way easier.