Question about Many parser

[mackoj]

Hi,

Should Many split a given input with it terminator parser then with it separator parser and give the rest to its element parser to process?

In a complex parser with multiple layers of Many it's hard to predict how it will work because it's the first Many that don't have Many in its element parser that will go through all the input.

In this naive approach to parse the input, I try to understand how Many behave.

import Foundation
import Parsing

let input = """
a, b, c, d
e, f, g, h
"""

let parserLine = Many {
  Rest()
} separator: {
  ","
} terminator: {
  Newline()
}

let fullFile = Many {
  parserLine
} separator: {
  Newline()
} terminator: {
  End()
}

do {
  print(try fullFile.parse(input[...]))
} catch {
  print(error)
}

Output

error: multiple failures occurred

error: unexpected input
 --> input:2:11
2 | e, f, g, h
  |           ^ expected ","
  |           ^ expected "\n" or "\r\n"

error: unexpected input
 --> input:1:1
1 | a, b, c, d
  | ^ expected end of input

[randomeizer]

Part of your problem here is using Rest() as the main parser. It will consume all characters, including the commas and newlines. Many doesn't 'split' based on the separator, then parse the chunks. It parses from beginning to end, first processing the main parser, then checking to see if the separator is next, and if not, the terminator, and if not that, then it loops back again.

The other issue you'll find here is that the terminator actually consumes the terminator, so it won't be available for the fullLine parser to use as the separator. The good thing is, if you write it correctly, you don't need it - the separator is actually optional.

Try something like this instead:

import Parsing

let input = """
a, b, c, d
e, f, g, h

"""

let parserLine = Many {
  Prefix { $0 != "," && !$0.isNewline }
} separator: {
  ","
} terminator: {
  Newline()
}

let fullFile = Many {
  parserLine
} terminator: {
  End()
}

do {
  print(try fullFile.parse(input[...]))
} catch {
  print(error)
}

You'll note that I've added an extra blank line at the end of the input text. You need this because the parserLine requires a Newline() to be present on every line, so the last line must be blank, basically.