Installing

You can get the latest release from pip:

$ pip install comber

or from the source:

$ pip install .

from comber import C, cs

addition = value + cs('+-') + value
assignment = C+ 'let' + keyword + '=' + (addition | function_call | value)

The C Parser

The C parser, on its own, consumes no text, produces no tokens, and always succeeds. It’s most useful for starting a parser that would otherwise begin with a string literal. E.g. this:

'let' + keyword + '=' + (function_call | value)

would actually throw a Python error because ‘let’ isn’t really a parser - yet! That’s where C comes in:

C+ 'let' + keyword + '=' + (function_call | value)

C starts off the sequence, so we can use any combination of parsers and string literals we like from there. It works similarly with alternatives, so if we wanted to allow set to be used as a synonym for let, we might do:

(C| 'let'|'set') + keyword + '=' + (function_call | value)

C can also be used to wrap a parser to protect it from optimization; for instance, embedding one sequence or alternative set inside another. If, for instance, we extended our grammar to allow a bare value to be a whole statement:

value = (keyword | number)@'value'
grammar = (import_statement | assignment | function_call | value)[0, inf]

Basic Combinators

Parsers can be combined in series with +:

name + address + pet

A sequence of parsers is evaluated left to right, each consuming text before the next is evaluated. If at any point in the sequence a parser fails, the entire sequence fails.

A set of alternatives is built with |:

name | idnumber | location

Alternatives are considered left to right, with the first successful match being the match for the entire set. Be careful! This means that for some sets of alternatives, the “obvious” parser may not be the one used, simply because it came after another match.

Both sequences and alternatives will flatten like combinators, such that:

name = firstname + lastname
salutation = C+ 'Hello' + name + '!'

is equivalent to:

salutation = C+ 'Hello' + firstname + lastname + '!'

If you need to mantain the logical separation (to parse correctly, or maintain the name of a subparser), wrap the subparser with C:

name = C(firstname + lastname)
salutation = C+ 'Hello' + name + '!'

Repetition

The most flexible option for specifying repetition is brackets:

keyword[0, 10, ',']

The above would parse keyword zero to ten times, separated by a comma. The separator is optional - without it, the result would simply parse keyword zero to ten times.

We could also specify parsing an exact number of times:

keyword[10]

Or, with a separator:

keyword[10, None, ',']

Infinity - math.inf - is a valid maximum value. For convenience, it can be imported directly from Comber:

from comber import inf

param_list = keyword[0, inf, ',']

There are several convenience combinators for common types of repetition.

For zero or more with a separator, using *:

parser*','

Or zero or more without a separator, using the unary +:

+parser

You can declare a parser as optional with ~:

~parser

Recursive Grammars

Consider a program in our toy grammar:

let word = exp(2, 16)
let maxint = minus(word, 1)

It’d be nice to simplify this by eliminating the variable “word” and pass the exp() call directly to minus(), but to allow that, we need to extend our grammar to consider a function call to be a value so we can use one as a function argument. But to do that, we’d need to use function_call in our definition of value - but function_call likewise needs to reference value.

To solve this issue, we can define one of them as defer.

from comber import C, rs, inf, defer

...

value = defer()
function_call = keyword + '(' + value**',' + ')'
value.fill(keyword | number | function_call)

This way, we can refer to value wherever we want, and only define its meaning when we’re ready. We can safely build fairly complex grammars this way, but be wary of performance.

Controlling Space

Before each subparser is run, Comber consumes leading whitespace. By default, this is spaces, tabs, and newlines. You can change this in two ways: set the whitespace property on the outermost parser to a string containing the new whitespace characters, or likewise passing a string as the second argument to the parser. In either case, the value None will disable the feature altogether.

number = rs(r'[0-9]+')@('number', int)

class Let:
    def __init__(self, let, variable, eq, value):
        self.variable = variable
        self.value = value
        # We can ignore `let` and `eq` (which will always contain "let" and "=", respectively).

    def __repr__(self):
        return f'Let({self.variable}, {self.value})'

assignment = (C+ 'let' + keyword + '=' + (function_call | value))@Let

assignment = (C+ 'let' + keyword + '=' + (function_call | value))@("assignment", Let)

grammar.analyze()
grammar('max(1, 4)')

Available Operators

Operators that Python allows to overridden

Unary operators:

And:

()        __call__   parse a string