LEPL 3.1

A Parser Library for Python 3 (and 2.6): Recursive Descent; Full Backtracking

LEPL is a recursive descent parser, written in Python, which has a a friendly, easy-to-use syntax. The underlying implementation includes several features that make it more powerful than might be expected.

For example, it is not limited by the Python stack, because it uses trampolining and co-routines. Multiple parses can be found for ambiguous grammars and it can also handle left-recursive grammars.

The aim is a powerful, extensible parser that will also give solid, reliable results to first-time users.

Release 3.1 has updated stream support. These have a clearer design (although remain, unfortunately, complex), are better documented, with clearer interfaces (abstract classes), and will (I hope) support handling the “offside rule” in a later release.

Features

• Parsers are Python code, defined in Python itself. No separate grammar is necessary.

• Friendly syntax using Python’s operators allows grammars to be defined in a declarative style close to BNF.

• Integrated, optional lexer simplifies handling whitespace.

• Built-in AST support with support for iteration, traversal and re–writing.

• Generic, pure-Python approach supports parsing a wide variety of data including bytes (Python 3+ only).

• Well documented and easy to extend.

• Unlimited recursion depth. The underlying algorithm is recursive descent, which can exhaust the stack for complex grammars and large data sets. LEPL avoids this problem by using Python generators as coroutines (aka “trampolining”).

• Parser rewriting. The parser can itself be manipulated by Python code. This gives unlimited opportunities for future expansion and optimisation.

• Support for ambiguous grammars (complete backtracking). A parser can return more than one result (aka “parse forests”).

• Parsers can be made much more efficient with automatic memoisation (“packrat parsing”).

• Memoisation can detect and control left-recursive grammars. Together with LEPL’s support for ambiguity this means that “any” grammar can be supported.

• Pluggable trace and resource management, including “deepest match” diagnostics and the ability to limit backtracking.