parsil-peg 0.1.2

A lightweight PEG parser combinator library for Python.

Parsil

A lightweight PEG parser combinator library for Python.

Parsil provides a small set of composable parsing rules for building recursive-descent parsers. It is designed to be simple, readable, and easy to extend.

Unlike parser generators, Parsil lets you describe grammars directly in Python using a concise and expressive DSL. Grammars are composed from small reusable rules, making them easy to read, test, and maintain.

Features

• Lightweight and dependency-free
• PEG-style parser combinators
• Recursive grammars
• Regular expression support
• Rule composition with operators
• Built-in parser transformations
• Positive and negative lookahead
• Automatic tokenization with optional whitespace
• Full-input parsing with EOF
• Structured parsing results (Success / Failure)
• Helpful parse error reporting

---

Installation

Install the latest release from PyPI.

pip install parsil-peg

Or install the latest development version.

git clone https://github.com/shahilahmed/parsil.git
cd parsil
pip install -e .

---

Quick Start

from parsil import *

grammar = Grammar()

grammar["number"] = (
    R(r"\d+")
    .map(int)
)

parser = Parser(grammar, "123")

result = parser.parse("number")

if result.ok:
    print(result.value)

Output

123

---

Building Grammars

A grammar is simply a mapping from rule names to parser rules.

from parsil import *

grammar = Grammar()

grammar["number"] = (
    R(r"\d+")
    .map(int)
)

grammar["start"] = (
    Ref_("number")
    + EOF()
).map(lambda v: v[0])

Rules are composed using Python operators and helper methods.

---

DSL Reference

Operators

Expression	Equivalent
a + b	Sequence(a, b)
a | b	Choice(a, b)

Repetition

Expression	Equivalent
rule.star()	Repeat(rule)
rule.plus()	Repeat(rule, minimum=1)
rule.optional()	Repeat(rule, maximum=1)

Transformations

Expression	Equivalent
rule.map(func)	Map(rule, func)
rule.skip()	Skip(rule)
rule.label(name)	Label(rule, name)
rule.token()	Token(rule)

Lookahead

Expression	Equivalent
rule.and_()	And(rule)
rule.not_()	Not(rule)

---

Helper Functions

Parsil also provides helper functions for constructing common rules.

Function	Equivalent
L(text)	Literal(text)
R(pattern)	Regex(pattern)
Ref_(name)	Ref(name)
EOF()	EOF()
And_(rule)	And(rule)
Not_(rule)	Not(rule)
Token_(rule)	Token(rule)

---

Tokens

Programming languages usually ignore whitespace between lexical tokens.

Instead of writing whitespace rules manually,

number = (
    R(r"\d+")
    + R(r"\s*").skip()
)

use Token.

number = (
    R(r"\d+")
    .token()
)

Likewise,

plus = L("+").token()
minus = L("-").token()
lparen = L("(").token()
rparen = L(")").token()

Token automatically consumes trailing optional whitespace, making grammars much cleaner and easier to read.

---

Rules

Every parser in Parsil is built by composing rules. Rules can be combined, transformed, and reused to build complex grammars.

---

Literal

Match an exact string.

rule = Literal("hello")

or

rule = L("hello")

Example

grammar["start"] = L("hello")

Input

hello

Output

Success("hello")

---

Regex

Match a regular expression.

rule = Regex(r"\d+")

or

rule = R(r"\d+")

Example

grammar["number"] = (
    R(r"\d+")
    .map(int)
)

Input

12345

Output

Success(12345)

---

Sequence

Match rules in order.

rule = (
    L("a")
    + L("b")
    + L("c")
)

Input

abc

Output

["a", "b", "c"]

---

Choice

Match the first successful rule.

rule = (
    L("yes")
    | L("no")
)

Input

yes

Output

"yes"

---

Repeat

Repeat a rule.

Zero or more

rule.star()

One or more

rule.plus()

Optional

rule.optional()

Example

grammar["digits"] = (
    R(r"\d")
    .plus()
)

---

Ref

Reference another grammar rule.

grammar["number"] = R(r"\d+")

grammar["start"] = Ref_("number")

This allows recursive grammars.

---

Map

Transform a matched value.

grammar["number"] = (
    R(r"\d+")
    .map(int)
)

Input

42

Output

42

---

Skip

Ignore a matched value.

grammar["start"] = (
    L("(").skip()
    + Ref_("expr")
    + L(")").skip()
)

Input

(42)

Output

42

---

Label

Replace the expected rule name used in parse errors.

grammar["identifier"] = (
    R(r"[A-Za-z_]\w*")
    .label("identifier")
)

Instead of

expected ['[A-Za-z_]\\w*']

the parser reports

expected ['identifier']

---

EOF

Require the parser to reach the end of the input.

grammar["start"] = (
    Ref_("expr")
    + EOF()
).map(lambda v: v[0])

Without EOF, trailing input is allowed.

Input

1 + 2 abc

would successfully parse 1 + 2.

Adding EOF() ensures the entire input is consumed.

---

And

Positive lookahead.

Match only if another rule matches, without consuming input.

rule = (
    L("a")
    + L("b").and_()
)

This succeeds only if "b" follows "a".

---

Not

Negative lookahead.

Match only if another rule does not match, without consuming input.

rule = (
    L("a")
    + L("b").not_()
)

Useful for excluding reserved words or terminating conditions.

---

Token

Match a rule and automatically consume trailing optional whitespace.

number = (
    R(r"\d+")
    .token()
)

Instead of

number = (
    R(r"\d+")
    + R(r"\s*").skip()
)

Token keeps grammars focused on language structure rather than whitespace.

Example

grammar["number"] = (
    R(r"\d+")
    .token()
    .map(int)
)

grammar["plus"] = (
    L("+")
    .token()
)

This accepts all of the following.

1+2
1 +2
1+ 2
1   +   2

without changing the grammar.

---

Parse Results

Every parse returns either a Success or a Failure.

Successful parse

result = parser.parse("start")

if result.ok:
    print(result.value)

Failed parse

result = parser.parse("start")

if result.failed:
    print(result.position)
    print(result.expected)

Example

Failure(position=4, expected=[')'])

The failure object contains:

Attribute	Description
position	Position where parsing failed
expected	Expected rule names or literals

This makes it straightforward to produce clear syntax error messages.

---

Examples

The examples/ directory contains complete parsers demonstrating how Parsil can be used to build real-world recursive-descent parsers.

examples/
├── calculator.py
└── lambda_calculus.py

---

Calculator

A simple arithmetic expression parser and evaluator.

Supported operators

• Addition (+)
• Subtraction (-)
• Multiplication (*)
• Division (/)
• Parentheses
• Operator precedence
• Left associativity

Example

>>> 1 + 2 * 3
7

>>> (1 + 2) * 3
9

>>> 8 / 2 + 5
9.0

This example demonstrates

• Recursive grammars
• References (Ref)
• Rule transformations (Map)
• Full-input parsing (EOF)
• Abstract syntax evaluation

---

Untyped Lambda Calculus

A parser for the untyped lambda calculus.

Grammar

term         ::= application

application  ::= atom+

atom         ::= variable
               | abstraction
               | "(" term ")"

abstraction  ::= "\" variable "." term

variable     ::= /[a-z][a-zA-Z0-9_]*/

Example

>>> x
Var(name='x')

>>> \x.x
Abs(param='x', body=Var(name='x'))

>>> \x.\y.x
Abs(param='x', body=Abs(param='y', body=Var(name='x')))

>>> f x
App(func=Var(name='f'), arg=Var(name='x'))

>>> f x y
App(
    func=App(
        func=Var(name='f'),
        arg=Var(name='x')
    ),
    arg=Var(name='y')
)

>>> (\x.x) y
App(
    func=Abs(
        param='x',
        body=Var(name='x')
    ),
    arg=Var(name='y')
)

This example demonstrates

• Recursive grammars
• Left-associative application
• Abstract syntax tree construction
• Positive and negative lookahead
• Automatic whitespace handling with Token
• Full-input parsing with EOF

---

Project Structure

parsil/
├── examples/
│   ├── calculator.py
│   └── lambda_calculus.py
│
├── parsil/
│   ├── grammar.py
│   ├── parser.py
│   ├── result.py
│   └── rules/
│       ├── base.py
│       ├── literal.py
│       ├── regex.py
│       ├── sequence.py
│       ├── choice.py
│       ├── repeat.py
│       ├── ref.py
│       ├── map.py
│       ├── skip.py
│       ├── label.py
│       ├── eof.py
│       ├── and_.py
│       ├── not_.py
│       └── token.py
│
├── tests/
├── LICENSE
├── README.md
└── pyproject.toml

---

Running Tests

Install the development dependencies and run the test suite.

pytest

Example

============================= test session starts =============================
collected 64 items

tests/test_and_.py         ....
tests/test_choice.py       ....
tests/test_dsl.py          ............
tests/test_eof.py          ..
tests/test_label.py        ....
tests/test_literal.py      ....
tests/test_map.py          ....
tests/test_not_.py         ....
tests/test_ref.py          ....
tests/test_regex.py        ....
tests/test_repeat.py       ......
tests/test_sequence.py     ....
tests/test_skip.py         ....
tests/test_token.py        ....

============================= 64 passed =============================

---

Contributing

Contributions are welcome.

If you would like to improve Parsil, please feel free to

• Report bugs
• Suggest new features
• Improve documentation
• Add examples
• Submit pull requests

Please include tests for any new functionality.

---

Requirements

• Python 3.8 or later

---

License

Parsil is released under the MIT License.

See the LICENSE file for details.

---

Links

PyPI

https://pypi.org/project/parsil-peg/

Source Code

https://github.com/shahilahmed/parsil

---

Roadmap

Future improvements may include

• Named captures
• Between
• SeparatedBy
• ManyTill
• Optional(default=...)
• Better diagnostic error messages
• Additional example grammars

---

Why Parsil?

Parsil aims to provide a small, elegant, and Pythonic PEG parser combinator library.

Rather than generating parsers from external grammar files, Parsil allows grammars to be written directly in Python using composable rule objects. This makes grammars easier to read, test, refactor, and extend.

Whether you're building a calculator, a configuration language, or a complete programming language parser, Parsil provides a concise and expressive toolkit for recursive-descent parsing.