astero

Write compilers, transpilers and code analysers without hand-maintaining the tables they depend on.

Every program that transforms code needs to know the same structural facts about its language: which positions introduce a variable, which read one, which fields a traversal must enter, how tightly each operator binds. Most projects write those facts out by hand, once per pass. They drift. When they drift the symptom is wrong output, not a crash.

astero takes one declaration of your syntax tree and computes those facts on demand. Python's grammar ships with the library, so for Python source there is nothing to declare.

pip install abilian-astero        # or: uv add abilian-astero

The distribution is called abilian-astero, because PyPI already had astero. The import name is astero:

from astero.lang_py import PY, VARS

Python 3.11 through 3.15. No runtime dependencies. Apache-2.0.

The problem it solves

Fifteen positions in Python can hold a variable name. Here are four of them:

def f(x):                      # a parameter
    import os as x             # an import alias
    try:
        pass
    except E as x:             # an exception name
        pass
    return [x for x in xs]     # a comprehension target

A renamer written with ast.NodeTransformer and a visit_Name method reaches one. That is not a hypothetical: it is the shape of a defect found in three separate compilers, and none of the three crashed. One never bound **kwargs, one inlined a function so that [99 for 99 in xs] came out, one set an assignment context by hand and produced JavaScript that computed NaN where Python gave 5.

With astero the list is a query, and the same query serves every pass that needs it:

from astero.hygiene import rename
from astero.lang_py import PY, VARS

rename(tree, {"x": "y"}, PY, VARS)      # reaches all fifteen

The set changes between Python releases. Version 3.12 added three positions. Code written against the query follows the language without being edited.

What you get

Declare a role for each field of each production: whether it introduces a name, refers to one, or holds a subtree. Six modules read that declaration.

module	what it gives you
grammar	which fields bind a name, which read one, which hold a bare identifier, which a traversal must enter, which productions can appear
rewriting	tree rewriting from pattern => result rules, with every Name's context recomputed for you
scopes	the scope tree of a module, and the names each scope binds
hygiene	renaming, and substitution that cannot capture a name
emit_rules	a code generator written as templates, with brackets derived from a precedence table
coverage	a test that fails when your dispatch table misses a production

Add a binding form to your language, edit the declaration, and all six follow.

Rewriting, without touching contexts

A rule is one string with => between pattern and replacement:

from astero import Pass, rules

fold = Pass("fold", rules("""
    _x + 0 => _x
    _x * 1 => _x
"""))

Identifiers spelled _name are metavariables, _ matches anything without binding, and *_xs splices the rest of a list. Rules never mention ctx or source positions, because both are recomputed from the shape of the result. That is what makes the AugAssign defect above impossible to write.

Your own IR, not just Python's

A grammar carries each production's constructor, so a rewrite builds your node classes. If your AST is generated from CPython's, the whole declaration is one line:

PRESCRYPT = lang_py.build(module=prescrypt.front.ast.ast, name="prescrypt")

If your IR is annotated dataclasses, from_dataclasses reads the fields and sorts off the annotations, leaving you only the roles to write. The PL/0 tutorial builds a complete compiler that way, for a language with no relationship to Python.

Why you can rely on it

Every derivation is compared against an independent source of truth, over the whole Python standard library, on Python 3.11 through 3.15.

derived	compared against	result
assignment contexts	what CPython's parser produces	every position in the standard library, no disagreements
scopes and their names	CPython's symtable	99.98% of blocks agree
emitted source	reparsing the emitted text	1,797 modules, 2,266,043 expressions
operand positions of an SSA IR	that compiler's own declaration	exact match

For assignment contexts that is ten authored role entries reconstructing all 1,457,931 positions across the 1,761 files of the 3.13 standard library. The tests hold the ratio, not the number. A regression fails; an improvement does not have to be chased.

The rewrite engine was first validated by reimplementing all six of latexify_py's tree transformations as rule sets. Differential testing against the originals found 121 output differences, every one a latent bug in the hand-written version.

Three compilers are built on it: a Python-to-C compiler, a Python-to-WebAssembly compiler, and a Python-to-JavaScript transpiler.

What astero is not

• Not a parser. Bring your own AST. astero starts from a tree.
• Not a code generator you run. Nothing is written to disk, and there is no build step. Every entry point is a function call at run time.
• Not a framework. It does not own your pipeline, your IR, or your main. Adopt one query in one function and leave the rest alone.

Lowering and cost models stay yours. astero answers questions about a declaration; it does not decide what your compiler should do with the answers.

Documentation

• Getting started: install it and run three queries against real code.
• TinyPy tutorial: a working compiler for a Python subset, with two back ends, in about 170 lines.
• PL/0 tutorial: the same, for a language that is not Python.
• User guide: every module, its API, and when to reach for it.
• Adopting astero: fitting it into a compiler you already have.
• API reference: generated from the source.

Status

Version 0.1.x. The API may still change between minor versions; the changelog says what moved. Every derivation is checked against CPython on five interpreters, and three compilers are built on it.

Development

make test     # pytest
make lint     # ruff, ruff format, ty, pyrefly, zuban, mypy
make docs     # build the documentation site

This code is version-sensitive, so a green run on one interpreter says little about the others. nox runs the suite on each:

nox -s tests            # 3.11 through 3.15
nox -s check            # everything `make lint` runs

Contributions are welcome. docs/src/guides/developer-guide.md describes how the library is organised and what a new derivation has to prove before it lands.