PyLGEN

From prototype to production: a Python-native compiler framework that brings the "Dragon Book" to life in Python, with clarity throughout.

PyLGEN gives you complete control over every stage of language processing. Build interpreters and compilers from scratch without leaving the Python ecosystem. Prototype rapidly in pure Python, then compile to native speed with Cython for production workloads.

[!note] Cython compilation requires a C compiler installed on system to compile the code

Why PyLGEN?

• Full pipeline control: You own every step: lexer, parser, AST, semantic analysis, and evaluation.

• Dual‑nature design: Pure Python for development and debugging; Cython for near‑C performance.

• Built for scale: Handles 2‑million‑line inputs with ~1.92x faster parsing than Lark + lark_cython.

• Python ecosystem integration: Leverage NumPy, SciPy, or any library from within your language.

📊 Benchmark at a Glance

more details in the benchmark section of documentation →

PyLGEN is not just a parser, it's a complete, production‑ready interpreter framework that outperforms industry‑standard tools.

🚀 Minimal Example

from pylgen.lexer import Lexer
from pylgen.grammar import AttributedGrammar
from pylgen.parser import ParserBuilder, ParserType

# 1. Define tokens & lexer
lexer = Lexer(mapping_function, r'\s+')
lexer[0, 'NUMBER'] = r'\d+'
lexer[1, 'PLUS']   = r'\+'

# 2. Define grammar with AST builders
G = AttributedGrammar(start=Symbol('E'))
G[E] += (E, plus, T), binary_reductor
G[E] += (T,),        single_reductor

# 3. Build the parser
parser = ParserBuilder.build_parser_from_attributed(G, ParserType.LALR1)

# 4. Parse, analyse, execute
ast = parser.parse(lexer.tokens)
# ... your semantic visitors & evaluator ...

🧩 Architecture

flowchart LR
    Source["Source Code"] --> Lexer["Lexer"]
    Lexer --> Parser["Parser LALR(1)"]
    Parser --> AST["AST"]
    AST --> Visitors["Visitors"]
    Visitors --> Result["Result / Errors"] 

All modules are fully usable from Python and Cython, prototype in Python, ship in C.

⚡ Quick Start

pip install pylgen-core

Then build your interpreter step by step, following the complete tutorial in the documentation.

📖 Learn More

• Step‑by‑step tutorial: Build a full arithmetic REPL from scratch.

• VecLang case study: Production‑grade language with vectors, functions, and slicing.

• Deep‑dive API tour: Understand every module inside out.

Read the full documentation →

PyLGEN: Where compiler theory meets Python pragmatism.