bcql_py 0.2.0

A Python parser for Blacklab Corpus Query Language

A Python parser for BlackLab Corpus Query Language

A full-coverage Python parser for the BlackLab Corpus Query Language (BCQL) that converts query strings into a Pydantic v2 AST (Abstract Syntax Tree) with lossless round-trip reconstruction and structured error reporting.

To get started, you can check out:

• A Quickstart guide
• bcql_py and BCQL general guides
• The full API reference
• Python code examples
• A Gradio demo

Features

• Complete BCQL coverage: token queries, sequences, repetitions, spans, lookarounds, captures, global constraints, relations, alignments, and built-in functions.
• Immutable Pydantic v2 AST: every node is a frozen BaseModel subclass with a node_type discriminator, making inspection and pattern matching straightforward.
• Lossless BCQL round-trip: to_bcql() reproduces the original query (preserving shorthand forms, quote characters, sensitivity flags, etc.).
• Position-aware syntax errors: BCQLSyntaxError carries the original query, the 0-based offset, and a caret-annotated message: ready to forward to a user or LLM.
• Optional semantic validation: a CorpusSpec describes which annotations, span tags, alignment fields, and dependency relations your corpus supports. Pass it as parse(query, spec=spec) to catch typos and unsupported features before they reach the corpus. See the tagset validation guide.
• Zero runtime dependencies beyond Pydantic.

Installation

pip install bcql_py

Or with uv:

uv add bcql_py

Try the demo

A small Gradio app under app/ lets you paste a BCQL query, pick or build a CorpusSpec, and inspect parse + validation results. The hosted demo runs on Hugging Face Spaces at BramVanroy/bcql_py_validation.

To run it locally:

uv sync --group app
uv run python app/app.py

Supported BCQL constructs

Category	Examples
Token queries	[word="man"], "man", [], [pos != "noun"]
Regex & literal strings	"(wo)?man", l"e.g.", "(?-i)Panama"
Boolean constraints	[lemma="search" & pos="noun"], [a="x" | b="y"]
Sequences	"the" "tall" "man"
Repetitions	[pos="ADJ"]+, []{2,5}, "word"?
Spans	<s/>, <s>, </s>, <ne type="PERS"/>
Position filters	"baker" within <person/>, <s/> containing "dog"
Captures	A:[pos="ADJ"], A:[] "by" B:[] :: A.word = B.word
Relations	_ -obj-> _, _ -subj-> _ ; -obj-> _, ^--> "have"
Alignments	"cat" ==>nl _, "cat" ==>nl? _
Lookaround	(?= "next"), (?<= "prev"), (?! "not")
Functions	meet(...), rspan(...), rfield(...)

See the cheatsheet for a quick-reference table of every operator.

Development

git clone https://github.com/BramVanroy/bcql_py.git
cd bcql_py
uv sync

# Run tests and doctests
uv run pytest

# Lint and format
make quality   # check only
make style     # auto-fix

ANTLR to generate the needed tools

BlackLab uses ANTLR to generate the parser/lexer in Java based on a g4 file. We could similarly generate Python files. However, after trying it out, I find the files obfuscated and unclear and I'm not fond of requiring an extra external (Java-based) library. That is not a slight to ANTLR; I am simply not familiar with the tool: I am sure it is incredibly powerful and useful if you know how to use it. To keep a clearer view of this library I therefore strive to make a Python-native implementation that is true to spec. It's also just a fun project that I do not wish to "automate away" (though I might regret that later). At a later time (TODO) I might implement functionality to cross-validate our implementation with the generated ANTLR parser and lexer. For now I will be satisfied with high coverage testing. In case of doubt I have followed the Bcql.g4 file.

If you'd like to try the ANTLR route yourself, you can try it as follows:

1. Install requirements (not included in our pyproject.toml file, you'll need to download these yourself!)

   uv pip install requests antlr4-tools antlr4-python3-runtime
   

2. Download the BlackLab G4 definition from GitHub. You can optionally specify a --branch or --tag, defaults to --branch dev.

   uv run python scripts/get_bcql_g4.py
   # Saved to parser/Bcql.g4
   cd parser/
   

3. Run ANTLR (you can update -v to the latest version if needed)

   antlr4 -v 4.13.2 -Dlanguage=Python3 Bcql.g4
   

Acknowledgments

• BlackLab
• Robert Nystrom's guide on "Crafting Interpreters", specifically the part on "Scanning". Token types and error handling in bcql_py is heavily inspired by his work.
• Jamis Buck's blog post on recursive descent parsers
• Berkeley course notes on BNF