contextlift
Measure whether your written business context makes an AI SQL agent more accurate.
Try it · Quickstart · Walkthrough · Method · Results
Every run this project has measured. Prose a person wrote pays, every time.
The same kind of metadata, read out of a BI tool by a machine, does not.
Try it
Nothing installed, no API key, no network:
uvx contextlift demo
That builds a seven-row warehouse in a temporary directory, answers six questions twice, and prints the verdict. The model answers in it are canned and it says so on screen.
For the headline measurement on public BIRD data, clone and run one target. About one second.
git clone https://github.com/moutasemalakkad/context-lift && cd context-lift
make example-bird
paired ablation on 39 questions (execution accuracy)
without context : 30.8% [18.6%, 46.4%]
with context : 61.5% [45.9%, 75.1%]
lift : +30.8% [+11.8%, +49.8%]
context fixed : 15 questions
context broke : 3 questions
McNemar p : 0.0075 significant at 0.05
ruling : REAL GAIN
What that number is
Most of the accuracy in an enterprise text-to-SQL system does not come from the model. It comes from someone writing down what the business means: the default reporting grain, which rows to exclude, how each metric is defined. This tool measures whether that writing-down was worth it.
The run above is 39 questions over three BIRD
databases. Predictions are generated once and committed, so it prints the same
numbers on your machine as on mine. tests/test_readme_numbers.py recomputes
every figure in that block from those predictions and fails if the two ever
disagree.
What made the difference. BIRD ships one human sentence per question, decoding a business phrase into a filter or a formula: "meetings in 2020 refers to type='Meeting' where YEAR(event_date)=2020". That sentence is not in the schema. It is what a data catalog is supposed to hold, and writing it down doubled the accuracy.
What the verdict word is for. REAL GAIN is printed only when the paired
test clears 0.05 on questions held back from curation. When a round cannot
resolve the effect it was looking for, the word is NOT PROVEN and the number
beside it is not a result. Expect to see both on your own data: that is the
tool working, not failing.
Everything measured so far
| where the context came from | questions | lift | ruling |
|---|---|---|---|
BIRD evidence sentences, 3 databases — make example-bird |
39 | 30.8% → 61.5%, +30.8 | REAL GAIN, p=0.0075 |
BIRD evidence sentences, 11 databases — make example-bird-enterprise |
65 | +18.5 [+6.5, +30.4] | REAL GAIN, p=0.0044 |
rules written by hand, 97-table warehouse — examples/beaver |
69 | 5.3% → 15.0%, +9.7 [+2.0, +18.3] | REAL GAIN, p=0.0391 |
Every lift above is per question, with its 95% paired interval. Answering one question three times gives three answers, not three independent facts, so a per-answer figure has a larger denominator and a narrower interval. The two levels are not interchangeable, so this table uses one of them throughout.
Each row lives in docs/assets/results.json and
names the committed file its numbers come from, plus the test that pins that
file. tests/test_results_chart.py re-reads every source and fails if a row
and its source disagree, so the chart cannot drift from the data. Two larger
rows are described in docs/CASE_STUDY.md rather than
drawn here: a 596-question warehouse holdout, the same 11-database study read
per answer instead of per question, and a round on a BI model whose metrics
carried no written descriptions.
That last one answers the question every customer asks first — can't you just
read my BI tool? The answer turns on one number, the share of metrics with a
description, and contextlift semantic --lookml <repo> prints it for free
before anyone spends anything.
How it works
Measured on a fresh clone: the first run takes about 14 seconds, because
uv builds the virtual environment and caches bytecode. Every run after that
is about 0.17 seconds. The recording above is a warm run.
That is contextlift demo: the whole loop on seven rows of data, with no key,
no cloud and no network. The model answers in it are canned and it says so on
screen; the warehouse, the SQL execution, the paired test and the verdict are
the real code. Re-record it with make demo-gif.
flowchart LR
C["<b>context</b><br/>catalog · BI model · markdown"]:::src
K["<b>answer key</b><br/>mine → gate → split"]:::src
A["<b>two arms</b><br/>same questions,<br/>one difference"]:::core
G["<b>generate</b><br/>ask the agent, twice"]:::core
S["<b>score</b><br/>run both queries,<br/>compare the rows"]:::core
B["<b>ablate</b><br/>paired test,<br/>95% interval"]:::core
V(["<b>REAL GAIN</b><br/>or <b>NOT PROVEN</b>"]):::out
C --> A
K --> A
A --> G --> S --> B --> V
classDef src fill:#ddf4ff,stroke:#0969da,stroke-width:1px,color:#0a3069
classDef core fill:#f6f8fa,stroke:#57606a,stroke-width:1px,color:#1f2328
classDef out fill:#dafbe1,stroke:#1a7f37,stroke-width:2px,color:#0a3622
An arm is the same questions answered under one condition. Two arms differ only by the context the model was given, so the gap between them is what the context was worth. Eight commands complete one measurement:
| command | what it does |
|---|---|
doctor |
check the machine can do the run at all |
gate |
audit the answer key for defects, before anything is spent |
split |
set aside questions to be read once, at the end |
lint |
refuse context that cannot help, before paying for a round |
round |
generate, score and compare both arms in one command |
bundle |
write the run and its evidence to a single HTML file |
next |
read the workspace and print the exact command to run next |
demo |
the whole loop on a toy database, offline |
contextlift --help lists those eight. contextlift --help-all lists all 48.
Three rules the tool enforces rather than suggests. Run every arm more than
once — temperature 0 is not reproducible, and on one engagement no single
repetition reached significance while the pooled three did (p=0.031). Baseline
before curation — it is the only number showing what curation was worth, and it
cannot be recovered afterwards. Hold questions out — scoring on the questions
that guided your curation measures how well you fitted them, so climb warns
unless you pass --holdout.
What gets compared
Not SQL text. SQL results. The predicted query and the gold query both run
against a database, and the row sets are compared under SQL semantics:
column-order independent, multiset unless the gold has ORDER BY, float
tolerant to 4 decimals, NULL-safe. Two queries written differently that return
the same rows count as a match. This is the Spider/BIRD execution accuracy
metric.
The gate runs upstream and is separate: it audits the answer key itself,
because one broken gold question silently corrupts every score after it.
Install
To run it on your own data:
uv tool install contextlift # or: pipx install contextlift
contextlift demo # check it works
To develop on it, from a clone:
make install # uv venv + editable install with dev deps, no cloud extras
source .venv/bin/activate # the `contextlift` command lives here
make test # fast suite, no cloud, no key
make test runs green from a fresh clone with no credentials. That is checked
by tests/test_fresh_clone.py, not just claimed.
Every make target runs through uv run and works without activating. The
bare contextlift command does not — without that activate line, the first
command in either guide fails with command not found.
The core is Python standard library over SQLite. Cloud backends are optional:
uv pip install --python .venv/bin/python -e ".[bigquery]" # --db bq://PROJECT/DATASET
uv pip install --python .venv/bin/python -e ".[gemini]" # the LLM judge
uv pip install --python .venv/bin/python -e ".[catalog]" # Google Knowledge Catalog
--python .venv/bin/python pins the interpreter. A VIRTUAL_ENV left over from
another checkout otherwise wins, and the extra lands there.
Documentation
| read | when |
|---|---|
| QUICKSTART | you want it running in three steps, the first one second long |
| WALKTHROUGH | one real loop end to end, every command and its output |
| ONBOARDING | you have a warehouse, an afternoon, and a blank file |
| FORMATS | every CSV, JSON, --spec directory, and --workspace layout in one table |
| PRACTICES | thirteen practices, each printed beside the measurement that earned it |
| METHODOLOGY | why each number is computed this way, with citations and known limits |
| CASE_STUDY | every measured result, with the command that produced it |
| ATTRIBUTION | how one written line gets credited or blamed for one question |
| CONTEXT-SOURCES | where each piece of knowledge belongs in a catalog, then the field notes from a live estate |
| SECURITY-AND-DATA | what leaves your estate and what does not |
| COLD-RUN | proving the loop on a dataset nobody here has seen |
| AUTHORING | adding a warehouse or an agent: two protocols, and what a new one must not do |
| ROADMAP | what is wanted, and what is deliberately not planned, with the reasons |
contextlift <verb> --help is the reference for every verb and flag. It is
generated from the code, so it cannot disagree with the code.
Contributing: CONTRIBUTING.md. Reporting a vulnerability:
SECURITY.md. Everything runs offline, so make install && make test is the whole setup.
Repo layout
src/context_lift/ the package: 48 commands, one per stage of the loop
tests/ the fast suite — no cloud, no key, no network
examples/ bird, beaver, thelook: each with its own README and runbook
docs/ methodology, results, security, and the two guides
scripts/ stage runners that are not verbs yet (LookML reader, convention miner)
docs/assets/ the README's chart and demo, and the script that draws them
deploy/ a Cloud Run job, for a round that outlives a laptop
License
Apache-2.0. Copyright 2026 Moutasem Akkad. See LICENSE and NOTICE.
Metadata
Release files for contextlift 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| contextlift-0.1.0.tar.gz | 3.5 MB | With provenance | Details for contextlift-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| contextlift-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for contextlift-0.1.0-py3-none-any.whl |
Total release size: 4.1 MB
Release files / contextlift-0.1.0.tar.gz
| Download URL | contextlift-0.1.0.tar.gz |
|---|---|
| Size | 3.5 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a1b26f20ef9097416abaf2e4ede9efd2fb9f16e931b51e208f61e3bccf215b3d
|
|
BLAKE2b-256 checksum How to use checksums |
99b19e9df088c40e6dc93f85baa7ca0bc53af2814bd9ae84eed6a76c0bfffb48
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.
Transparency logRelease files / contextlift-0.1.0-py3-none-any.whl
| Download URL | contextlift-0.1.0-py3-none-any.whl |
|---|---|
| Size | 566.2 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a32f49541ae7b1a1f3311f6cdf798ea35db473cb5aa3d45dc1f172b7d3562695
|
|
BLAKE2b-256 checksum How to use checksums |
4d0300ee5280ec158976a3629df58a4c78e2f5a69e5dceacea2a6e3dfb0ba8a1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.
Transparency logSecurity for contextlift 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for contextlift-0.1.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.