Skip to main content

Varda

CI PyPI Python Docs Status: experimental

Dimensional modeling for LinkML.

Experimental. Varda is published so it can be tried, not because the design has settled. The vocabulary will change, sometimes in ways that break a model written against an earlier version. Pin varda~=0.4.0.

Varda is a profile of LinkML: a small vocabulary that lets you say a class is a fact table, that a column is a semi-additive measure, that a dimension keeps history. It then checks those claims and generates from them.

A model annotated with Varda is still an ordinary LinkML schema. Every other LinkML tool — gen-pydantic, gen-owl, gen-json-schema — will read it happily and ignore what it does not understand. The suite runs those four generators over the shipped examples and asserts that nothing of Varda's own appears in what they emit, because a claim like this one is worth what it is tested at.

A model that names uuid writes imports: - varda, which resolves through a map Varda prints for the tool that needs it:

$ varda importmap --json > im.json
$ gen-erdiagram --importmap im.json mart.yaml
FctSale:
  annotations:
    varda:role: FACT
    varda:fact_type: TRANSACTION
    varda:grain: [order_number, product_key]
    varda:grain_statement: one row per product per line of a receipt
  attributes:
    order_number:
      annotations:
        varda:role: DEGENERATE_DIMENSION
    product_key:
      range: integer
      annotations:
        varda:role: FOREIGN_KEY
        varda:references: DimProduct
    customer_key:
      range: integer
      annotations:
        varda:role: FOREIGN_KEY
        varda:references: DimCustomer
    net_amount:
      range: decimal
      unit:
        symbol: EUR
      annotations:
        varda:role: MEASURE
        varda:additivity: ADDITIVE
$ varda check model.yaml
8 tables checked against 49 rules (varda 0.4.0): 0 errors, 0 warnings

$ varda generate model.yaml --out out/
wrote out/docs/model.md
wrote out/python/mart.py
wrote out/sql/assertions.sql
wrote out/sql/mart.sql

Install

pip install varda

Python 3.11+. The only runtime dependency is linkml-runtime.

What it gives you

Fifteen annotations. Seven on tables — role, grain, grain_statement, hierarchies, fact_type, scd, physical_name. Eight on columns — role, references, additivity, semi_additive_over, physical_name, max_length, precision, scale. That is the whole core vocabulary, and it is deliberately the whole core vocabulary. Units are LinkML's own unit, which Varda reads rather than restates.

Forty-nine rules that catch the mistakes worth catching. A code names its concern, so the band tells you where to look:

V001–V005 the annotations themselves — typos, bad enum values, unknown prefixes, unknown fields
V101–V104 roles — what a table is, what a column is, and where each role is legal
V201–V204 grain — a fact with no grain, a grain naming a column that does not exist
V301–V307 identity — a dimension with no natural key, a business key that repeats, two identities with nothing to tell them apart
V401–V405 references — a foreign key pointing at a fact, one naming no target
V501–V506 time — a version period on a dimension that keeps no versions
V601–V607 hierarchies — a level that is not a column, a path of one level
V701–V707 measures — an unclassified measure, a semi-additive one that never says what it cannot cross, a decimal one that never says what it keeps
V801–V804 physical naming and types — two classes emitting one table, a width on a column that has none, a range naming nothing

The V7xx family exists because additivity is where the expensive errors live. A structural mistake usually breaks a query. An additivity mistake returns a number that looks entirely reasonable and is wrong, to someone who will act on it.

Four generators — sql, docs, assertions and sqlalchemy — producing runnable DDL, a Markdown reference, the model's claims as queries over the data, and SQLAlchemy Core table definitions. All deterministic: no timestamps, no environment, same model in and same bytes out, so the output can be committed and diffed.

The annotations reach runtime. sqlalchemy emits a MetaData and one sa.Table per table — Core, not the ORM — with every annotation on the info mapping SQLAlchemy stores and never interprets. A consumer holding the module, with no Varda installed, can refuse a query before it runs:

sum(quantity_on_hand) grouped by ['product_key'] → REFUSED
    quantity_on_hand is SEMI_ADDITIVE over date_key

It names no database — generic types throughout, import sqlalchemy as sa and nothing else — so one module serves every engine, while the DDL goes on naming one at a time. The two are held together by execution: both are run against DuckDB at every constraint level and the resulting catalogs compared.

Three levels of enforcement — varda generate mart.yaml --constraints asserted. A constraint is two things at once: a claim about the data, and an instruction to check it on every write. Checking dominates a bulk load, and some warehouses cannot do it at all, so the two can be separated. What the database stops policing does not stop being checked — it moves into sql/assertions.sql, which runs once per load and reports what it finds instead of aborting a transaction.

Four dialects — varda generate mart.yaml --dialect sqlserver. Named rather than assumed: there is no neutral SQL, and a TIMESTAMP emitted for valid_from is a row-version counter on SQL Server rather than a time.

Extending it

Varda's core is small on purpose. Anything specific to how your organization works — cost centers, retention, data classification, ownership — goes in an extension under your own prefix.

The smallest useful extension needs no Python at all. Write a LinkML schema declaring your vocabulary, then a varda.toml:

[[extension]]
name = "acme"
prefix = "acme"
profile = "profiles/acme.yaml"

From then on acme:cost_center is a first-class annotation: checked for typos, its enum values enforced, and listed by varda ext. Misspell it and you get

ERROR V001  DimStore
        unknown table annotation 'acme:cost_center'; declare it in
        acme.yaml or fix the typo

Full documentation: https://mluttikh.github.io/varda/

An extension with code behind it adds rules and generators through varda.ext, and ships as an installable package advertising the varda.extensions entry point. See SPEC.md for the interface and tests/fixtures/acme_ext/ for a complete worked example.

One party, one namespace; extensions add, they never redefine. An extension may introduce annotations, enums and rules under its own prefix. It may not add a value to TableRole or change what SEMI_ADDITIVE means — every generator dispatches exhaustively on those, and the registry refuses at load rather than warning.

Commands

varda check MODEL validate; --strict fails on warnings too, --skip-imported leaves another schema's findings to that schema
varda generate MODEL --out DIR validate, then write artifacts; --force to build from a model that does not conform
varda verify MODEL --out DIR fail if the files under DIR are not what generate would write; --diff to show how
varda rules list every rule, -v for reasoning
varda ext describe active extensions and their vocabulary
varda importmap print the LinkML import map

Exit codes are part of the contract: 0 success, 1 the model or run failed, 2 the invocation was wrong.

Status

0.4.0 — experimental. Published so it can be tried, not because the design has settled. Expect the vocabulary to change, sometimes in ways that break a model written against an earlier version. No warehouse of real size has been modeled in it yet, and no third party has written an extension. Pin it:

varda~=0.4.0

The cost of a break is bounded. A Varda model is annotated YAML, so a break is a find-and-replace over your schema rather than a migration of anything you have loaded, and the model stays an ordinary LinkML schema either way — gen-pydantic, gen-owl and the rest carry on regardless.

One thing is settled: Varda will not grow syntax or fork the metamodel. The annotation-only design is the premise, not a stage.

Rule codes are not settled yet. They were all renumbered during 0.1 so that a code names its concern — V6xx is hierarchies, V7xx is measures — and more renumbering before 1.0 is possible if a band stops describing what is in it. A code is never reused for a different rule, and at 1.0 the numbers freeze.

Building the documentation

pip install -e ".[docs]"
mkdocs serve

Then open http://127.0.0.1:8000/varda/ — note the /varda/ path, which comes from site_url because this is a GitHub Pages project site rather than a user site. Plain http://127.0.0.1:8000/ redirects there.

The vocabulary, rules and command-line pages are generated from the package itself into a git-ignored docs/reference/, so they cannot drift from the code. mkdocs serve regenerates them on every rebuild and watches src/ as well as docs/ — edit a rule's docstring and the page updates.

To build the static site the way CI does:

python scripts/gen_reference.py
mkdocs build --strict

--strict turns a broken internal link into a failed build. The generator is a standalone script rather than a plugin, so the site also builds under Zensical and ProperDocs from the same mkdocs.yml — see docs/design.md for why that matters.

License

MIT for the code. The profile vocabulary in src/varda/profile/varda.yaml is CC0, so it can be reused anywhere without attribution — a vocabulary that constrains its own reuse is not much of a vocabulary.

Metadata

Release files for varda 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for varda 0.4.0
File Size Uploaded Provenance Actions
varda-0.4.0.tar.gz 145.4 kB With provenance Details for varda-0.4.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for varda 0.4.0
File Interpreter ABI Platform Actions
varda-0.4.0-py3-none-any.whl Python 3 none any Details for varda-0.4.0-py3-none-any.whl

Total release size: 242.7 kB

Release files / varda-0.4.0.tar.gz

Download URL varda-0.4.0.tar.gz
Size 145.4 kB
Tags Source
SHA-256 checksum
How to use checksums
a9649ad6dde36e171ec648e8addc6eaed88b274700182c8e31f37c9d4198bd2f
BLAKE2b-256 checksum
How to use checksums
27eb52ea0e1eb87043f3128a441217576696a00b2652a1361e32b954d414d983
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 10, 2026.

Transparency log

Release files / varda-0.4.0-py3-none-any.whl

Download URL varda-0.4.0-py3-none-any.whl
Size 97.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
1a085eb4abb6953a04e4bc936cf7bfa97df9f582b6fe0b1b6167bc4490a7b82a
BLAKE2b-256 checksum
How to use checksums
b83eea43c137d094e928693772a3aa4688b30e544685ba3131951f70023818bb
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 10, 2026.

Transparency log

Security for varda 0.4.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for varda-0.4.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 10, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page