Varda

Dimensional modeling for LinkML.

Experimental. Varda is published so it can be tried, not because the design has settled. The vocabulary will change, sometimes in ways that break a model written against an earlier version. Pin varda~=0.3.0.

Varda is a profile of LinkML: a small vocabulary that lets you say a class is a fact table, that a column is a semi-additive measure, that a dimension keeps history. It then checks those claims and generates from them.

A model annotated with Varda is still an ordinary LinkML schema. Every other LinkML tool — gen-pydantic, gen-owl, gen-json-schema — will read it happily and ignore what it does not understand.

FctSale:
  annotations:
    varda:role: FACT
    varda:fact_type: TRANSACTION
    varda:grain: [order_number, product_key]
    varda:grain_statement: one row per product per line of a receipt
  attributes:
    order_number:
      annotations:
        varda:role: DEGENERATE_DIMENSION
    product_key:
      range: integer
      annotations:
        varda:role: FOREIGN_KEY
        varda:references: DimProduct
    customer_key:
      range: integer
      annotations:
        varda:role: FOREIGN_KEY
        varda:references: DimCustomer
    net_amount:
      range: decimal
      unit:
        symbol: EUR
      annotations:
        varda:role: MEASURE
        varda:additivity: ADDITIVE

$ varda check model.yaml
8 tables checked against 48 rules (varda 0.3.0): 0 errors, 0 warnings

$ varda generate model.yaml --out out/
wrote out/docs/model.md
wrote out/sql/mart.sql

Install

pip install varda

Python 3.11+. The only runtime dependency is linkml-runtime.

What it gives you

Fifteen annotations. Seven on tables — role, grain, grain_statement, hierarchies, fact_type, scd, physical_name. Eight on columns — role, references, additivity, semi_additive_over, physical_name, max_length, precision, scale. That is the whole core vocabulary, and it is deliberately the whole core vocabulary. Units are LinkML's own unit, which Varda reads rather than restates.

Forty-eight rules that catch the mistakes worth catching. A code names its concern, so the band tells you where to look:

The V7xx family exists because additivity is where the expensive errors live. A structural mistake usually breaks a query. An additivity mistake returns a number that looks entirely reasonable and is wrong, to someone who will act on it.

Two generators, sql and docs, producing runnable DDL and a Markdown reference. Both are deterministic: no timestamps, no environment, same model in and same bytes out, so the output can be committed and diffed.

Four dialects — varda generate mart.yaml --dialect sqlserver. Named rather than assumed: there is no neutral SQL, and a TIMESTAMP emitted for valid_from is a row-version counter on SQL Server rather than a time.

Extending it

Varda's core is small on purpose. Anything specific to how your organization works — cost centers, retention, data classification, ownership — goes in an extension under your own prefix.

The smallest useful extension needs no Python at all. Write a LinkML schema declaring your vocabulary, then a varda.toml:

[[extension]]
name = "acme"
prefix = "acme"
profile = "profiles/acme.yaml"

From then on acme:cost_center is a first-class annotation: checked for typos, its enum values enforced, and listed by varda ext. Misspell it and you get

ERROR V001  DimStore
        unknown table annotation 'acme:cost_center'; declare it in
        acme.yaml or fix the typo

Full documentation: https://mluttikh.github.io/varda/

An extension with code behind it adds rules and generators through varda.ext, and ships as an installable package advertising the varda.extensions entry point. See SPEC.md for the interface and tests/fixtures/acme_ext/ for a complete worked example.

One party, one namespace; extensions add, they never redefine. An extension may introduce annotations, enums and rules under its own prefix. It may not add a value to TableRole or change what SEMI_ADDITIVE means — every generator dispatches exhaustively on those, and the registry refuses at load rather than warning.

Commands

Exit codes are part of the contract: 0 success, 1 the model or run failed, 2 the invocation was wrong.

Status

0.3.0 — experimental. Published so it can be tried, not because the design has settled. Expect the vocabulary to change, sometimes in ways that break a model written against an earlier version. No warehouse of real size has been modeled in it yet, and no third party has written an extension. Pin it:

varda~=0.3.0

The cost of a break is bounded. A Varda model is annotated YAML, so a break is a find-and-replace over your schema rather than a migration of anything you have loaded, and the model stays an ordinary LinkML schema either way — gen-pydantic, gen-owl and the rest carry on regardless.

One thing is settled: Varda will not grow syntax or fork the metamodel. The annotation-only design is the premise, not a stage.

Rule codes are not settled yet. They were all renumbered during 0.1 so that a code names its concern — V6xx is hierarchies, V7xx is measures — and more renumbering before 1.0 is possible if a band stops describing what is in it. A code is never reused for a different rule, and at 1.0 the numbers freeze.

Building the documentation

pip install -e ".[docs]"
mkdocs serve

Then open http://127.0.0.1:8000/varda/ — note the /varda/ path, which comes from site_url because this is a GitHub Pages project site rather than a user site. Plain http://127.0.0.1:8000/ redirects there.

The vocabulary, rules and command-line pages are generated from the package itself into a git-ignored docs/reference/, so they cannot drift from the code. mkdocs serve regenerates them on every rebuild and watches src/ as well as docs/ — edit a rule's docstring and the page updates.

To build the static site the way CI does:

python scripts/gen_reference.py
mkdocs build --strict

--strict turns a broken internal link into a failed build. The generator is a standalone script rather than a plugin, so the site also builds under Zensical and ProperDocs from the same mkdocs.yml — see docs/design.md for why that matters.

License

MIT for the code. The profile vocabulary in src/varda/profile/varda.yaml is CC0, so it can be reused anywhere without attribution — a vocabulary that constrains its own reuse is not much of a vocabulary.