dbml-sharepoint

Turn a DBML schema plus a YAML mapping into an idempotent, fail-closed, browser-console deploy.js.txt that provisions SharePoint Online lists, columns, lookups, indexes, permission levels, groups and ACLs, with no tenant admin rights, no premium licence, and nothing installed on the target. If you can open the site and press F12, you can deploy.

schema.dbml + mapping.yaml + release.yaml
        |
        v   dbml-sharepoint build
+--------------------------------+
| deploy-manifest.md   <- read   |
| deploy.js.txt        <- paste  |
| rollback.js.txt      <- escape |
+--------------------------------+
        |
        v   paste into the site's browser console (F12)
   SharePoint Online lists, ready to use

Documentation: firmfooting.github.io/dbml-sharepoint. It covers getting started, concepts, per-artifact contracts, the full mapping / DBML / CLI reference, a generated API reference, and the development philosophy.

Why

• Design as code. Your list schema lives in DBML: reviewable, diffable, renderable as an ERD on dbdiagram.io, with indexes declared beside their tables. Deployment and presentation mapping (prefixes, templates, versioning, views, ACLs) lives in YAML next to it.
• Deploy with nothing but a browser. The generated script runs in the site's own console under your own login, calling only documented SharePoint REST/CSOM endpoints. No PnP, no CSOM installs, no app registrations, no Graph consent. See why not PnP, site scripts, or Graph? for an honest comparison.
• Fail closed, rerun safely. Every write is preceded by read-only preflights: wrong site aborts, an existing list needs the exact provenance marker and matching immutable shape, and an existing field needs matching immutable shape. Mutable drift is narrowly reconciled and read back. Reruns skip work the script can verify is already correct, which it decides by reading the live site, not by comparing release tags.
• Real column support. Text, note, choice (+ defaults), person, date, number, boolean, hyperlink, same-site lookups (including deferred circular and self-lookups), calculated columns (formulas in the mapping), indexes, and unique constraints.
• Security is part of the schema. Custom permission levels, site groups (with automated owner assignment via CSOM and optional run-scoped operator self-enrolment), broken-inheritance list ACLs with an exact allowlist reconciliation mode that removes undeclared grants.

Install

uv tool install dbml-sharepoint
# or: pip install dbml-sharepoint
# or, without installing anything: uvx dbml-sharepoint

The solution templates are part of the package, so an install is all you need to use them, no clone required.

This page documents main. --time-zone below arrived after 0.4.0, so if the released version refuses a flag you see here, take main directly:

uv tool install git+https://github.com/firmfooting/dbml-sharepoint

Or work from a clone, if you are contributing:

git clone https://github.com/firmfooting/dbml-sharepoint
cd dbml-sharepoint
uv sync
uv run dbml-sharepoint version

Quickstart

Run it with no arguments and pick one of the shipped templates:

dbml-sharepoint

The wizard copies the template you choose into a project directory of your own, sets your list-name prefix, site URL and time zone, and offers to build it. It changes identity only. The schema and the mapping structure are the tested artifacts and are copied as they ship. Everything it does is also available as flags; it prompts only at a terminal, and prints help in CI or a pipe.

Or drive it with flags

A complete worked example lives in examples/project-tracker:

dbml-sharepoint build \
  --schema examples/project-tracker/schema.dbml \
  --mapping examples/project-tracker/mapping.yaml \
  --release examples/project-tracker/release.yaml \
  --site-url https://yourtenant.sharepoint.com/sites/your-site \
  --time-zone Region/City \
  --site-role default \
  --out ./build

Then:

1. Read build/deploy-manifest.md: it opens with step-by-step run instructions and must show 0 validation errors. (build/index.md lists every artifact, including the reporting/ queries.)
2. Open https://yourtenant.sharepoint.com/sites/your-site/_layouts/15/settings.aspx (a classic page; the script's wrong-site guard needs _spPageContextInfo) signed in as a Site Owner.
3. F12 -> Console -> paste the whole of build/deploy.js.txt -> Enter.
4. Watch the [SP-DEPLOY] lines; success ends with a summary and errors: [].

The pasteable scripts end in .js.txt, not .js. They exist to be opened and copied, never executed from disk, and on Windows a .js file is bound to Windows Script Host, so double-clicking one runs a provisioning script outside the browser. .js.txt opens in a text editor everywhere. Editors that colour by extension will treat them as plain text; rename a copy if you want highlighting while reviewing.

The three inputs

See examples/project-tracker/README.md for a guided tour of all three.

What the generated script does

Phased, logged ([SP-DEPLOY]), each phase fail-closed:

0. Read-only preflights (site identity, rights, existing-schema shape), then permission levels and site groups (settings reconciled; owner corrected via CSOM where possible; optional operator self-enrolment).
1. Lists and non-lookup columns (existing fields verified immutable-shape, mutable settings reconciled and read back).
2. Deferred lookups (circular/self references).
3. Indexes and field defaults.
4. Broken-inheritance ACL assignment; reconcile: exact removes undeclared direct grants.
5. Optional seed rows (via the extension protocol).

Phases are numbered from the phases manifest (src/dbml_sharepoint/analysis/phases.py). Reference steps by name; numbers renumber automatically when the structure changes.

rollback.js.txt deletes the declared lists. It exists for one case: a failed first provision on a site with no real data. Never run it against real records.

Styling: every mapping inherits the fleet style standard: semantic severity tokens, icons and shapes on SharePoint's own formatting classes; see the style guide.

Assessment: every build emits a read-only assess.js.txt (+ assess-manifest.md) that probes a target site's capabilities across three tiers: always-run enumerations (permissions, list templates, lock state, retention labels, locale, features), pack-driven attempt-probes (sealed/AllowDeletion/formatter surfaces, list collisions, version-trim, CSOM availability), and a printed not-assessable honesty block, then prints a COMPATIBLE / DEGRADED / BLOCKED verdict for the pack. It makes no changes; paste it in the site's console before a first deploy.

Reporting: every build also ships build/reporting/: one Power Query (M) file per list (plus dictionary, model-info and user-added-column audit queries), a SQLCMD views script for warehouse-landed copies, guide.md with the Power BI relationship table, and data-dictionary.md. Point the queries' SiteUrl parameter at the deployed site. dbml-sharepoint report emits the same queries without needing a site URL (schema-only layout: powerquery/, sql/, guide.md, data-dictionary.md).

Extension protocol

Organisation-specific behaviour (identity seeding, classification projection, per-site policy) stays out of the core. Implement DeploymentExtension (hooks: expand_column, seed_lists, extra_validators, manifest_extras) in your own package and register it under the dbml_sharepoint.extensions entry-point group:

[project.entry-points."dbml_sharepoint.extensions"]
my_org = "my_org_deployer.extension:MyOrgExtension"

The core CLI resolves --extension my_org; or ship your own thin CLI that composes the core pipeline programmatically (see dbml_sharepoint.cli for the composition points).

Limitations (honest ones)

• SharePoint Online only, same-site lookups only (SharePoint cannot span webs with a lookup).
• Clean first provision + same-release resume. A schema upgrade whose immutable shapes changed (field types, lookup targets, list templates) fails closed for explicit migration rather than guessing.
• Calculated columns can't reference Lookup/Person columns or [Today]: SharePoint's rules. The validator checks that every reference names a column of the entity, which catches [Today] and typos but not a Lookup or Person operand: '=[Owner]' builds exit 0 and fails at paste time with an HTTP 500. Check operand types yourself; see the DBML reference.
• The browser-paste model means an interactive operator; that is the point (no stored credentials, no app principal), but it is not unattended CI.

Repository map

One module per concern, grouped into layer packages; the packaging spine sits at the package root:

The shipped solution templates live in src/dbml_sharepoint/solutions/, inside the package, because only files under it reach the wheel and the wizard's audience is somebody who ran uvx dbml-sharepoint and never cloned this repository. Not to be confused with templates/ below, which is Jinja.

Templates mirror that: templates/*.js.j2 are the four pasteable scripts; templates/_*.js.j2 are shared partials (provenance header, site guard + apiUrl/odataName, cached digest, read transport, write headers); templates/deploy/_*.js.j2 are deploy.js.txt's phase bodies. Not every partial goes into every script. assess.js.txt deliberately omits _http_write.js.j2, which is what makes its read-only guarantee structural rather than a promise.

Conventions: underscore-prefixed names are module-private. Anything imported across modules is public and unprefixed. Extension CLIs compose clear_generated -> validate -> manifest -> emit_bundle rather than re-implementing emission.

Full documentation lives at firmfooting.github.io/dbml-sharepoint, built from website/: a Docusaurus site; cd website && npm install && npm start to browse it locally.

Development

uv sync
uv run pytest                               # full suite (incl. the semantic Jinja template lint)
uv run ruff check src test website/scripts scripts  # lint
uv run mypy                                 # strict typing: src, test, website/scripts
uv run j2lint --ignore jinja-statements-indentation single-statement-per-line -- src/dbml_sharepoint/templates
uv run prek run --all-files markdownlint-cli2  # markdown style

License

MIT: see LICENSE.