terminusdb-migrations

Alembic-style migration history for TerminusDB.

Version 0.2 adds repository-backed migration files, explicit revision chains, upgrade and downgrade execution, database revision tracking, interrupted migration detection, and a Python operation DSL on top of the native TerminusDB Migration API.

Status: pre-1.0. Review migration files and dry-run destructive changes before using them against production data.

This is a community package, not an official TerminusDB project.

Installation

With uv:

uv add terminusdb-migrations

With pip:

python -m pip install terminusdb-migrations

Python 3.11+ is supported.

Internal package compatibility remains:

terminusdb-async    >=0.1.0,<1.0.0
terminusdb-pydantic >=0.1.0,<1.0.0

Patch releases do not raise these lower bounds unless they actually need newer API.

Repository-backed migrations

A project keeps migration files in Git:

migrations/
├── README.md
└── versions/
    ├── 20261004_001_initial.py
    ├── 20261005_002_add_description.py
    └── 20261006_003_rename_title.py

Each file declares revision, down_revision, upgrade, and downgrade.

Migration callbacks are deliberately synchronous and declarative:

def upgrade(op: Operations) -> None:
    ...

They only build an in-memory migration plan. Database I/O remains asynchronous inside the runner. Async functions, generator functions, and callbacks that return a value/coroutine are rejected before migration state is changed.

The same chain can be applied independently to development, staging, and production databases.

Initialize

tdb-migrate init

Use another location when needed:

tdb-migrate init --migrations-dir db/migrations

Runtime commands require an initialized <migrations-dir>/versions directory. A missing directory is treated as configuration error, not as an empty migration history. This prevents a typo in --migrations-dir from producing a false successful deployment.

Create a revision

tdb-migrate revision -m "add discipline description"

Or provide a stable custom revision id:

tdb-migrate revision \
  --rev-id 20261004_001 \
  -m "add discipline description"

Generated migration:

from terminusdb_migrations import Operations, optional

revision = "20261004_001"
down_revision = None
message = "add discipline description"


def upgrade(op: Operations) -> None:
    op.create_class_property(
        "Discipline",
        "description",
        optional("xsd:string"),
    )


def downgrade(op: Operations) -> None:
    op.delete_class_property(
        "Discipline",
        "description",
    )

The next generated revision automatically points at the current repository head.

Linear revision graph

Version 0.2 intentionally supports one linear history:

base
  |
  v
20261004_001
  |
  v
20261005_002
  |
  v
20261006_003

Disconnected, duplicated, cyclic, or branched histories are rejected before any database migration is attempted.

Branching histories and merge revisions are planned for a later release.

Current revision

tdb-migrate current --db edtech

Example:

current: 20261005_002
head:    20261006_003

History

tdb-migrate history --db edtech

Example:

20261006_003 <- 20261005_002  rename title
20261005_002 <- 20261004_001 *  add description
20261004_001 <- base  initial schema

The asterisk marks the current database revision.

Upgrade

Upgrade to head:

tdb-migrate upgrade head --db edtech

Upgrade one step:

tdb-migrate upgrade +1 --db edtech

Upgrade to an explicit revision:

tdb-migrate upgrade 20261006_003 --db edtech

Dry run:

tdb-migrate upgrade head \
  --db edtech \
  --dry-run

Downgrade

Revert one revision:

tdb-migrate downgrade -1 --db edtech

Return to an explicit revision:

tdb-migrate downgrade 20261004_001 --db edtech

Return to the state before the first migration:

tdb-migrate downgrade base --db edtech

Downgrade executes the migration file's downgrade function. It does not move the TerminusDB branch pointer backwards. The rollback therefore becomes new, auditable TerminusDB history.

Operation DSL

Migration functions receive an Operations object.

Create and delete a class

def upgrade(op: Operations) -> None:
    op.create_class(
        {
            "@id": "Discipline",
            "@key": {
                "@type": "Lexical",
                "@fields": ["code"],
            },
            "code": "xsd:string",
            "name": "xsd:string",
        }
    )


def downgrade(op: Operations) -> None:
    op.delete_class("Discipline")

Create and delete a property

op.create_class_property(
    "Discipline",
    "description",
    optional("xsd:string"),
)

op.delete_class_property(
    "Discipline",
    "description",
)

Rename a property

op.move_class_property(
    "Discipline",
    "title",
    "name",
)

Cast a property

By default a cast is strict: if any existing value cannot be converted, TerminusDB rejects the migration.

op.cast_class_property(
    "Discipline",
    "credits",
    "xsd:integer",
)

Fallback replacement is intentionally not exposed in 0.2 for TerminusDB 12.0.7. The server accepts a Default/value descriptor at the JSON parser boundary but currently does not execute that fallback correctly. Passing default=... therefore raises UnsupportedCastDefault locally instead of sending a migration that ends in an HTTP 500.

Tracking issue: issue #2.

Change a key

op.change_key(
    "Discipline",
    {
        "@type": "Lexical",
        "@fields": ["code"],
    },
)

Key changes are marked destructive.

Expand an enum

op.expand_enum(
    "EducationLevel",
    ["specialist"],
)

Raw native operation

op.raw(
    {
        "@type": "SomeFutureMigrationOperation",
        "...": "...",
    }
)

This keeps the wrapper open to new TerminusDB migration operations without waiting for a convenience method in this package.

Type helpers

from terminusdb_migrations import list_of, optional, set_of

optional("xsd:string")
list_of("LearningOutcome")
set_of("Competency")

Irreversible migrations

Not every destructive change has a meaningful downgrade.

Mark that explicitly:

def downgrade(op: Operations) -> None:
    op.irreversible(
        "legacy_code values were deleted and cannot be reconstructed"
    )

The downgrade fails before any migration request is sent.

Destructive migrations

Deleting classes or properties and changing keys are marked destructive.

They require explicit permission:

tdb-migrate upgrade head \
  --db edtech \
  --allow-destructive

The same flag is available for downgrade.

Use dry-run first when possible.

For a chain of native Migration API operations, --dry-run validates the whole path in one TerminusDB dry-run request. This preserves the virtual intermediate schema, so a revision may depend on classes or properties created by an earlier revision in the same path.

A path containing op.schema_document(...) cannot currently be represented inside that same native transaction. Version 0.2 fails explicitly with UnsupportedDryRun instead of reporting a false successful validation. Full mixed-operation dry-run via a temporary TerminusDB branch is tracked in issue #1.

Database revision state

The library stores migration state in an internal TerminusDB document of type:

TerminusDBMigrationState

It tracks:

revision
pending_revision
direction
owner

State acquisition uses TerminusDB's TerminusDB-Data-Version header as an optimistic compare-and-set. Two runners cannot both acquire the same clean revision: one wins the state update and the other observes the changed branch version. The runner also checks that the stored revision still matches the revision it planned from, so a stale deployment cannot replay an older migration.

Completion is tied to the owner token that acquired the pending state.

The internal schema class is automatically excluded from the legacy Pydantic-to-schema diff.

Dirty state protection

Schema migration and revision-state update use separate TerminusDB API requests. To make interruptions detectable, the runner first records the migration as pending, then runs it, then advances the current revision.

If the process stops in the middle, current shows a dirty state and further upgrade or downgrade commands refuse to continue.

This prevents the database from silently pretending that an interrupted migration never started.

Recovering from an interrupted migration

First inspect the actual database schema and determine what was applied.

Then reconcile the revision marker explicitly:

tdb-migrate stamp 20261005_002 \
  --db edtech \
  --force

Or return the marker to base:

tdb-migrate stamp base \
  --db edtech \
  --force

Stamp changes only migration metadata. It does not execute schema operations.

Python API

from terminusdb_async import AsyncTerminusClient
from terminusdb_migrations import (
    MigrationContext,
    MigrationRepository,
)


async def migrate() -> None:
    repository = MigrationRepository("migrations")

    async with AsyncTerminusClient(
        "http://localhost:6363",
        organization="admin",
        database="edtech",
        username="admin",
        password="root",
    ) as db:
        context = MigrationContext(
            db,
            repository,
            author="schema-bot",
        )

        state = await context.current()
        print(state.revision)

        await context.upgrade("head")

Connection configuration

Database commands accept:

--url
--org
--db
--branch
--username
--password
--token
--author

Environment variables:

TERMINUSDB_URL
TERMINUSDB_ORG
TERMINUSDB_USER
TERMINUSDB_PASSWORD
TERMINUSDB_TOKEN

Example:

export TERMINUSDB_URL=http://localhost:6363
export TERMINUSDB_ORG=admin
export TERMINUSDB_USER=admin
export TERMINUSDB_PASSWORD=root

tdb-migrate current --db edtech
tdb-migrate upgrade head --db edtech

Legacy model diff

The 0.1 API remains available:

tdb-migrate plan \
  --models app.models:MODELS \
  --db edtech

And:

tdb-migrate apply \
  --models app.models:MODELS \
  --db edtech \
  --dry-run

For repeatable deployment, explicit migration files are recommended.

Scope of 0.2

Included:

• migration files stored in the application repository;
• revision and down_revision metadata;
• validated linear revision graph;
• current and history commands;
• forward upgrades;
• explicit downgrades;
• relative targets such as +1 and -1;
• revision state stored in TerminusDB;
• dirty-state detection;
• optimistic concurrency control for migration-state acquisition;
• stale-revision rejection;
• sequential chained dry-run for native migration operations;
• recovery with stamp;
• operation DSL;
• irreversible downgrade support;
• legacy model diff compatibility.

Deferred:

• revision --autogenerate;
• branching migration histories;
• merge revisions;
• automatic rename intent inference;
• automatic backfill generation;
• hard TerminusDB branch reset as ordinary downgrade;
• mixed schema-document/native dry-run via a temporary branch (issue #1).

Autogeneration can be layered on top of the existing Pydantic schema diff in a future release.

Package family

Development

uv sync --group dev
uv run pytest -q
uv build --no-sources

CI tests Python 3.11, 3.12, and 3.13.

Development follows GitHub Flow on GitLab: short-lived branches are merged into main through Merge Requests.

License

MIT License. See LICENSE for the full text.