vsr

A Python Trio implementation of a Viewstamped Replication style replicated state machine, inspired by the VSR protocol shape used by TigerBeetle.

This is an educational implementation. It models protocol structure and now supports multi-process TCP nodes using tnetstring3 as the wire/WAL/config encoding. It is not TigerBeetle's production storage, networking, repair, pipelining, recovery, or verification machinery.

Development

Install dependencies:

uv sync

Build the package locally:

uv build

Install from PyPI:

uv pip install vsr-trio
python -m pip install vsr-trio

Install from a local checkout:

uv pip install .
python -m pip install .

Example applications live outside this package. During local development, use the sibling ../vsr_apps monorepo, where each packages/<name>/ directory is an independent Python package that depends on the vsr-trio distribution and imports the vsr package.

Multi-process cluster

Create a static cluster configuration:

uv run vsr init-cluster \
  --nodes n0=127.0.0.1:9000,n1=127.0.0.1:9001,n2=127.0.0.1:9002 \
  --storage file

Storage backends are pluggable: file keeps the educational framed file storage, sqlite stores the same logical blobs in SQLite, and lmdb stores them in LMDB.

The primary batches client requests into a single Prepare until --max-batch-size is reached or --max-batch-delay expires. It may keep up to --max-pipeline-depth uncommitted prepares in flight, while commits still advance only in operation-number order.

Start each node in a different terminal, directory, VM, or machine that has the same .vsr/cluster.tnet file. App clients use the generated .vsr/client.tnet file, which contains only client request authority. Install an app package, or run from the ../vsr_apps workspace, then pass the app package import name:

uv run vsr \
  --app vsr_banking \
  --app vsr_inventory \
  --app vsr_catalog \
  --app vsr_locks \
  start-node n0
uv run vsr \
  --app vsr_banking \
  --app vsr_inventory \
  --app vsr_catalog \
  --app vsr_locks \
  start-node n1
uv run vsr \
  --app vsr_banking \
  --app vsr_inventory \
  --app vsr_catalog \
  --app vsr_locks \
  start-node n2

Send client commands from the app workspace:

uv run vsr-bank add balance 10
uv run vsr-bank add balance 32
uv run vsr-bank get balance
uv run vsr-inventory stock sku-1 5
uv run vsr-inventory reserve sku-1 3 --client-id cart-1 --request-no 1
uv run vsr-inventory available sku-1

Run explicit maintenance flows:

uv run vsr recover-node n2 --from-node n1 --force
uv run vsr sync-node n2 --force
uv run vsr move-node n2 127.0.0.1:9012

recover-node is a single-source educational local copy and prints a warning. It can copy stale state. Prefer sync-node for the quorum-checked maintenance path.

Use a different cluster directory when needed:

uv run vsr --cluster-dir /tmp/vsr-a init-cluster
uv run vsr --cluster-dir /tmp/vsr-a --app vsr_banking start-node n0

Notes

• Membership is static. A node "joins" by starting with the same cluster config and its own node name.
• vsr-trio is the installable distribution. vsr is the import package and CLI command. Example apps are separate packages, developed in the sibling vsr_apps monorepo during local work; repeat --app to compose multiple installed packages in one cluster.
• Each node creates its own directory under .vsr/nodes/<node-name>/.
• Storage uses an append-only tnetstring WAL, redundant checksummed superblocks, checksummed metadata, and redundant checksummed checkpoints for committed app state and client replies. It is still not TigerBeetle's real journal/superblock/checkpoint design.
• Consensus state is owned by a single Trio actor per replica. TCP handlers and outbound workers communicate with it through bounded Trio channels.
• See docs/vsr/static_membership.md for the node lifecycle command matrix and why dynamic membership is out of scope.
• See docs/vsr/storage.md for the WAL, metadata, checkpoint, and startup recovery model.
• See docs/vsr/use_cases.md for banking, inventory reservation, and other replicated state machine scenarios.
• See docs/vsr/implementation_boundaries.md for the explicit list of TigerBeetle subsystems this educational implementation does not model.
• See docs/vsr/release.md for the PyPI release checklist.

License

vsr is distributed under the MIT License. See LICENSE.

Check and format

make check

Or run the individual tools:

uv run ty check
uv run ruff check
uv run ruff format
uv run pytest