mrdb INTERNALS

The one file an agent should read before working on mrdb. It folds NAVIGATION.md, docs/layers.md, and the essential contract rules that used to be spread across mrdb/docs/*. User-facing docs live in ../pages/mrdb-docs/mrdb-guide/; engineering history in ../pages/mrdb-docs/mrdb-story/.

The three rings (import law)

• Ring 1 CORE (core/) imports only itself; its TS mirror lives under assets/core/ (browser SDK) and assets/server/ (node kernel).
• Ring 2 SDK (mrdb/__init__, mrdb.engine) is the ONLY bridge between core and everything above.
• Ring 3 BATTERIES (extensions/*) import only Ring 2, never core internals, never each other's privates.
• The app tier (mrdb/cli.py, mrdb/cli_ops.py) sits above the batteries: it may use Ring 2, battery facades, and core internals (operator commands are app-tier privilege). The user-owned project content (repo-root app.py, config.py, ui/) is outside the governed tree.
• scripts/check_imports.py enforces all edges against a baseline.

Layer law (normative)

The core chapters mirror the tiers; import direction is strictly downward; same-layer edges allowed. One module, one layer, no exceptions - a module missing from this table FAILS the checker (rules M1/M2).

L0  model.schema, model.codec, model.tsident, model.uuid7,
    storage.base.backend (protocol-only seam), storage.base.fs,
    storage.base.lock, storage.base.names,
    wiring.config, wiring.loader, runtime.shardmap, runtime.httpsrv
L1  wiring.registry,
    storage.drivers.posix, storage.drivers.s3, storage.drivers.select,
    storage.drivers.s3_publish, storage.drivers.s3_botocore,
    storage.range_engine, storage.records.format
L2  storage.records.reader, storage.records.reader_core,
    storage.records.writer, storage.records.writer_core
L3  storage.records.overlay
L4  database.table
L5  database.actions
L6  database.memory, database.db, database.engine, database.reader_pool,
    database.index_maint, query.plan, query.where, query.exec,
    runtime.owner, runtime.client
L7  runtime.live, runtime.coordinator, runtime.replica
L8  mrdb/__init__, mrdb.engine, mrdb/cli, mrdb/cli_ops        (SDK + app tier)
L9  extensions/*, mmr/*                                       (batteries + host)

Checker rules: R1 layer creep; R2/R3 core importing root/extensions; R4 extensions import only mrdb/mrdb.engine; R5 no underscore-private reach- through (use public seams); R6 no function-level imports (lazy only to break a cycle / defer heavy dep, with a comment naming it); R7 no cross-battery imports except declared EXTENSION.requires.

Maintenance rule: adding a core module requires a row here in the same changeset; moving a module updates the table and nothing else.

Domain bands (how to read the core tiers)

Each core domain follows the same grammar - contracts, engine, providers, facade - so the L0-L7 chain reads as four per-domain stacks rather than one flat ladder:

wiring/ is the plugin contract, not composition policy: core consumes the hook buckets downward (runtime/live.py reads guards/routes at request time), so registry.py cannot move above core.

Task -> files to read (and only these)

Cross-runtime parity pairs

Same concept, findable under the same name on both sides:

A change to one side must keep the other side's goldens green.

Invariants (must hold through any change)

1. Byte-frozen format: frames, base header/index, CRC placement, generation retention. Goldens are normative. The one approved amendment (P9/D29) added META-ONLY fields (meta.epoch, meta.segs).
2. Frozen npm surface: every export keeps name and behavior; new subpaths enter as additive {browser: null} entries.
3. Frozen Python SDK: mrdb + mrdb.engine pinned by tests/backend/unit/test_engine_sdk.py.
4. Lock order: per-table lock(s) first, db-global last. Never inverted.
5. Cross-runtime lockstep: one normative contract (goldens + wire-contract.md), two implementations.
6. Single drainer-writer per kernel; idempotent LWW re-application.
7. Readers are snapshots; compaction-race retries stay internal to the reader, both runtimes.
8. One owner per db dir; fencing = epoch segments + version- conditional publish; takeover only through the normal claim path with fail-closed rules (foreign or possibly-alive owners never fenced).
9. Byte-range serving: no caller may full-read a large object; reads go through engine_for(storage).wrap(handle, name); read_all capped at MAX_READ_ALL_BYTES (1 MiB), allow_large=True = reviewable escape hatch.

The executable contract web (do not break silently)

All under tests/backend/runtime/golden/ unless noted:

• contract.json - machine-readable constants (versions, bounds, fold policy, lease timing, reserved names); asserted by BOTH suites.
• cases.json - codec row vectors both runtimes decode identically.
• table.json, sharded.json - byte-exact journal+base fixtures.
• rejections.json - malformed-input corpus (16 cases) both runtimes reject identically; tolerated pins prevent silent tightening; journal cases pin py-fails-closed / ts-degrades.
• reader-vectors.json, auth-vectors.json - semantic vectors on shared fixtures.
• range-cache.json - scripted range-cache conformance executed by both suites; divergences encoded per runtime IN THE DATA.

Rule: semantic changes land in the contract first; both runners stay green. If you add a limitation or divergence, document it (guide 09 for S3) or encode it in a contract - never leave it implicit.

Contract conventions (for range-cache-style suites)

• Runners: Python tests/backend/parity/test_parity_conformance.py, TypeScript tests/frontend/tests/parity-conformance.test.ts. Python is the reference runtime - write cases against its behavior first, then port the runner.
• Resource bytes are deterministic: byte i = i & 0xff.
• Outcome classes map each runtime's exceptions to shared names: value (programmer error), eof (beyond the committed resource), error (anything else).
• Counter names use the TS spelling (sourceFetches); runners normalize (RangeEngine.snapshot().source_fetches).
• Adding an area: copy range-cache.json, keep cases small enough to review at a glance - the matrix grows by accretion, not by generality.

Subtle seams (edit carefully, run contracts after)

• core/model/codec.py + plan cache (weakref-keyed 4-tuple entries, sweeps past 512): encode/decode fast paths must keep byte-identical output.
• core/storage/records/writer_core.py: fused frame path has an optimistic pass with restart-on-null fallback preserving validation order.
• core/storage/records/reader_core.py: ParsedBlock blob+arrays form + binary search; BLOCK_CACHE_MAX=1024 budget; tests scale off it dynamically.
• range_engine.py: serve-through assembly means reads NEVER depend on cache residency; seeds (mmaps) are zero-copy and capacity-exempt.
• POSIX append/publish syscall order is trace-pinned FROZEN (test_posix_trace.py) - do not reorder fsync/rename sequences.

Standing rules (engineering law, from the architecture plan)

1. No new role on .owner unless an existing role is retired in the same changeset.
2. No new cross-runtime mechanism without naming its contract artifact or conformance suite AT DESIGN TIME ("keep in sync by hand" is rejected at review).
3. Every mechanism declares its consumers in its module docstring.
4. Speculation is staged, not landed early.

The full decision register (D1-D31), anti-patterns list, twin strategy matrix, and amendment history live in ../pages/mrdb-docs/mrdb-story/97-design-register.md.

Bench gates and methodology

• P0 anchor (tree 5ab49381, AMD Ryzen 9 PRO 8945HS): flock acquire 6.94 us; single-row commits memory 130 us / file 179 us; batched 2.23 us/row; bench.py write ~470k/s vs sqlite ~970k/s, reads ~460k/s vs ~103k/s.
• Gates: no metric worse than 10% vs baseline; contended handoff may differ but must not exceed 2x; scale-invariance (bench_scale.py flat vs prefill) is a standing gate.
• Sensitive signal for lock/write work: memory-backend single-row commits (fsync is a RAM no-op there).
• Trace before timing: an added hot-path syscall is a seam bug regardless of timing (P5 rule; the syscall-order trace test pins it).
• Same-session A/B when comparing trees: extract the old tree via git archive, bench via PYTHONPATH alternating old/new runs so machine drift cancels; bench.py's sqlite rows are the drift control.
• Benches are manual-not-CI by policy; S3 numbers are directional only (note bucket+region).

Known deferred seams and declined work

• Database.open(path_or_url, *, storage=, coordinator=) injection was DEFERRED with a named design trap: an injected coordinator's strict name walk refuses symlinked components, while commit-path locking locks THROUGH the memory-table tmpfs target - a memory-table-aware coordinator lock-resolution rule must be specified first.
• Local readers retain full baseCrc verification; a future remote driver must pin object version + prove a trusted whole-object checksum before range-only reads may skip a download.
• lock.py monolith split DECLINED (hygiene not risk) at review round 5; revisit if lock.py grows new responsibilities beyond lease + flock + claim guard.

Gates (run before declaring done)

cd mrdb && timeout 600 python -m pytest tests/backend -q -p no:cacheprovider
cd mrdb && timeout 600 bun test tests/frontend/tests
python scripts/check_imports.py            # from repo root
python scripts/fix_style.py                # dry-run; --write to apply
timeout 300 python scripts/check_types.py  # diff vs recorded baseline

Bench gates: tests/backend/performance/bench_scale.py must stay flat vs prefill (scale-invariance); bench.py sqlite rows are the drift control. Record notable runs (machine/load/tree state) when they change the story.

Known flake: test_posix_trace.py::test_append_recovery_keeps_the_frozen_ legacy_syscall_order can fail under load; rerun alone before diagnosing.