Macrame

A bitemporal graph ledger for knowledge management — embedded, single-file, no server.

Macrame stores concepts linked by typed, weighted relationships — where both concepts and relationships change over time, and the history of those changes is itself a first-class asset. Everything lives in one .db file on disk. No database server, no network protocol, no external service.

Why Macrame

as_of became as_of_valid and as_of_recorded in 0.13.2, and this is a break. Through 0.13.1 a single ts was compared against links.valid_from/valid_to — valid time — for the graph's shape and against transaction_log.recorded_at — transaction time — for concept attributes under AttributeMode::AtTime. So as_of("2020-06-01") returned the edges that held in 2020 labelled with what was believed in 2020: a title corrected today to fix a 2020 typo came back uncorrected. Right answer to "what did we believe"; wrong answer to "what was true", and the name promised the second.

The semantics were written down a release before they were changed, deliberately, so the change could be reviewed against a stated position rather than argued in the commit that made it (D-160, 0.12.17 → D-174, 0.13.2).

To migrate: as_of(t) for topology-at-t under current belief is now as_of_valid(t). as_of(t) paired with AttributeMode::AtTime, if what you wanted was belief as of t, is now as_of_recorded(t). Setting both is new and asks the bitemporal question. reconstruct() is unchanged.

as_of_recorded folds the hot transaction_log, so it raises RecordedInstantUnreachable on a database whose log has been archived — it takes a connection and no archive path, and answering from a partial fold would return nearly the right topology. AttributeMode::AtTime folds the same log for the text and raises the same error for the same reason (0.13.16). reconstruct() takes the path and answers the same question.

Quick Start

Rust

[dependencies]
macrame-db = "0.22"

use macrame::prelude::*;

async fn main() {
    let db = Database::open("knowledge.db").await?;

    db.upsert_concept(ConceptUpsert::new("quantum", "Quantum Computing")
        .valid_from("2026-01-01T00:00:00.000000Z"))
        .await?;

    db.upsert_concept(ConceptUpsert::new("entanglement", "Quantum Entanglement")
        .valid_from("2026-01-01T00:00:00.000000Z"))
        .await?;

    db.assert_edge(EdgeAssertion::new("quantum", "entanglement", "ENTAILS")
        .valid_from("2026-01-01T00:00:00.000000Z")
        .weight(1.0))
        .await?;

    let subgraph = db.traverse()
        .start_node("quantum")
        .max_depth(3)
        .execute(db.read_conn(), None)
        .await?;
}

Python

pip install macrame-db

import macrame

T0 = "2026-01-01T00:00:00.000000Z"

with macrame.Database.open("knowledge.db") as db:
    db.write_concepts([
        macrame.ConceptUpsert("quantum", "Quantum Computing", valid_from=T0),
        macrame.ConceptUpsert("entanglement", "Quantum Entanglement", valid_from=T0),
    ])
    db.assert_edge(
        macrame.EdgeAssertion("quantum", "entanglement", "ENTAILS", valid_from=T0)
    )
    graph = db.load_subgraph("quantum", 3, 1 << 20)
    print(graph.dijkstra("quantum"))

Architecture Highlights

Eight Doctrine Invariants

Every design decision derives from these invariants:

1. The boundary is sacred — Everything above libSQL is ours; everything below it is upstream. Never patch the engine.
2. Two clocks, never mixed — Valid time and transaction time are independent axes. No code path derives one from the other.
3. Assertions are immutable — Rows in links are never updated in place. The past is never rewritten; it is only ever superseded.
4. The ledger is a table, not the log — Transaction-time reconstruction reads transaction_log, not WAL or CDC frames.
5. No physical deletion in hot tables — Rows leave through the archive path only. Ad-hoc DELETE aborts at the trigger layer.
6. Derivative state is disposable — links_current is a rebuildable materialization. Drift is detectable, recoverable by rebuild.
7. Embeddings are immutable per version, excluded from the ledger — Vectors live in per-model tables; they never appear in transaction_log payloads.
8. Fidelity is a parameter, never a silent default — as_of_valid(ts), as_of_recorded(ts) and reconstruct(ts) say what they mean in their signatures.

Concurrency Model

• One writer — a dedicated Tokio task holds the sole write-capable connection
• Many readers — WAL journaling; readers never block on writer
• Two-tier priority channels — high-priority (user-driven) preempts low-priority (background)
• Shared as Arc<Database>, not cloned — every method but close() takes &self, and close() takes self because shutdown has exactly one owner; Arc::into_inner(db).expect("last handle").close().await is the pattern. Clone is refused rather than missing: cadence_stop is a cloneable watch::Sender whose drop is what stops the snapshot task, so a second copy keeps it running against a closing database with nothing raising an error (D-203)
• Cooperative chunking — bounded to ~3 ms per chunk, sized from each chunk's measured hold rather than from a constant, under four per-path ceilings (90 edges, 70 concepts, 600 annotations, 30 embeddings) and a 35-row floor

Operational Controls (0.13.0)

Everything below defaults to leave it alone. None of them spells "off" as the absent value, which is the failure the tri-state types exist to prevent (D-155) — a default that silently disables a mechanism reaches every caller who never heard of the knob.

analyze() misses the ~3 ms chunk budget by ~6× on a populated ledger, and that is reported rather than hidden. Measured: 19.1 ms at 40,000 edges. ANALYZE is one indivisible statement, and PRAGMA analysis_limit = 400 damps its cost 3–4× without making it independent of the table — the claim that it did was corrected in D-166 after §8's acceptance asked for a measurement instead of an argument. metrics().budget_violations() names analyze after every call and is meant to. The exemption could not be decided while one kind covered both analyze() and the optimize() that close() runs unprompted (D-168); that split shipped in 0.13.24, and both kinds stayed budget-counted for opposite reasons — Analyze's miss is structural and constant, Optimize's is rare and therefore the informative one (D-197).

Schema Versioning

open() climbs the ladder one rung at a time, each in its own BEGIN IMMEDIATE transaction. A database stamped newer than the build is refused, never downgraded.

v15 is the last rung that changed a primary key before the 1.0 freeze, and v8 was the one before it. Since then, v16, v20, v22, v24, v25, v26 and v28 add tables, v21 adds a column, v19 redefines a trigger, v27 and v29 reissue triggers from the baseline's constants, and v17, v18 and v23 change indexes. None of them touches a key. D-036 forbids a primary-key diff after 1.0, and D-032 is what reserves the pre-1.0 window for exactly this: v15 widens links because two lineages are allowed to believe different things about one edge, and the old key refused that pair with a bare UNIQUE error naming a storage key the caller has never seen. branch_id goes last, so the five-column covering seek the archive sweep runs per candidate row survives the change. On v8: rowid_pk INTEGER PRIMARY KEY costs id the primary key, and D-036 forbids a primary-key change after 1.0 (D-119). It also drops idx_annotations_label and idx_lc_tgt_active, which shipped in the v7 baseline with no query that seeks on them — measured at −7.9% off assert_edge (D-089, D-118).

Rust Implementation

Module Map

Python Bindings (v0.16.0)

Key design decisions

• Synchronous surface — The Write Actor serialises every write through one channel, so exposing await advertises concurrency the architecture does not grant.
• Opaque Subgraph — A #[pyclass] with forwarded accessors; .to_dict() for callers who want the copy. It paid for itself in 0.8.0: the crate re-represented EdgeRef and no binding signature moved, because there is no converted copy whose layout had to follow (D-101, D-123).
• Open intervals cross as None — Not a sentinel datetime, because datetime.max cannot survive .astimezone() east of UTC.
• Absent content crosses as None — load_subgraph does not fetch document text unless asked (content=True). "" cannot mark not loaded, because it is a valid value of the type (D-116, D-123).
• Every error is typed — 58 exception classes under MacrameError (59 including the base), of which seven are intermediate groups for catching sets — IntegrityError, ValidationError, VectorError, TemporalError, WriterError, BudgetError, BranchError — leaving 51 leaves. BranchError is 0.15.0's, over the six refusals a lineage can raise. DbError::kind() returns one of twelve ErrorKind values — the same seven groups plus the five ungrouped failures, so the Rust and Python taxonomies are one thing rather than two — and its match has no wildcard arm, so a variant added without a classification does not compile (D-242). The exception class is a step further: a variant added without one is caught by binding_parity_tests, which runs rather than compiles: DbError is #[non_exhaustive] since 0.13.34, so the mapping's wildcard arm is mandatory and the compiler no longer checks completeness (D-207).
• metrics shipped on — The wheel ships with the metrics feature enabled because feature flags do not survive into binary artifacts. It has been a Rust default since 0.12.11 too, so the two sides no longer differ (D-154).
• Parity is a wave, not a side effect — 0.13.0 closed six gaps at once (W6): the archive-session and chunk-row constants, registered_models() / declared_dimension(), the maintenance calls and tuning keywords above, and a clock seam for tests. A gap opened in the release that created the feature is the one that never becomes a convention — the constants had gone eight releases for want of being anyone's next task.
• The chunk-row constants are ceilings, not sizes — CHUNK_ROWS_EDGES and its three siblings bound the adaptive loop from above; a populated database converges below them. Dividing a batch by one to predict transaction count reads a 0.11.0 fact (D-161).
• Three methods are deliberately absent — raw(), read_conn() and shadow_step. The first two would dissolve the single-writer property or hand out a shared connection; the third is safe in Rust but its epoch obligation would cross as a convention where it is currently a type, on a method whose failure mode is a stale projection swapped over a live one without erroring (D-165). A test asserts all three stay absent, so adding one means answering the register rather than deleting a line.

Performance (measured, not gated)

Measured at 0.21.0 on the reference machine (Windows 11, NVMe SSD, 32 GB RAM, release build): the median of three criterion sessions of benches/budgets.rs, with the control/select_1 row reading 1.45–1.78 µs in every group. Each figure is the operation alone; fixture setup and teardown sit outside the timed region (D-302). The bench fixtures are smaller than the populations some budgets name (2,000 concepts, a few thousand edges and log rows), so a met row is met at bench scale.

The 90-row edge chunk meets its budget on an empty table and sits at it on a populated one. Until 0.21.0 it took ~4 ms, and 9–12 ms into 8,000 edges: every edge insert fires three triggers, and the undo journal SQLite keeps for that one statement spilled to a temporary file on every row. The writer now keeps its temporary storage in memory (Tuning::writer_temp_store, default Memory, D-306), which takes the two rows to 2.22 ms and 3.01 ms (2.88–3.12 ms across the three sessions) with consistency and durability unchanged. 90 is still the ceiling bulk_import's chunk loop starts from, not the size it commits: the actor times each transaction and sizes the next from it (D-143, D-146), so a chunk that runs over shrinks the one after it.

Three costs are shapes rather than points, and those are what to rely on. A single edge assertion does not grow with out-degree: the rows above span hub out-degrees of 0, 666 and 2,666. Its real cost is the number of versions per edge key, which archival caps (D-134). A concept upsert does not grow with the concept's history: examples/concept_guard_cost_probe.rs measures 0.27 ms for a concept with 10,000 versions (D-300). Rehydration is linear to about 1,000 concepts per call and dearer per concept beyond that, because FTS5's merge cost is per transaction.

How to read the numbers. Session-to-session spread on this machine reaches ~29% (D-070), so read shapes and orders of magnitude, and treat a smaller move between two runs as noise. The budgets are deliberately not CI gates (D-055): an absolute ≤ 5 ms on a shared runner is a claim about whichever machine picked up the job. Regressions are found by comparing a machine with itself:

cargo bench --bench budgets -- --save-baseline before

then --baseline before after the change. Hardware-independent properties, such as plan shape, operation counts and growth ratios, are gated in tests/. The full budget table, with each row's mechanism and the figures earlier releases measured, is §9 of the architecture docs. Figures before 0.20.0 include fixture teardown and do not compare directly with these.

A second harness answers a different question. The budgets above are about operations this crate bounds; benchmarks/ runs a TPC-BiH-style workload taxonomy (TPCTC'13 T/K/R classes) over a generated bitemporal history, on the trunk and on a divergent lineage, in Python and in Rust — 13 correctness assertions gate every run, because a benchmark that can time a wrong answer is worse than none. It is a separate package: nothing there is built by an ordinary cargo test here, and none of it ships in the crate. Its headline shapes are that fork latency is flat across 3× the data (empirical O(1)), that transaction-dimension work is an order pricier than valid-dimension work, that branch cost is ancestry rather than divergence — a fork with no writes costs what a diverged one costs — and that chain reads stay flat from depth 1 to 5 and cost 15% more at depth 10. A third instrument lives beside them: benchmarks/diagnostics/ is the spike ladder that attributed and fixed the fresh-file bulk-import defect (D-274, D-275), and it is where the WAL recipe on the bulk path was measured; the vector-build dissection that produced bulk_embeddings (D-276) ran as examples/vector_build_probe.rs. The materialization-skipping bulk (D-277) was scoped by examples/bulk_skip_probe.rs — the mirror trigger is 2.2× of the random-pair bulk, and the chunked rebuild re-derives the projection for 191 ms per 16k.

Known Risks

| Overlaps already in a ledger are reported, not repaired | A ledger written before the interval guard, or by raw SQL, keeps its overlaps. verify_concept_intervals() names them, and a read refuses only the instants both beliefs claim | | The 90-row edge chunk sits at 3 ms on a populated table | 2.22 ms on an empty table and 3.01 ms into 8,000 edges (2.88–3.12 ms across sessions) since the writer's statement journal stopped spilling to disk (D-306); with TempStorePolicy::File the earlier ~4 ms and 9–12 ms apply. bulk_import sizes chunks on elapsed time from that ceiling, so a chunk over the bound shrinks the next (D-143, D-301). Re-priced index by index in 0.21.0 (D-304): the three transaction_log indexes are load-bearing on the write itself (dropping one costs up to 2.5 ms), and the cheapest real candidate saves 0.84 ms against a read whose cheaper alternative was once 68 s — none of it worth trading | | Covering index wins over selective | EXPLAIN QUERY PLAN assertions on every index-sensitive query, plus pins on the plans ANALYZE statistics have moved | | Snapshot chain divergence | verify_snapshot_chain() reports but does not repair (snapshots are disposable) |

--all-features is not a configuration this project supports or gates, and 0.10.0 stopped publishing a test count for it. --all-features is metrics + property-tests together, which puts the R15-prone binaries back inside the main run — the exact arrangement the step above exists to avoid. Measured 2026-08-07: 4 of 4 runs crashed at one attempt, and 4 of 5 still went red at the six-attempt retry budget the quarantined step uses. A required job that fails four times in five is not a gate, it is noise that teaches people to re-run CI without reading it. Run --features metrics and --features property-tests as the two separate steps CI does (D-140).

Minimum Supported Rust Version

1.88, verified rather than declared — cargo +1.88.0 check --all-features --all-targets passes and 1.85 does not. The constraint comes from libsql-ffi's build dependency chain (bindgen → which → home), not from this crate's own code (which needs only 1.73).

Documentation

• Architecture specification — normative surfaces: §4 (schema) and Appendix A (API)
• Architecture Quick Reference — API, schema, decisions, performance. Current to v0.22.0: the header and footer name the release, the schema ladder §4 reads v29 against SCHEMA_VERSION in src/schema/migrations.rs, and §5's API prose carries the surface through the overlay and the adoption law (D-289…D-316). tests/doc_currency_tests.rs gates those four claims — header, footer, ladder rung, and how far the newest decision cited may trail the register — and tests/doc_sync_tests.rs and tests/perf_claim_tests.rs pin the analytics types, the Subgraph accessors and the performance numbers. None of them can check that a passage is true, so where this disagrees with the architecture set, the architecture set wins — it is the normative one.
• Python bindings — §14: async→sync boundary, error tree, stubs
• Decision register — D-001…D-313 with rationale
• Release notes — one document per minor, most recently v0.22.0 (the overlay, the walk and adoption: D-307…D-316, per-lineage concept versions behind the single-lineage switch, adoption across lineages, the change feed, one-write supersession and the conditional pointer move, schema v25…v29, two breaking changes), v0.21.0 (one state rule for concepts: D-303…D-305, concept_disorder and the rung that repairs 0.20.0's projections, old snapshots skipped, one breaking change the user spent), v0.20.0 (the valid axis for state: D-291…D-302, reconstruct_as_of, the concept interval guard and verify_concept_intervals, one runtime refusal that is new), v0.19.0 (the blob store: D-281, D-287 and D-288, content-addressed bytes reclaimed by the archive rather than a refcount) and v0.18.0 (the flexible ledger: D-278…D-286, concepts.extra, the KV sidecar, the relaxed edge-kind charset and opt-in attributes on the subgraph loader)

Naming

Distribution macrame-db, import macrame — on both crates.io and PyPI. The Rust side has no caveat: a crate's [lib] name is namespaced per build graph, so macrame-db providing macrame collides with nothing. site-packages is flat.

The PyPI package macrame is an unrelated, effectively abandoned build tool (0.0.1, 2021). If it installs a top-level macrame/, then installing both leaves two distributions contending for one directory — pip warns on file conflicts, so this is a known and non-silent risk. Importing as macrame_db is the fallback if it ever matters.

License

See LICENSE for details.