basalt-vault — the Basalt Protocol v1 in Python
An independent implementation of the Basalt Protocol,
held to the same language-neutral fixture corpus as the
TypeScript reference in
conformance/.
Conformance: L0–L4 (§14.1) — Transport, Content, Index, Security, Gatekeeper. L5 (history) is not implemented.
The load-bearing claim is in §0 and §13: two independent implementations given
the same content MUST emit byte-identical index objects. This package
reproduces every file in conformance/vaults/minimal/expected/ — generated by
the TypeScript implementation — byte for byte, and the reverse holds too: the
TypeScript check command confirms that indexes written by this package
reproduce exactly.
$ basalt conformance-regen --vault ../conformance/vaults/minimal/vault --out /tmp/regen
$ diff -r /tmp/regen ../conformance/vaults/minimal/expected/ # no output
Install
pip install basalt-vault
This installs the basalt package and the basalt command. boto3 is an
optional extra, so the core protocol — indexing, link resolution, ACL
evaluation — runs with no cloud dependency installed:
pip install 'basalt-vault[s3]'
Topic search is a second extra, because gensim brings numpy and scipy with it:
pip install 'basalt-vault[topics]'
From a checkout, uv sync (with --extra s3 or --extra topics) sets up the
same thing for development.
Dependencies are deliberately few: ruamel.yaml, because §13.4 requires
comments be preserved when an implementation edits YAML it did not author, and
regex, because §9.6's inline-tag grammar and §10.5's tokenizer are written
in Unicode property classes (\p{L}, \p{N}) that the standard re module
does not implement. The gatekeeper is http.server.
The self-check, §14.3
§14.3 makes this command required rather than optional, "because without it the answer is unknowable":
basalt check --vault ../conformance/vaults/minimal/vault \
--principal u:reader --path "Notes/Columns.md"
vault 01JBQ7X8Z4N3VYK9M2T5HRWC6P Minimal Conformance Vault
audience=gated bypass_traverse=false
integrity 0 errors, 1 warning(s)
warn UNRESOLVED_LINK Notes/Hexes.md "Does Not Exist"
index fresh
rebuild reproduces stored indexes byte for byte
access u:reader on Notes/Columns.md
deny read by deny:g:staff
allow list by allow:everyone
allow traverse by allow:everyone
…
It reports all four things §14.3 asks for: integrity (§10.7), index freshness (§10.6), whether a rebuild reproduces the stored indexes byte for byte (§13), and the ACL evaluation of a named principal against a named path (§8.7) — naming the ACE that decided each right, which is what makes a band-ordering bug visible.
Commands
basalt check self-check (§14.3)
basalt index rebuild every index from content alone; --write to persist
basalt mint mint node ids for unbound objects (§4.3)
basalt add store a note, image or file as a NEW object (§12.7)
basalt gist export a folder to a GitHub gist, or import a gist into a vault
basalt serve run the gatekeeper (§12)
basalt effective effective rights with the deciding ACE per right (§8.7)
basalt tree the visible tree for a principal
basalt digest the content digest and index freshness (§10.6)
basalt find find notes by topic (optional, not protocol; needs the topics extra)
Every command takes --vault DIR, --git DIR (a folder in a git working
tree: each write is one commit, --remote NAME to pull and push), --gist ID
(a vault inside a GitHub gist: text only, at most 300 files), or an
S3-compatible store:
basalt check --bucket my-vault --endpoint https://….r2.cloudflarestorage.com --flavor r2
basalt check --bucket my-vault --endpoint http://localhost:9000 --flavor minio
The flavors exist because the three stores disagree in ways §1 pushes into the
adapter: R2 requires region auto and rejects the integrity headers newer
SDKs send by default, MinIO needs path-style addressing, and no two of them
compute an etag for a multipart upload the same way — which is exactly why
§10.1 records content_hash separately and treats it as authoritative.
Layout
src/basalt/
order.py §13.3 the one comparator all ordering goes through
keys.py §1.1, §2 key validation, path algebra, collisions
canonical.py §13.1 canonical JSON; json.dumps is wrong four ways
yamlio.py §13.4 YAML in, YAML out, comments intact
frontmatter.py §5 parsing with error recovery, kinds
links.py §9 extraction, exclusion zones, the six resolution steps
acl.py §7, §8 principals, roles, the four-band evaluation
security.py §8.5, §8.6 descriptor chains and the traverse composition
indexer.py §10 every index, plus the merkle digest
vault.py §3 the facade: load, scan, reindex, read, mint
store.py §12.7 creating objects: notes, images, any file
gatekeeper.py §12 the HTTP read surface
cli.py §14.3 the self-check and friends
topics.py (none) topic search; optional and not protocol, see below
gist.py (none) `basalt gist export` and `import`
transport/ §1 the five operations: local, memory, git, gist, S3
tests/
test_conformance.py the L0-L2 corpus
test_acl.py the L3 corpus
test_expected.py byte-exact index reproduction
test_gatekeeper.py L4, including the §12.5 cache-leak test
test_store_conformance.py the §12.7 corpus, shared with ts/ and rust/
test_git.py the git store, against repositories on disk
test_gist.py the gist name corpus, export, import and the gist store,
against a local stand-in for the gist API
…
Topic search (optional, not protocol)
basalt find "volcanic rock" --vault ../conformance/vaults/demo/vault
basalt find --like "Geology/Basalt.md" --vault ../conformance/vaults/demo/vault
1.00 Basalt Geology/Basalt.md
0.88 Columnar Jointing Geology/Columnar Jointing.md
0.71 Columns Start Here.md
search.json matches words: a query for "volcanic stone" finds only the notes
that contain them. basalt find fits a latent semantic model (gensim: TF-IDF,
then a truncated SVD) and ranks notes by how close they are in topic, so a note
about hexagonal basalt columns answers a query about volcanic stone without
sharing a word with it. From Python it is basalt.topics.TopicSearch, whose
find and related take the principal to ask as.
It is not part of the protocol, and these are the places that shows:
- Python only. The TypeScript and Rust implementations have no equivalent and the gatekeeper has no route for it; §12.3 is the protocol's read surface and this is not on it.
- Nothing is written to the vault. A fitted model is floating point that moves with library versions, so it is not an index (§0) and cannot be held to byte-identical output (§13). It is fitted on demand and kept in memory.
- A model never learns from a note its requester cannot read. It is fitted per principal over only the notes that principal may read, because a model's vocabulary and loadings carry the text it was fitted to (§8.8), and cached by which notes those are rather than by who asked (§12.5). The cost is one fit per distinct visibility set; 300 notes fit in about a fifth of a second.
- It needs a few notes to generalise from: at least three readable ones,
and
basalt findsays so rather than returning nothing. The number of topics defaults to about the square root of the note count, since a model with as many dimensions as notes is a keyword match;--topicsoverrides it. - Scores below 0.3 are dropped, a cutoff chosen on a 300-article corpus
where the lower scores were mostly noise.
min_scorechanges it from Python.
Tests
uv run pytest
312 tests, of which 15 need the topics extra and are skipped without it. The
157 that come from conformance/ define nothing themselves —
every expectation is loaded from the corpus, because §14.2 is explicit that no
implementation defines its own answers for behavior the spec covers. The rest
cover the conditions §4–§12 describe in prose that no single fixture exercises
end to end: duplicate-id repair, orphaned sidecars, protected folders, the
traverse chain, inline ACLs replacing folder entries, §8.8's search leak rule
in both directions, and §12.3's 404/403 split. The topic tests are about the
same thing for a model: that its vocabulary never holds a private note's words,
and that a model warmed by the owner is not served to the reader.
Python-specific notes
Three places where the obvious Python is the wrong Python:
json.dumpscannot produce §13.1.sort_keyssorts by code point rather than by the UTF-8 byte sequence §13.3 specifies,ensure_asciiescapes exactly what §13.1 forbids escaping, and U+2028/U+2029 are left literal. The serializer incanonical.pyis written out.boolis a subclass ofint. Anisinstance(value, int)test in a JSON writer rendersTrueas1. Checked in that order, with a test.- UTF-8 is order-preserving, so Python's own
strordering happens to agree with §13.3 — the ordering that differs is UTF-16 code unit order, which a JavaScript implementation has to correct for. This package still sorts ons.encode("utf-8"), because §13.3 specifies bytes and sorting the bytes cannot drift.
See the implementation report for the spec ambiguities found while writing this.
Metadata
Release files for basalt-vault 1.0.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| basalt_vault-1.0.0.tar.gz | 92.2 kB | With provenance | Details for basalt_vault-1.0.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| basalt_vault-1.0.0-py3-none-any.whl | Python 3 | none | any | Details for basalt_vault-1.0.0-py3-none-any.whl |
Total release size: 199.9 kB
Release files / basalt_vault-1.0.0.tar.gz
| Download URL | basalt_vault-1.0.0.tar.gz |
|---|---|
| Size | 92.2 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
62d529eaf793286b8ef2a39d6bc7acd3e7426f3d2b77d13e282749808b199e88
|
|
BLAKE2b-256 checksum How to use checksums |
c99eda23ae46838257599e06996b85a526da74ec337d0f25376649dd883e9966
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.
Transparency logRelease files / basalt_vault-1.0.0-py3-none-any.whl
| Download URL | basalt_vault-1.0.0-py3-none-any.whl |
|---|---|
| Size | 107.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
4ae63c45f369a6ae2b71c35ad97196ae76ce01c73cb16d10bfeca8060313407a
|
|
BLAKE2b-256 checksum How to use checksums |
765573491ef29376234f3ac1ba8507186d97e8197d8efbef2e11a85b416f528f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.
Transparency logSecurity for basalt-vault 1.0.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for basalt_vault-1.0.0.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 13, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.