VOYD

Ranking is not permission.

A vector index scores relevance. Nothing in an ordinary retrieval path is ever asked the other question — may this fact reach a prompt? — so a search answers with a confident score and no opinion about whether the hit was allowed to be there: an expired row the sweeper has not reached, a fact somebody revoked this morning, a vector from a model that was swapped.

Every other data path in a mature stack solved this years ago. HTTP has middleware. SQL has views and row-level security. The filesystem has permission bits the kernel checks whether or not you remembered to ask. Retrieval has a ranking function and a hope, so the missing answer gets reimplemented inside every query — a tenant filter here, an expire_at clause there, once per call site, per service, per language, forever.

The failure is silent in the worst possible direction. A missing authorization filter returns more documents, not fewer. It does not raise, it does not log, and on a retrieval workload it reads as better recall. There is no error to page on and no test that naturally fails.

Deleting the row does not close it either. MongoDB's TTL monitor runs about once a minute; an object-lifecycle rule runs about once a day. In that window the document is genuinely still on disk and is returned, correctly, as a well-scored result. A faster index does not help and a faster sweeper only narrows it.

deletion   a storage event      eventually consistent, by nature
refusal    a retrieval promise  immediate, by construction

VOYD is the second one, placed where it cannot be bypassed: the wire.

---

The part that surprised me

Once refusal is on the wire, it has to be pure — no database under it, no I/O, documents in and the admissible ones out. That reads like a tax. It turned out to be the most valuable property in the project, and it paid twice.

A policy change can be diffed before it ships. A pure function can be asked about a policy that is not deployed. voyd-plan asks it twice and fails a pull request that widens the boundary — naming the documents, and the role.

Your reranker cannot leak. Because the check is pure and cheap, it can run last, always, after arbitrary page-shaping code:

pure rules  →  your transform  →  every rule, terminally  →  the wire

Reranking, de-duplication and caching have always lived outside the boundary, and outside is what makes them dangerous — anything downstream of a filter can undo it. Usually by accident: merging a cached list, falling back to the unfiltered candidate pool because an empty page looked like a bug. So they move inside, and the guarantee survives:

A transform cannot widen what a read returns. Not because it was reviewed. Because the boundary is downstream of it.

There is a test that starts a real proxy with a policy whose transform exists solely to inject forgotten facts, and reads through it with a pymongo client that has never heard of this package. The transform is not disabled, sandboxed or reviewed. It runs. It returns the documents. They do not arrive.

---

No code

pip install voyd        # or: uv add voyd

Read Known gaps before you rely on it — being installable is not the same as being proven.

You get three commands — voyd-wire, voyd-plan, voyd-wire-health — and the vocabulary to write the file they read. There is nothing here for an application to import.

New here? quickstart.md is twenty minutes, and the first step costs nothing at all.

Declare the rules once, in a file that is not your application:

# voydfile.py
from voyd import guard, deadline, revocable, tenant

@guard("notes", on_delete="revoke")
class Notes:
    expire_at = deadline()
    forgotten = revocable()
    tenant_id = tenant()

Point the boundary at your database:

voyd-wire --config voydfile.py --target localhost:27017

Then change one connection string. That is the whole integration. No import is added to your application, no handle replaces a collection, no read path is rewritten, and nobody has to remember anything. A driver in any language — Python, Node, Compass, a notebook — cannot read a forgotten fact, because the boundary binds the connection and there is nothing to reach past.

Your credentials stay yours. The boundary forwards the handshake and then asks the deployment who authenticated — it does not authenticate on your behalf and does not believe a claim a client asserts, because a rule whose only input is the caller's own assertion is not an authorisation system. So a client connecting to the boundary carries exactly the credentials it would have carried to the cluster.

Reach it with directConnection=true, or run it with --advertise-self; without one of the two your driver reads the cluster's own host list and connects straight around it. Without --tls-cert it binds loopback only, which is a decision rather than a default: a plaintext boundary reachable from a network would carry in the clear every document it had just refused to serve.

---

What a read gets

An expired document and a revoked one stay exactly where they are. The boundary does not delete them and does not wait for anything to:

around the boundary   5 documents on disk
through the boundary  2 — the expired and the revoked are refused,
                          and the other tenant was never in scope

And the verb already in everybody's code gets the better meaning, if you ask for it. on_delete="revoke" turns a client's deleteOne into a mark:

db.notes.delete_one(...)  -> deleted_count=1   (the driver is satisfied)
reachable now             the row is gone from every read
rows on disk              unchanged — nothing was destroyed
the deadline              pulled in, so the reaper collects the bytes
                          on the schedule they already had

A credential you need out of prompts now and on disk for the investigation are contradictory requirements for DELETE and the same requirement for this. It is opt-in, because silently redefining delete for an operator who did not ask is the kind of surprise this project exists to remove — and declaring it without a revocable() field is refused at load, since there would be nowhere to record that the fact was forgotten.

---

The vocabulary

in a voydfile	means
deadline()	this field holds the instant after which the fact is gone
revocable()	a mark an operator sets to forget it now, irreversibly
holdable()	the reversible kind: a hypothesis, not an instruction
tenant()	the tenant id — required in a reduction and checked per document
restricted_to(claim)	admit only callers whose claim overlaps this audience
clearance(order, roles)	an ordered ladder, the caller's rung read from their roles
subjects(key=...)	an array whose elements are subjects in their own right
embedded_with(model)	refuse a vector produced by a different model
budget(n)	refuse once the prompt has no room left
distinct()	refuse a repeat of content already on the page
sealed()	ciphertext at rest, under a key scoped to the tenant
auto_embed(model)	the server embeds this text; refuse a client's own vector
sanitized(on_match=...)	text for a model: invisible characters removed (exact), instruction-shaped signatures refused or cut (a tripwire, not a guarantee)
mask(strip=, visible_to=)	admit the document, serve this field as null (or not at all)

Beside them, and deliberately not one of them:

@transform(collection)	shape the page — rerank, de-duplicate, annotate — inside the boundary, where it cannot widen a read
rerank(collection, ...)	the built-in one: maximal marginal relevance, so ten chunks of one contract stop crowding out nine other contracts
@stage(name)	an aggregation stage mongod does not have, run by the boundary on admitted documents when a pipeline names it
@operator(name)	an expression operator mongod does not have, used as a whole field value of $addFields/$set, per admitted document

None of them is an enforcement point. A rule is what the terminal pass re-asks and what voyd-plan can reason about; a transform gets no credit for filtering and no attestation. See ethos.md.

mask() rewrites a value instead of refusing a document. A contract is readable and its counterparty's tax id is not:

@guard("contracts")
class Contracts:
    expire_at = deadline()
    tenant_id = tenant()
    ssn       = mask()                      # served as null
    internal  = mask(strip=True)            # the key is removed
    salary    = mask(visible_to=("hr",))    # null unless the caller's roles hold hr

The field is rewritten in the reply bytes after every rule and after every transform, so a transform cannot put a masked value back — whatever it returns is masked again on the way out. A command that names a masked field where a value can come from is refused before it is sent: a filter ({ssn: "..."} is a yes/no about the value), a sort, distinct("ssn"), a $group or $project that reads "$ssn", $$ROOT, $getField, $text, a server-side update that copies it, and an explain of any of those. Excluding it — {ssn: 0}, $unset — is always allowed, and so is writing it. findAndModify is allowed only with a fields projection that removes it, because its reply is not a cursor. Masked values are counted as masked_total in /metrics and in a handle's receipts(), apart from refusals: nothing was refused. visible_to reads the caller's roles from the server, never from the client, and an unidentified caller sees the mask.

Top-level fields only — the attribute name is the path. Mask the parent of a nested value; elements of a subjects() array are not masked one by one.

budget(n) and distinct() are set-relative: they refuse a document because of the other documents on the page, so the same document is admitted alone and refused in company. No index filter and no policy engine can express that — $vectorSearch decides each candidate before the page exists, and enforce(subject, object, action) has nowhere to put the rest of the set.

A rule is a protocol, not a list. Three members — reason, refuses(doc), clause() — and a stranger's rule is a first-class one, written directly in the policy file beside the builtins:

@guard("notes")
class Notes:
    expire_at = deadline()
    region    = Jurisdiction(allowed="eu")     # your dataclass, not ours

Anything in a class body that is half a rule raises at load, by name. The two ways to get the protocol wrong are to omit a member and to misspell one, and from the loader they look identical — a skipped rule would mean a boundary that comes up announcing what it refuses while serving everything the rule was written to stop.

A policy file that is wrong fails when it is loaded, not when somebody's query returns the wrong rows.

Text that talks to the model

A stored chunk can carry instructions aimed at whatever reads it: "ignore previous instructions" on a scraped page, a zero-width or tag-block payload a reviewer cannot see, an HTML comment that vanishes when rendered, a markdown image whose URL sends the conversation to somebody else's server. sanitized() marks a field as text for a model:

@guard("notes")
class Notes:
    text = sanitized()                                   # refuse on a match
    body = sanitized(on_match="neutralise",              # cut the match
                     patterns=[("wire_money", r"wire \$\d+ to")],
                     without=["html_comment"])

It makes two claims, and they are not equally strong.

Invisible characters are removed, exactly. Zero-width characters, bidi overrides and the Unicode tag block are code points in known ranges, so this is a fact about bytes. None of them leaves a declared field. The cost is typographic: emoji joined with ZWJ come apart into their members, and ZWNJ-shaped scripts render differently.

Signatures are a tripwire, not a guarantee. Five built-in patterns catch the copy-pasted shapes, and your own are added in the voydfile. The invariant is exact about the list: no text that leaves matches a declared signature once invisibles are removed and the text is NFKC- and case-folded. A paraphrase, another language, or an instruction split across two chunks is not on the list and goes through. A document that quotes an injection to discuss it is matched like one that carries it, because a pattern cannot tell quotation from use.

Under neutralise, a match found only by folding (full-width letters) has no exact span to cut, so that document is refused instead. It runs in the terminal pass, after every transform. distinct, count and reducing pipelines on the collection are refused, because they return the text without the document it came from. Refusals count under injection_signature; rewrites count in neutralised_total.

---

Two enforcement points, and only one of them is the guarantee

Each rule is pushed into the query where a query can express it, and re-checked per document on the way out.

The second one is the promise. A $vectorSearch hit does not pass through the collection query, so every pushed-down clause in this package is an optimisation — the database doing cheap work — and none of them is the boundary. The per-document check is what every read path ends at, including the search path, and it is pure: no database, no connection, no I/O. That purity is not an aesthetic; it is what lets the same check run inside a wire proxy at all.

The asymmetry only runs one way. A rule with no query half is slower. A rule with only a query half would be a hole.

A short page is filled from further down the ranking

Refusing on the way out has a cost the client can see: limit: 10 over a collection whose nearest rows are mostly expired comes back with two. Deadlines are deliberately not pushed into the vector index (see voyd/engine/search.py), so the index cannot skip them itself.

So a lone $vectorSearch on a guarded collection is over-fetched. The proxy multiplies limit and numCandidates by the collection's backfill factor, judges the wider page exactly as it judges any batch, and only then cuts it back to the limit the client sent:

limit: 10, 8 of the 10 nearest expired
backfill=1          2 documents
backfill=4          10 documents, in the index's score order

@guard("notes", backfill=4)     # the default; 1 is off, 20 is the ceiling

The cut runs after the terminal pass and only removes, so it cannot put a refused document on the wire and no page is longer than the client asked for. It spans getMore, so a small batchSize gets the same page. A transform sees the wider page — a reranker choosing from forty candidates rather than ten — and the cut keeps the first limit of its order.

Only the lone stage is widened. $match, $sort, $skip, $limit, $sample or anything else after the search means something different over forty candidates than over ten, so those pipelines go out as sent, and a reduction ($group, $project, $count) has the refusal pushed into it instead. Both are capped at 10,000, the most Atlas accepts. If more than three quarters of a widened page is refused, the page is short, exactly as it is without backfill; the factor is paid on every lone vector search, refused rows or not.

---

Shaping the page, inside the boundary

A rule decides whether one document may reach a prompt. A transform decides what the page looks like — what order, which of them, annotated how. Reranking, de-duplication, a cache.

Those have always been a separate system, running after retrieval, and that is what makes them dangerous rather than useful. A reranker downstream of a filter can put back what the filter removed, and almost never on purpose: it merges a cached list, or falls back to the unfiltered candidate pool because an empty page looked like a bug, or reorders a list it was handed by reference.

# voydfile.py
rerank("notes", diversity=0.3)          # the one that ships

@transform("notes")                      # or your own, any code
class Diversify:
    name = "whatever you call it"

    def on_egress(self, docs, *, request):
        return my_reranker(docs)

rerank() is maximal marginal relevance. A vector index returns the most similar documents, which on a real corpus means the most similar documents to each other — ten chunks of one contract outranking one chunk each from ten contracts:

index order        a0 a1 a2 a3 a4 a5    one cluster, six deep
diversity=0.0      a0 a1 a2 a3 a4 a5    exactly the index's order
diversity=0.7      a0 b0 c0 a1 a2 a3    one from each, then the rest

Relevance comes from arrival order, not from a query vector: a wire boundary does not reliably have one, and every ranked page already carries the index's own judgement. That makes this rank-based MMR rather than the textbook form, which is a real technique and not the same one, so it is named as what it is.

It runs inside:

pure rules  →  transform  →  every rule, terminally  →  the wire

The terminal pass is not a stage you compose. It is the last thing that touches a document, always, and a transform cannot be placed after it because there is nowhere after it. Everything a transform returns — reordered, merged, restored from a cache, invented — is checked before it leaves. So a transform may reorder, drop, annotate and even inject, and a forgotten fact still cannot come out.

The first pass is a different argument: a transform is never shown a forgotten fact. That is defence in depth, not the guarantee — code that never receives a fact cannot mishandle it.

A transform is not an enforcement point and must never be written as one. Dropping a document for a security reason here duplicates a rule badly: the rule is what is re-asked terminally, and the rule is the half voyd-plan can tell you about before you ship it. A transform gets no credit and no attestation.

Two members, name and on_egress, checked at load the way a half-written rule is. A transform that raises is skipped and its input carried forward — the opposite of the decision for a rule, and for the opposite reason: a rule that fails open is a leak, a transform that fails open is impossible, and refusing a whole read over a broken reranker would be an outage caused by an optimisation.

Cumulative rules are held back to the terminal pass, so a budget() charges the page that is served rather than the one that was proposed and then reranked down.

A collection with no transforms runs the loop it always ran.

What it costs. One laptop, 1024-dimension vectors, measured rather than estimated:

                    page of 10   page of 50   page of 200
voyd[rerank]              ~2ms         ~2ms          ~9ms
pure Python               ~3ms        ~72ms      declines

NumPy is an accelerant, not a dependency — pip install voyd[rerank]. Without it the same arithmetic runs in Python, and past a measured ceiling the transform declines and says so rather than adding a second to every read: an optimisation is not allowed to be the slow part.

---

Stages mongod does not have, run on what was admitted

A policy file can add pipeline vocabulary. @stage declares a stage and @operator an expression operator; any driver, mongosh or Compass then uses them in an ordinary aggregate, mixed with native stages:

# voydfile.py
@operator("$wordCount")
def word_count(doc, args, ctx):          # once per admitted document
    return len(str(args).split())

@stage("$stats")
def stats(args, docs, ctx):              # once, over the admitted set
    ctx.publish(args["as"], {"n": len(docs)})
    return docs

db.manuals.aggregate([
  { $match: { tenant_id: "acme" } },                     // mongod
  { $addFields: { words: { $wordCount: "$text" } } },    // the boundary
  { $stats: { as: "corpus" } },                          // the boundary
  { $match: { $expr: { $gt: ["$words", 50] } } },        // mongod, on a temp
  { $sort: { words: -1 } },                              // mongod, on a temp
])

The pipeline is split at its virtual steps, and the order is the guarantee:

native prefix   →  the server, on the client's connection
                →  every rule, sanitized(), mask()       (the ordinary egress)
virtual step    →  the boundary, on admitted documents only
                →  every rule again, on what it returned
native suffix   →  mongod, on a temporary collection of that output
                →  every rule again, on what carries a source _id
                →  the client, as one batch

A refused document never reaches a stage or an operator. The prefix is drained through the same check a find goes through, so a step is handed what this caller would have been served, field for field — masked values already null, sanitized() text already neutralised, and the expired, revoked and cross-tenant rows simply absent.

A step cannot widen a read. It may add fields, drop documents and reorder them. Every document it returns is traced by _id to one it was handed, at most as many times as it was handed; the rest are dropped. What survives is judged again with the fields the verdict reads put back from the admitted source, so a step cannot erase a mark, move a deadline or change a tenant on the way out.

Earlier outputs are ordinary inputs to later steps. A field an operator sets is a field: a later operator reads it by path ({"$redactEmails": "$summary"}), a native $match, $sort or $group on the temporary collection filters on it. A stage can ctx.publish(name, value) — corpus statistics, an idf table — and every later step reads it as $$name.field: resolved in a virtual step's arguments, and passed as let to the native suffix. It is computed over admitted documents, so a pipeline variable carries nothing a refused row contributed. In virtual steps $$NOW (and ctx.now) is one instant for the whole read; in the native suffix it is the server's.

Operator arguments arrive resolved: "$field" is that document's value, "$$name" a published value, {"$literal": x} is x untouched. An operator runs only as the whole value of one field in $addFields/$set, beside field paths, variables and literals. Anything richer that contains a registered operator is refused; so is setting _id or a dotted field.

The native steps after a virtual one run in mongod, on a temporary collection, because mongod is the only engine that runs $group or $setWindowFields exactly. The prefix is never $out into one — that would copy refused documents somewhere no policy guards. What is written is what a virtual step produced from admitted documents and nothing else, stored in arrival order so a ranking survives, and dropped in a finally. A pipeline that ends in a virtual step creates nothing.

Reductions after a virtual step are safe, and before one they are not. A $group before the first virtual step is refused, because its output has nothing left on it to judge. After one, it runs over a collection holding only admitted, masked, neutralised documents, so whatever it counts is a function of what this caller was allowed to see — the reduction a refused aggregate tells a client to do "on your side", done on the client's side of the boundary.

The temporary namespace is a database of its own, __voyd_tmp by default (--virtual-db). Collection names are t_<instance>_<unix seconds>_<uuid>. Every proxy sweeps it at startup and every so often, dropping any collection older than --virtual-max-age (600 seconds) — its own and a crashed instance's alike — and drops nothing whose name it did not make, and nothing in any other database. No command naming that database is served through the boundary: not a find, an aggregate, a write, a change stream, listCollections, or a renameCollection out of it. listDatabases still shows its name, which carries an instance id, a time and a uuid and nothing else.

The connection it runs on is the boundary's own, opened per worker from the same --target URI and credentials. Not the client's session: the client may address nothing in that database, and the boundary's housekeeping has no business inside a transaction the client is running.

Refused outright, with an error the driver raises: explain of a pipeline with a virtual step; $lookup, $unionWith and $graphLookup anywhere; $out and $merge; after a virtual step, any stage that reads something other than its input ($collStats, $currentOp, $documents, $vectorSearch, …), because the suffix runs with the boundary's credentials; a virtual step on an unguarded collection or on aggregate: 1; on a tenant() collection, a pipeline whose prefix does not $match one tenant. More than --virtual-max-docs (1000) documents at any step is an error, never a truncation, and a step that raises fails the read with no partial result. A pipeline naming no registered name is forwarded untouched, and so is an unregistered $foo, which mongod answers as it always would.

VOYD never holds model credentials and never calls a model. A stage is local, deterministic code in your policy file — chunking, counting, redacting, ranking. The boundary guarantees what context the client receives; the client owns inference, with its own key, on what it was served. examples/virtual_stages.py chains all of the above against a live, an expired, a revoked and another tenant's document, and then builds the prompt on the client side.

---

What a policy change would let through

Because the per-document check is pure — a function of a document, a spec and a clock, with no database under it — it can be asked about a policy that is not deployed. voyd-plan asks it twice and reports the difference:

voyd-plan --current voydfile.py --proposed voydfile.new.py \
          --target $URI --database app --all

the boundary moves, in the admitting direction
  notes  tenant_removed
    reads were scoped by 'tenant_id' and no longer are: a read can
    return documents belonging to any tenant

documents that become reachable
  notes  +9 of 200 read
           9  were refused as revoked

200 documents, read
newly reachable: 9

The exit code is the product. 1 when something becomes reachable, 0 when nothing does — so a policy change that opens the boundary fails a pull request and says which documents and why. A change that closes it exits zero on purpose: a tool that blocked those would teach people to bypass it, and a read path that got shorter is visible to whoever it refused.

Without --target it reports the structural half only, which needs no cluster and no credentials: a @guard deleted, a tenant() dropped, a subjects() array that stops being subjects, a mask() removed or loosened — strip=True becoming a null, or visible_to gaining a member. Those are facts about the policy, so they are not weakened by a sample and do not disappear against an empty collection.

It refuses to answer three questions, by name. budget() and distinct() are set-relative — they refuse a document because of the other documents on the page, and a sample is not a page — so they are set aside and printed rather than evaluated one document at a time. clearance() and restricted_to() decide by who is asking, so they are planned only against a caller you name with --as. And the tenant is enforced by the handle against the scope a read is bound to rather than by a rule, so a plan reports that the boundary moved, not which rows crossed it. A plan that quietly folded any of the three into a total would be this project's own complaint, one level up.

--at runs the clock at another instant, which makes "what would this policy have refused last Tuesday?" a real question — a deadline and a hold are both functions of time. It moves the clock and not the data: nothing here keeps history, and the documents are the ones on disk now.

Before anything is installed

Two questions, two costs, and neither one is a deployment.

"Where in my code is the hole?" — costs nothing at all. The scanner is a single stdlib file in its own distribution with no dependencies and no import of this package, because the first thing a stranger runs must cost them nothing:

python3 scanner/voyd_scan app/ services/

notes: `expire_at` (declared by a write or index); `forgotten` (declared by a write or index)

25 of 25 read(s) against them do not:
  refuse.py:79   notes  (filter does not name `expire_at`, `forgotten`)
  embed.py:163   notes  (filter does not name `expire_at`, `forgotten`)

It works out what the mark is rather than being told: a TTL index names its own field, and beyond that, the fields most reads filter on are the convention — so a field most reads name and some do not is a deviation from your own spec. It gets stronger on larger codebases, because the majority that establishes the convention is bigger.

"How much is exposed right now?" — costs a read-only URI. --audit compares the cluster against no policy at all, so every document the policy would refuse is a document reachable today:

The same arithmetic, asked about the present. --audit compares the cluster against no policy at all, so every document the policy would refuse is a document reachable right now:

voyd-plan --audit --proposed voydfile.py --target $READONLY_URI --all

reachable today, and refused by this policy

  records  1000 of 4100 read are reachable now and would be refused
         812  are past an expire_at the TTL monitor has not reached
         188  carry an erasure mark and are still being served

No proxy, no sidecar, no connection string changed, nothing in a query path — a read-only credential and a batch job. That is only possible because the check is a pure function: a boundary whose enforcement lives inside a running process has nowhere to stand to ask this.

what it costs you	what it tells you
nothing — one stdlib file	which reads in your code can serve a forgotten fact
a read-only URI	how many documents are reachable right now that should not be
one line in CI	what a policy change would let through, before it merges
one connection string	nothing gets out, in any language, ever again

In the place a policy is actually reviewed

A check nobody runs is a man page, so the plan ships as an action:

- uses: actions/checkout@v4
  with: { fetch-depth: 0 }   # the policy in force is read out of git
- uses: ranfysvalle02/VOYD@main   # or a tag, which pins both halves

That is the whole configuration. The action installs voyd from its own checkout rather than from PyPI, so pinning its ref pins the planner and the policy vocabulary together — a voydfile that loads in the check has to be one that loads at the boundary. It reads the voydfile at the pull request's base, compares it to the one on the branch, posts the plan as a comment it updates in place, and fails the job if the boundary opened. No cluster and no secret: the structural findings are facts about the policy, and they are the ones a reviewer is least equipped to see in a diff — a @guard deleted is one removed line.

Add target: ${{ secrets.MONGODB_URI }} and the findings get document counts beside them. fail-on-open: false makes it a reporter instead of a gate, which is a reasonable way to adopt it and a bad way to keep it.

A plan that could not be computed — a voydfile that will not load — exits 2 and fails the job before anything is posted, whatever fail-on-open says. A check that answered "nothing found" when it had not run is the one failure this could have that would be worse than not existing.

Whose access did it widen?

"The boundary widened" is the wrong granularity for a review. --as-each takes a table of callers and plans the same change once for each:

voyd-plan --current voydfile.py --proposed voydfile.new.py \
          --target $URI --database app --all --as-each roles.json

per caller
  caller          newly reachable   examined
  tier1-support               412        824
  analyst                       0        824
  clinician                     0        824

what tier1-support gains
  records       412  were refused as not_cleared

newly reachable: 412 (worst caller: tier1-support)

The headline is the worst caller, not the sum: one document reachable by four roles is one document that got out, not four, and a number that grew when somebody added a read-only role to the JSON would be measuring the role table. The data is read once however many callers there are — not an optimisation, a correctness requirement, since a second role handed an exhausted cursor reports zero and looks clean.

A plan somebody can still believe next year

--attest PATH writes the result, the SHA-256 of each policy file's contents, when it ran and what ran it. --sign env:NAME adds an HMAC over the whole envelope.

voyd-plan --verify plan.att.json --sign env:VOYD_ATTEST_KEY \
          --current voydfile.py --proposed voydfile.new.py

intact:  digest and signature both check out
current: the attested policies are the ones on disk
attested 2026-09-22T13:38:39+00:00: fails_open=True, newly reachable=412

Four outcomes, and they are deliberately four rather than a boolean:

intact	nothing has changed since it was produced
edited	the payload no longer matches its digest — caught with no key
wrong key or altered	the digest was recomputed; the signature was not
stale	intact, but a policy file has changed since. A different finding, and usually the more interesting one

What it proves. That the envelope has not changed since something holding the key produced it, and — via the digests — that it is a verdict about the exact bytes of those two files.

What it does not. That the plan ran against a real cluster, that the sample was representative, or who produced it. It is a symmetric MAC: anyone who can verify can forge. Evidence of integrity, never of origin. The key is taken as env:NAME or file:/path and never on the command line, because argv reaches the process table and the build log.

---

The server owns the encoding

An embedding is not a vector, it is a (vector, model) pair, and a vector without its model is an orphan. Comparing orphans does not fail — it returns a number between -1 and 1. Measured against a real embedding API, the same text, both 1024-wide, two generations of one vendor's model:

identical text, old model vs new       cosine -0.053
unrelated text, both on the new one    cosine +0.301

A model swap does not degrade ranking, it inverts it: unrelated text scores five times higher than the document you were looking for, with no error and no log. embedded_with(model) refuses the document. auto_embed(model) removes the way it comes to be wrong — the index holds text, mongot embeds it on write and embeds the query with the same model at read time, so nothing in your process ever computes a vector and nothing can drift from the index.

A client that sends its own queryVector at that collection has put the embedder back, through a driver that never read the policy file. The boundary refuses it by name and says which form works instead.

---

The erasure refusal cannot perform

Refusal binds this read path, so it has nothing to say about a replica, a snapshot, or the backup somebody restores next year — none of those run it. sealed() is how a field opts into being destroyable instead: ciphertext at rest under a key scoped to the tenant, so destroying the key makes every copy unreadable at once.

refusal          immediate            this application's read path
crypto erasure   ~60s (key cache)     every copy that exists anywhere
the TTL reaper   ~60s (measured)      this deployment's disk

Each one's window is the other's guarantee, which is the argument for having both rather than choosing. The boundary orders them accordingly: unreachable first, unreadable second.

sealed() without tenant() is refused at load. A scope with no name is one key for the whole collection, and destroying it to forget one subject would take every other tenant's rows with it.

---

Running the tests

uv sync --all-extras
uv run pytest -q -m 'not needs_mongo'   # the pure half: no database, ~0.2s
uv run --extra crypto pytest -q         # including the erasure path
docker compose up -d mongo              # a real mongod and a real mongot
uv run pytest -q                        # everything except `slow`
uv run pytest -q -m ""                  # everything

The pure files — the per-document check, every decision in the policy package, the planner that asks it about a policy nobody deployed, the wire codec, the custody rungs and the sweep below — need no database at all, and CI runs them in a step with no services:. If that step ever needs one, the boundary has stopped being pure.

The live tests drive a real voyd-wire in front of a real deployment with a plain pymongo client that has never heard of this package, because the claims are about queries and bytes and a mock would only prove the mock was filtered. Between them they cover refusal (expired, revoked, off-scope, and delete become a revocation, holding across every batch of a cursor); what a driver keeps when its connection string points here (sessions and causal consistency, transactions committed and aborted, several cursors interleaved on one socket, eight clients paging at once, and a real election caused with replSetStepDown and followed without a restart); the erasure a refusal cannot perform (a key destroyed, and the ciphertext read back by a reader that never held it); and a refusal travelling to what was made out of the fact, children marked before the source.

They need a cluster, which they find in VOYD_TEST_MONGO_URI, VOYD_MONGO_URI or a .env beside the compose file; with none of those they skip by name rather than passing quietly. The failover test needs the three-node set — docker compose up -d rs — because Atlas Local is a single node, so every routing assertion against it would pass by having nowhere else to go.

Every live test works in a database named voyd_test_<epoch>_<uuid> and drops it on the way out whatever happened — including the failures, since a cleanup that only runs on the happy path stops running exactly when a test starts leaving rows behind. The epoch in the name is what lets each run also sweep databases abandoned by an earlier one, bounded to a two-hour window so it can never reach a suite running concurrently on the same cluster.

The erasure tests destroy a real key and then try to read the ciphertext back, so they need pymongocrypt — the crypto extra, and nothing else. They deliberately do not need crypt_shared or mongocryptd: those are for automatic encryption, where the driver must analyse a command to learn which fields to encrypt, and this boundary has already parsed the command and already knows. Gating them on the stricter question is how they came to skip in CI, silently, on the claim this file leads with.

auto_embed against a cluster that really embeds is marked slow and reads VOYD_ATLAS_URI. Atlas Local registers no model, so it declines the declaration and falls back to an ordinary vector index — a different outcome, not a weaker one, and a test that accepted the fallback would assert the opposite of what it claims.

---

When you do not need this

• One service, one language, one read path, one person maintaining it. The filter in your query is the same guarantee and one fewer process.
• No deadlines, no revocation, no tenants — nothing to forget.
• A retrieval path that does not reach a model, where a wrong row is a bug rather than a disclosure.

The case for a boundary starts at the second read path, and it compounds at the second language.

---

Known gaps

• Nobody has run this but its author. Every number here comes from this repository's own benchmarks on one machine and one cluster.
• A boundary is a process, so it is a hop, a thing to deploy, and a thing that can be down. It follows a failover and re-resolves on the server's own NotWritablePrimary, but it is not a replacement for your driver's topology.
• A sealed read decrypts before it refuses, which costs the boundary its purity: holding keys makes it a custody holder.
• No policy file can see an index created somewhere else. sealed() on a field that an Atlas index auto-embeds is a contradiction nothing here can detect.
• voyd-plan --at replays the clock against today's documents. Answering it against the documents as they were needs history this package does not keep.
• A change stream over a guarded collection is refused, not filtered: a change event is not a document, and the rules read top-level fields. See docs/why-not-native.md.
• A transform cannot widen a read, and that is the only promise made about one. It can still be slow, wrong, or expensive, and nothing here bounds how long somebody's reranker runs inside the egress path.
• Backfill fills a lone $vectorSearch and nothing else. A pipeline with a stage after the search, a $search, or a $rankFusion still comes back short by whatever was refused, and a widened page that is mostly refused is short too.
• sanitized() catches the injections its signatures describe and no others. It reads strings and lists of strings; text nested deeper in a field is not examined. A collection declaring it cannot answer distinct, count or a reducing aggregate.
• A mask() closes the read paths this boundary can see. A write whose count depends on the value — update({ssn: X}, ...) answering nModified: 1 — is refused like a read, but an index built on the field still exists on the server, and a caller with a direct connection is outside every guarantee here. A refused filter is strict on purpose: a bare key named field, path or key whose value is a masked field's name is refused as if it were a stage argument.
• A read with a virtual step does not stream. The whole set is drained, held in memory, bounded by --virtual-max-docs, and answered as one batch; a $vectorSearch prefix gets neither backfill nor prefilter.
• Copies exist at rest. A temporary collection holds a caller's admitted, masked output for the length of the read, and for up to --virtual-max-age if the proxy dies mid-read. It is refused through the boundary, but a direct connection with rights on that database, the server's profiler and logs, and currentOp all see it.
• Virtual steps run with no time bound but the client's own. A slow one holds its connection's request loop; nothing here cancels it.
• A budget() or distinct() rule is re-asked of what a step returns, so fields a step adds count against the budget, and duplicate rows from an $unwind are judged as duplicates.
• The terminal pass after a step restores the fields the verdict reads from the admitted source, so a step cannot change them — even to a value the policy would accept.
• There is no observe-only mode. voyd-wire enforces or it is not in the path; it cannot yet run alongside a read logging what it would have refused. voyd-plan --audit answers most of that question without a proxy at all, which is why this has not been urgent.

---

Reading this repository

This file is the reference. The others are not the same argument at different lengths — each answers a question this one does not:

quickstart.md	four steps, smallest first; the first costs nothing at all
use-cases.md	twenty-two places a fact ranks well and must not reach the prompt, each with its voydfile and the edge where refusal stops
ethos.md	what a policy file is for, and the four tests that keep logic out of it. Read this before writing rules
blog.md	the story: every failure in this domain is disguised as its own opposite, including one in this project's own CI
docs/why-not-native.md	change streams, $where, views, $$USER_ROLES, TTL, RBAC, Queryable Encryption — what each one gives you and where the line is
docs/cosine.md	the embedding-model failure, reproducible without an API key, with its provenance and its limits
docs/ranking-is-not-permission.md	the long-form design argument for putting a boundary on the wire
genius.md	the commercial case: one property, three motions, and the corrections that keep it honest

MIT.