🇬🇧 English · 🇫🇷 Français
⬛ NoireBox
The black box for AI agents — tamper-evident journal, guardrails, verifiable proof.
Your AI agent writes meeting notes that bind your clients. Eighteen months from now — who can prove what it exactly produced, and why? NoireBox seals every decision in a SHA-256 hash-chained + Ed25519-signed journal, flags poisoned transcripts before they reach the agent, and exports an attestation anyone can verify offline — without trusting you.
Works with any agent, in French and English. The journal, the API, the MCP tools and the verifier are domain- and language-agnostic; each language gets a ~250 KB detector trained from a reproducible dataset (seed 42) — a new market = one dataset generator +
make train. Already have a guardrail? Keep it — it's just another event source. 🇪🇺 Built for the European market (GDPR & AI Act) — engineered in France.
Quickstart · Real LLM scene · Specs · Threat model · ADRs
Built for the European market (GDPR & AI Act)
European AI vendors face a compliance reality their US competitors don't: proving — not promising — what their systems did. NoireBox is engineered around exactly that obligation.
| Requirement | How NoireBox answers |
|---|---|
| GDPR art. 5(2) — accountability: the controller must demonstrate compliance | Tamper-evident journal of what the AI produced, when, on which input |
| GDPR art. 15/20 — data subject rights (access, portability) | Signed export of everything related to a meeting — verifiable by the subject's own auditor |
| EU AI Act art. 12 — automatic event logging for risk systems | Every agent decision sealed at runtime; log integrity is cryptographic, not a promise |
| DPIA / DPO workflows | Attestation exportable for the DPO; incident taxonomy feeding risk documentation |
| Sovereignty | Self-hosted, no telemetry, no cloud dependency, Ed25519 keys stay on your infrastructure — deploys anywhere (including EU-only clouds) |
Honest scope (we're a building block, not a certification): NoireBox proves the integrity of your AI's record. Data minimization of payloads and HSM-grade key storage are the deployer's responsibility in v0 — see the threat model. A tool that oversells compliance is a liability; one that states its perimeter is auditable.
Why
- GDPR: a platform claiming "GDPR-compliant" must be able to demonstrate what its AI produced, on which data, and when.
- EU AI Act (art. 12): risk systems must keep automatic event logs. Today, virtually every LLM stack logs for debugging — not for proof.
- The market gap: observability tools (Langfuse, LangSmith…) are built for devs, remain modifiable after the fact, and produce nothing presentable to an auditor, a DPO or a client.
Honest positioning (state of the art)
| Tool | What it does | What it doesn't |
|---|---|---|
| Langfuse / LangSmith / Helicone | LLM observability, debugging, traces | No tamper-evident proof, nothing exportable for an audit |
| Garak / PyRIT / promptfoo | offline model-level red-teaming | No runtime guardrail, no proof journal |
| halo-record / gate-oc-audit | hash-chained journals for coding agents | No business domain, no third-party attestation, no French |
| NoireBox | immutable Ed25519 journal → RFC 3161 anchoring → exportable attestation + standalone verifier (bundled FR/EN guardrail = one pluggable event source; fleet anchoring: one TSA seal covers N journals) | — |
NoireBox applies the Certificate-Transparency model to AI-agent journals:
many journals, one root signed by a timestamp authority, inclusion proofs that
anyone verifies offline (noirebox/merkle.py, make demo-fleet). To our
knowledge, no other agent-audit tool does this.
The core is the journal. The guardrail is a plugin.
No ambiguity about what the product is:
- One NoireBox per agent or service — like one flight recorder per
aircraft. One instance = one SQLite file, one key pair, one chain. The
fleet layer (
noirebox/merkle.py) aggregates proofs (32-byte chain heads), never events: your journal never leaves your infrastructure. - The journal is the product — chain, signatures, append-only storage,
RFC 3161 anchoring, export, standalone verifier. It is domain-agnostic,
agent-agnostic and language-agnostic: an event is a free-typed
typeplus a payload, nothing more.chain.py/store.py/anchors.pyimport zero detection code — delete the whole guardrail and everything still runs. - Prevention is pluggable — a guardrail is just one event producer
among others. Already have Lakera, Llama Guard, your own LLM-judge, your
own regexes? Keep them. Journal their verdicts the same way
(
POST /api/v1/events,type: "incident") and their catches become tamper-evident and third-party verifiable instead of rewritable app logs. - NoireBox ships a bundled guardrail (regex + trained ML) as a working example of that plugin contract — and as a convenience if you have none. It is swappable by design (ADR 003).
Prevention varies per stack. Proof is universal.
Quickstart
./start.sh # venv + deps + tests + API on http://127.0.0.1:8768/docs
make demo # the full story in one command (see below)
Or with Docker — no Python needed, the ML engine ships inside the image:
docker run -p 8768:8768 ghcr.io/slabbdev/noirebox:latest
# API + OpenAPI docs on http://127.0.0.1:8768/docs — data persists in ./data
Standalone demos:
make demo # 100% real: micro-model → journal → auditor → caught pirate
make demo-mcp # NoireBox as an MCP tool (agent protocol)
.venv/bin/python demo/demo_scan.py # regex guardrail on 2 transcripts
.venv/bin/python demo/demo_tamper.py # tampering → the chain explodes
Verify an export as a third party (auditor, DPO, client):
curl -s http://127.0.0.1:8768/api/v1/export > export.json
.venv/bin/python verifier/verifier.py export.json # exit 0 = chain intact
Chain timestamping — the outside witness (RFC 3161)
The journal proves integrity, but when was it sealed? A server announcing its own dates is the suspect writing its own report. And the threat model had one open gap: an operator holding the private key could regenerate the whole chain with valid signatures.
The anchor closes both. One call seals the current chain head with a TSA
(Timestamp Authority, RFC 3161): only the 32-byte hash leaves (zero data,
zero GDPR exposure), the TSA signs "I received hash X at time T", and the
token is journaled as an anchor event — the journal seals its own external
proof. A regenerated chain shows a head the old token doesn't cover: caught
at verification time, without any prior external publication.
make tsa # local TSA: OpenSSL, own key, 0 €, works offline
NOIREBOX_TSA_URL=http://127.0.0.1:3318 ./start.sh
curl -X POST localhost:8768/api/v1/anchors # seal the current head
TSA is a config choice, not a dependency: self-hosted OpenSSL for sovereign deployments, any public or qualified TSA for production — same protocol. Full story and visuals: docs/VULGARISATION.md §9, decisions in ADR 006/007.
The bundled guardrail — one plugin, two engines
| Engine | Size | Languages | Role | Dependency |
|---|---|---|---|---|
regex |
~0 | FR+EN | obvious cases, zero cost | none |
ml |
243–293 KB | fr & en (lang param) |
paraphrases the regex misses, local | scikit-learn |
Architecture decision — ADR 002: we evaluated Meta's
Llama Prompt Guard 2 (the industry classifier, ~90 MB) and rejected it
knowingly: gated license, torch/transformers as dependencies, and binary
output without an audit taxonomy. The ADR also documents the integration
path if you ever need it (adapter ≈ 40 lines behind the same engine
interface). The tier-2 judge stays a local LLM (llama-guard3:1b via
Ollama) — same binary we already ship, zero new deps.
The real LLM scene (local Ollama)
A real LLM (qwen2.5:0.5b, 397 MB, local via Ollama — never committed,
fetched by ollama pull) receives the poisoned transcript:
brew install ollama && ollama serve && ollama pull qwen2.5:0.5b # once
make demo-llm # the scene
Measured outcome (temperature 0, reproducible):
- Without NoireBox: the model rewrites the attacker's instructions into
"its" own summary — competitor's email and
DROP TABLE utilisateursincluded. The attack succeeds. - With NoireBox: 4 poisoned lines removed before the call, incident sealed in the journal, and the model's real output is clean.
See also deploy/DEPLOIEMENT.md for €0 deployment (instant cloudflared tunnel, permanent Hugging Face Space) or a €3–6/mo VPS.
The ML micro-detector — trained in-repo, two languages, ~250 KB each
make train # FR: gen_dataset.py (5,400 examples) → detector.joblib
make train-en # EN: gen_dataset_en.py (5,400 examples) → detector_en.joblib
A new language is a dataset, not a rewrite: the trainer, the registry and
the API (lang: "fr" | "en") are shared. Adding Spanish = one
gen_dataset_es.py + ml/train.py --lang es.
Honest evaluation (the part that earns credit in reviews):
- the test sets share templates with their train sets → accuracy 1.000 measures consistency, not generalization;
- the generalization proof lives elsewhere: held-out sentences (slang,
typos, never-seen formulations) are frozen in
tests/test_ml_guardrail.py— in both languages — correct category detected, zero false positives on trap-clean sentences ("send the report to the accountant", "I rotated my password"); - the EN held-out suite caught a real weakness in v1 ("take orders from me" scored 0.37) → dataset extended → model retrained → suite green. That loop is the ML workflow, and it's in the git history.
| Artifact | Size |
|---|---|
models/detector.joblib + models/detector_en.joblib (versioned) |
293 + 243 KB |
data/dataset.jsonl + data/dataset_en.jsonl (versioned) |
~1.3 MB |
Plugging NoireBox in — 3 real ways
1. REST (any language):
curl -X POST https://noirebox.example.com/api/v1/transcripts/scan \
-H 'Content-Type: application/json' \
-d '{"meeting_id": "MTG-42", "text": "<transcript>", "engine": "ml"}'
2. Python SDK (noirebox/client.py):
from noirebox.client import NoireBoxClient
nb = NoireBoxClient("https://noirebox.example.com")
nb.scan("MTG-42", transcript) # guardrail before the agent
nb.log_event("llm_output", {...}) # seal the agent's output
nb.verify() # check the chain
3. MCP (the agent protocol — Claude Desktop and every MCP client):
4 tools exposed (noirebox_scan, noirebox_log_event, noirebox_verify,
noirebox_attestation). Config (claude_desktop_config.json):
{ "mcpServers": { "noirebox": {
"command": "/path/to/noirebox/.venv/bin/python",
"args": ["-m", "noirebox.mcp_server"] } } }
Stdio JSON-RPC implementation with no external SDK: the protocol stays
readable end to end (demo/demo_mcp.py).
Use cases — any agent that "decides"
| Agent | What NoireBox brings |
|---|---|
| Meeting assistant (demo scenario) | provable notes + transcript anti-injection |
| Coding agent | immutable journal of executed actions (files, commands) |
| AI customer support | proof of what was promised to the client, when and why |
| Legal / medical RAG pipeline | traceability of sources used and outputs produced |
| Business copilots (finance, HR…) | exportable attestation for internal audit / DPO / client |
The journal is universal (free-typed events). The shipped guardrail
corpus specializes in FR meeting transcripts — extending it to another
domain = adding examples to the dataset and re-running make train.
The code — core first, plugins after
noirebox/ ← package (≈ PSR-4 namespace)
├── chain.py THE CORE: SHA-256 hash chain + Ed25519 signatures
├── store.py THE CORE: append-only SQLite, parameterized queries, lock
├── attestation.py THE CORE: signed digest of the chain state
├── anchors.py THE CORE: RFC 3161 anchoring (TSA witness)
├── pdf_export.py THE CORE: DPO-ready attestation PDF
├── main.py FastAPI routes (the HTTP layer)
├── schemas.py Pydantic DTOs (request validation)
├── client.py Python SDK
├── mcp_server.py MCP tools server (stdio JSON-RPC)
├── guardrail.py PLUGIN: 4 FR/EN attack categories caught by regex
├── ml_guardrail.py PLUGIN: trained micro-models (fr + en, ~250 KB each)
└── llm_agent.py PLUGIN DEMO: real Ollama agent behind the guarded pipeline
API
| Route | Auth | Role |
|---|---|---|
POST /api/v1/token |
— | Exchange client_id/client_secret for a 1 h JWT |
POST /api/v1/events |
🔒 | Record an event (prompt, output, eval…) |
GET /api/v1/events |
🔒 | Paginated event list |
GET /api/v1/verify |
open | Verify the chain in place |
POST /api/v1/transcripts/scan |
🔒 | Guardrail: detect injections, journal the incident |
GET /api/v1/attestation |
open | Signed attestation of the current state |
GET /api/v1/attestation.pdf |
open | Attestation as a DPO-ready A4 PDF |
POST /api/v1/attestation/verify |
open | Verify a submitted attestation |
POST /api/v1/anchors |
🔒 | RFC 3161 anchor: seal the current chain head with a TSA |
GET /api/v1/export |
🔒 | Full auditable export |
🔒 = requires Authorization: Bearer <token> when NOIREBOX_CLIENTS=id:secret,…
is set; verification routes stay open by design — one never locks the
verification (ADR 004). Rate limit: 60 req/min per client.
Bundled plugin — v0 attack taxonomy
instruction_override— "ignore all instructions", "you are now…"data_exfiltration— "send … to email@… / https://…"pii_request— "give me the passwords / banking data"tool_abuse— "run DROP TABLE / rm -rf / curl"
See corpus/attaques.json. Extending = add regex patterns or dataset
examples, then make train.
Tests
91 tests: cryptography (tampering, reordering, wrong key), regex and ML
guardrails on held-out sentences in FR and EN, API agent, MCP server, SDK
client against a real uvicorn server (ephemeral port), real LLM agent
(skipped if Ollama is absent — never simulated), OAuth2 JWT auth + rate
limiting + DPO-ready PDF attestation + RFC 3161 anchoring against a real
local TSA (incl. the insider chain-regeneration attack), and the full
third-party verifier contract. Everything replays locally: make test
re-trains both language models and runs the full suite.
Consuming AI-agent decisions in your own pipeline? Gate your builds on the integrity of the journal — the verifier ships as a GitHub Action (Marketplace):
- uses: slabbdev/noirebox-verify@v1
with:
export-path: export.json
Roadmap
- OAuth2 (JWT bearer) + rate limiting — ADR 004
- PDF attestation export for DPOs — ADR 005
- Local LLM judge for doubtful cases — tier 2 of ADR 001
- RFC 3161 timestamping of the chain head — ADR 006/007, self-hosted TSA included
- Fleet anchoring (Merkle) —
noirebox/merkle.py: one TSA seal covers N journals (Certificate-Transparency pattern); inclusion proofs are ~log2(N) hashes, verified offline — seemake demo-fleet - Reconciliation plugin — invariants over the journal ("every decision
must have an outcome"): two-event pattern sketch in
demo/demo_payout.py(make demo-payout), schema discussion in issue #3 - Fleet hub: scheduled aggregation of many instances (console, alerting)
- Rotate TSA anchors across multiple authorities (distribute trust)
- Prometheus + Grafana metrics
- HSM/KMS private key migration (threat model)
- Product page rebuilt for GitHub Pages — premium dark landing in
docs/, auto-deployed by.github/workflows/pages.yml(enable Pages → Source: GitHub Actions)
Support
NoireBox is free and MIT — verification included, forever. If it saved you time (or a compliance headache), a coffee is the best way to say it helps:
Scan or click — buymeacoffee.com/samlabbe
License
MIT — see LICENSE.
Release files for noirebox 0.3.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| noirebox-0.3.0.tar.gz | 714.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| noirebox-0.3.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 1.3 MB
Release files / noirebox-0.3.0.tar.gz
| Download URL | noirebox-0.3.0.tar.gz |
|---|---|
| Size | 714.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
843d103891b26df7fee164652265eaddf435153b67f681e0b988eb90b2c967ba
|
|
BLAKE2b-256 checksum How to use checksums |
9996b01678dba8bcc1207d1375f06145d4a1b444ca4edd5e5a70d54ffcc79dae
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.7
|
Release files / noirebox-0.3.0-py3-none-any.whl
| Download URL | noirebox-0.3.0-py3-none-any.whl |
|---|---|
| Size | 587.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
ad4deaa05ba133d0a2916d19967cdde812850ec1e10f492f87e44494f7bb14a9
|
|
BLAKE2b-256 checksum How to use checksums |
51aec22dfe8da45800e0c957a6718080f303465c89f1c8dea755a91e76ce2184
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.14.7
|