Skip to main content

🇬🇧 English · 🇫🇷 Français

⬛ NoireBox

The black box for AI agents — tamper-evident journal, guardrails, verifiable proof.

PyPI Python License: MIT Lint: ruff Image on GHCR Made in France

Your AI agent writes meeting notes that bind your clients. Eighteen months from now — who can prove what it exactly produced, and why? NoireBox seals every decision in a SHA-256 hash-chained + Ed25519-signed journal, flags poisoned transcripts before they reach the agent, and exports an attestation anyone can verify offline — without trusting you.

Works with any agent, in French and English. The journal, the API, the MCP tools and the verifier are domain- and language-agnostic; each language gets a ~250 KB detector trained from a reproducible dataset (seed 42) — a new market = one dataset generator + make train. Already have a guardrail? Keep it — it's just another event source. 🇪🇺 Built for the European market (GDPR & AI Act) — engineered in France.

Quickstart · Real LLM scene · Specs · Threat model · ADRs


Built for the European market (GDPR & AI Act)

European AI vendors face a compliance reality their US competitors don't: proving — not promising — what their systems did. NoireBox is engineered around exactly that obligation.

Requirement How NoireBox answers
GDPR art. 5(2) — accountability: the controller must demonstrate compliance Tamper-evident journal of what the AI produced, when, on which input
GDPR art. 15/20 — data subject rights (access, portability) Signed export of everything related to a meeting — verifiable by the subject's own auditor
EU AI Act art. 12 — automatic event logging for risk systems Every agent decision sealed at runtime; log integrity is cryptographic, not a promise
DPIA / DPO workflows Attestation exportable for the DPO; incident taxonomy feeding risk documentation
Sovereignty Self-hosted, no telemetry, no cloud dependency, Ed25519 keys stay on your infrastructure — deploys anywhere (including EU-only clouds)

Honest scope (we're a building block, not a certification): NoireBox proves the integrity of your AI's record. Data minimization of payloads and HSM-grade key storage are the deployer's responsibility in v0 — see the threat model. A tool that oversells compliance is a liability; one that states its perimeter is auditable.

Why

  • GDPR: a platform claiming "GDPR-compliant" must be able to demonstrate what its AI produced, on which data, and when.
  • EU AI Act (art. 12): risk systems must keep automatic event logs. Today, virtually every LLM stack logs for debugging — not for proof.
  • The market gap: observability tools (Langfuse, LangSmith…) are built for devs, remain modifiable after the fact, and produce nothing presentable to an auditor, a DPO or a client.

Honest positioning (state of the art)

Tool What it does What it doesn't
Langfuse / LangSmith / Helicone LLM observability, debugging, traces No tamper-evident proof, nothing exportable for an audit
Garak / PyRIT / promptfoo offline model-level red-teaming No runtime guardrail, no proof journal
halo-record / gate-oc-audit hash-chained journals for coding agents No business domain, no third-party attestation, no French
NoireBox immutable Ed25519 journal → RFC 3161 anchoring → exportable attestation + standalone verifier (bundled FR/EN guardrail = one pluggable event source; fleet anchoring: one TSA seal covers N journals) —

NoireBox applies the Certificate-Transparency model to AI-agent journals: many journals, one root signed by a timestamp authority, inclusion proofs that anyone verifies offline (noirebox/merkle.py, make demo-fleet). To our knowledge, no other agent-audit tool does this.

The core is the journal. The guardrail is a plugin.

NoireBox architecture

No ambiguity about what the product is:

  • One NoireBox per agent or service — like one flight recorder per aircraft. One instance = one SQLite file, one key pair, one chain. The fleet layer (noirebox/merkle.py) aggregates proofs (32-byte chain heads), never events: your journal never leaves your infrastructure.
  • The journal is the product — chain, signatures, append-only storage, RFC 3161 anchoring, export, standalone verifier. It is domain-agnostic, agent-agnostic and language-agnostic: an event is a free-typed type plus a payload, nothing more. chain.py/store.py/anchors.py import zero detection code — delete the whole guardrail and everything still runs.
  • Prevention is pluggable — a guardrail is just one event producer among others. Already have Lakera, Llama Guard, your own LLM-judge, your own regexes? Keep them. Journal their verdicts the same way (POST /api/v1/events, type: "incident") and their catches become tamper-evident and third-party verifiable instead of rewritable app logs.
  • NoireBox ships a bundled guardrail (regex + trained ML) as a working example of that plugin contract — and as a convenience if you have none. It is swappable by design (ADR 003).

Prevention varies per stack. Proof is universal.

Quickstart

pip install noirebox       # the ML engine ships inside the wheel (FR + EN)
pip install noirebox[pdf]  # + the DPO-ready PDF attestation (optional extra)
noirebox serve             # API on http://127.0.0.1:8768/docs

From source:

./start.sh          # venv + deps + tests + API on http://127.0.0.1:8768/docs
make demo           # the full story in one command (see below)

Or with Docker — no Python needed, the ML engine ships inside the image:

docker run -p 8768:8768 ghcr.io/slabbdev/noirebox:latest
# API + OpenAPI docs on http://127.0.0.1:8768/docs — data persists in ./data

Standalone demos:

make demo                # 100% real: micro-model → journal → auditor → caught pirate
make demo-mcp            # NoireBox as an MCP tool (agent protocol)
.venv/bin/python demo/demo_scan.py     # regex guardrail on 2 transcripts
.venv/bin/python demo/demo_tamper.py   # tampering → the chain explodes

Verify an export as a third party (auditor, DPO, client):

curl -s http://127.0.0.1:8768/api/v1/export > export.json
.venv/bin/python verifier/verifier.py export.json   # exit 0 = chain intact

The verifier needs the NoireBox package on the auditing machine — pip install noirebox is enough (no model, no framework).

Chain timestamping — the outside witness (RFC 3161)

The journal proves integrity, but when was it sealed? A server announcing its own dates is the suspect writing its own report. And the threat model had one open gap: an operator holding the private key could regenerate the whole chain with valid signatures.

The anchor closes both. One call seals the current chain head with a TSA (Timestamp Authority, RFC 3161): only the 32-byte hash leaves (zero data, zero GDPR exposure), the TSA signs "I received hash X at time T", and the token is journaled as an anchor event — the journal seals its own external proof. A regenerated chain shows a head the old token doesn't cover: caught at verification time, without any prior external publication.

make tsa                                    # local TSA: OpenSSL, own key, 0 €, works offline
NOIREBOX_TSA_URL=http://127.0.0.1:3318 ./start.sh
curl -X POST localhost:8768/api/v1/anchors  # seal the current head

TSA is a config choice, not a dependency: self-hosted OpenSSL for sovereign deployments, any public or qualified TSA for production — same protocol. Full story and visuals: docs/VULGARISATION.md §9, decisions in ADR 006/007.

The bundled guardrail — one plugin, two engines

Engine Size Languages Role Dependency
regex ~0 FR+EN obvious cases, zero cost none
ml 243–293 KB fr & en (lang param) paraphrases the regex misses, local scikit-learn

Architecture decision — ADR 002: we evaluated Meta's Llama Prompt Guard 2 (the industry classifier, ~90 MB) and rejected it knowingly: gated license, torch/transformers as dependencies, and binary output without an audit taxonomy. The ADR also documents the integration path if you ever need it (adapter ≈ 40 lines behind the same engine interface). The tier-2 judge stays a local LLM (llama-guard3:1b via Ollama) — same binary we already ship, zero new deps.

The real LLM scene (local Ollama)

A real LLM (qwen2.5:0.5b, 397 MB, local via Ollama — never committed, fetched by ollama pull) receives the poisoned transcript:

brew install ollama && ollama serve && ollama pull qwen2.5:0.5b   # once
make demo-llm                                                     # the scene

Measured outcome (temperature 0, reproducible):

  • Without NoireBox: the model rewrites the attacker's instructions into "its" own summary — competitor's email and DROP TABLE utilisateurs included. The attack succeeds.
  • With NoireBox: 4 poisoned lines removed before the call, incident sealed in the journal, and the model's real output is clean.

See also deploy/DEPLOIEMENT.md for €0 deployment (instant cloudflared tunnel, permanent Hugging Face Space) or a €3–6/mo VPS.

The ML micro-detector — trained in-repo, two languages, ~250 KB each

make train        # FR: gen_dataset.py (5,400 examples) → detector.joblib
make train-en     # EN: gen_dataset_en.py (5,400 examples) → detector_en.joblib

A new language is a dataset, not a rewrite: the trainer, the registry and the API (lang: "fr" | "en") are shared. Adding Spanish = one gen_dataset_es.py + ml/train.py --lang es.

Honest evaluation (the part that earns credit in reviews):

  • the test sets share templates with their train sets → accuracy 1.000 measures consistency, not generalization;
  • the generalization proof lives elsewhere: held-out sentences (slang, typos, never-seen formulations) are frozen in tests/test_ml_guardrail.py — in both languages — correct category detected, zero false positives on trap-clean sentences ("send the report to the accountant", "I rotated my password");
  • the EN held-out suite caught a real weakness in v1 ("take orders from me" scored 0.37) → dataset extended → model retrained → suite green. That loop is the ML workflow, and it's in the git history.
Artifact Size
models/detector.joblib + models/detector_en.joblib (versioned) 293 + 243 KB
data/dataset.jsonl + data/dataset_en.jsonl (versioned) ~1.3 MB

Plugging NoireBox in — 3 real ways

1. REST (any language):

curl -X POST https://noirebox.example.com/api/v1/transcripts/scan \
     -H 'Content-Type: application/json' \
     -d '{"meeting_id": "MTG-42", "text": "<transcript>", "engine": "ml"}'

2. Python SDK (noirebox/client.py):

from noirebox.client import NoireBoxClient
nb = NoireBoxClient("https://noirebox.example.com")
nb.scan("MTG-42", transcript)          # guardrail before the agent
nb.log_event("llm_output", {...})      # seal the agent's output
nb.verify()                            # check the chain

3. MCP (the agent protocol — Claude Desktop and every MCP client): 4 tools exposed (noirebox_scan, noirebox_log_event, noirebox_verify, noirebox_attestation). Config (claude_desktop_config.json):

{ "mcpServers": { "noirebox": {
    "command": "/path/to/noirebox/.venv/bin/python",
    "args": ["-m", "noirebox.mcp_server"] } } }

Stdio JSON-RPC implementation with no external SDK: the protocol stays readable end to end (demo/demo_mcp.py).

Use cases — any agent that "decides"

Agent What NoireBox brings
Meeting assistant (demo scenario) provable notes + transcript anti-injection
Coding agent immutable journal of executed actions (files, commands)
AI customer support proof of what was promised to the client, when and why
Legal / medical RAG pipeline traceability of sources used and outputs produced
Business copilots (finance, HR…) exportable attestation for internal audit / DPO / client

The journal is universal (free-typed events). The shipped guardrail corpus specializes in FR meeting transcripts — extending it to another domain = adding examples to the dataset and re-running make train.

The code — core first, plugins after

noirebox/            ← package (≈ PSR-4 namespace)
├── chain.py         THE CORE: SHA-256 hash chain + Ed25519 signatures
├── store.py         THE CORE: append-only SQLite, parameterized queries, lock
├── attestation.py   THE CORE: signed digest of the chain state
├── anchors.py       THE CORE: RFC 3161 anchoring (TSA witness)
├── pdf_export.py    THE CORE: DPO-ready attestation PDF
├── main.py          FastAPI routes (the HTTP layer)
├── schemas.py       Pydantic DTOs (request validation)
├── client.py        Python SDK
├── mcp_server.py    MCP tools server (stdio JSON-RPC)
├── guardrail.py     PLUGIN: 4 FR/EN attack categories caught by regex
├── ml_guardrail.py  PLUGIN: trained micro-models (fr + en, ~250 KB each)
└── llm_agent.py     PLUGIN DEMO: real Ollama agent behind the guarded pipeline

API

Route Auth Role
POST /api/v1/token — Exchange client_id/client_secret for a 1 h JWT
POST /api/v1/events 🔒 Record an event (prompt, output, eval…)
GET /api/v1/events 🔒 Paginated event list
GET /api/v1/verify open Verify the chain in place
POST /api/v1/transcripts/scan 🔒 Guardrail: detect injections, journal the incident
GET /api/v1/attestation open Signed attestation of the current state
GET /api/v1/attestation.pdf open Attestation as a DPO-ready A4 PDF
POST /api/v1/attestation/verify open Verify a submitted attestation
POST /api/v1/anchors 🔒 RFC 3161 anchor: seal the current chain head with a TSA
GET /api/v1/export 🔒 Full auditable export

🔒 = requires Authorization: Bearer <token> when NOIREBOX_CLIENTS=id:secret,… is set; verification routes stay open by design — one never locks the verification (ADR 004). Rate limit: 60 req/min per client.

Bundled plugin — v0 attack taxonomy

  • instruction_override — "ignore all instructions", "you are now…"
  • data_exfiltration — "send … to email@… / https://…"
  • pii_request — "give me the passwords / banking data"
  • tool_abuse — "run DROP TABLE / rm -rf / curl"

See corpus/attaques.json. Extending = add regex patterns or dataset examples, then make train.

Tests

91 tests: cryptography (tampering, reordering, wrong key), regex and ML guardrails on held-out sentences in FR and EN, API agent, MCP server, SDK client against a real uvicorn server (ephemeral port), real LLM agent (skipped if Ollama is absent — never simulated), OAuth2 JWT auth + rate limiting + DPO-ready PDF attestation + RFC 3161 anchoring against a real local TSA (incl. the insider chain-regeneration attack), and the full third-party verifier contract. Everything replays locally: make test re-trains both language models and runs the full suite.

Consuming AI-agent decisions in your own pipeline? Gate your builds on the integrity of the journal — the verifier ships as a GitHub Action (Marketplace):

- uses: slabbdev/noirebox-verify@v1
  with:
    export-path: export.json

Roadmap

  • OAuth2 (JWT bearer) + rate limiting — ADR 004
  • PDF attestation export for DPOs — ADR 005
  • Local LLM judge for doubtful cases — tier 2 of ADR 001
  • RFC 3161 timestamping of the chain head — ADR 006/007, self-hosted TSA included
  • Fleet anchoring (Merkle) — noirebox/merkle.py: one TSA seal covers N journals (Certificate-Transparency pattern); inclusion proofs are ~log2(N) hashes, verified offline — see make demo-fleet
  • Reconciliation plugin — invariants over the journal ("every decision must have an outcome"): two-event pattern sketch in demo/demo_payout.py (make demo-payout), schema discussion in issue #3
  • Fleet hub: scheduled aggregation of many instances (console, alerting)
  • Rotate TSA anchors across multiple authorities (distribute trust)
  • Prometheus + Grafana metrics
  • HSM/KMS private key migration (threat model)
  • Product page rebuilt for GitHub Pages — premium dark landing in docs/, auto-deployed by .github/workflows/pages.yml (enable Pages → Source: GitHub Actions)

Support

NoireBox is free and MIT — verification included, forever. If it saved you time (or a compliance headache), a coffee is the best way to say it helps:

Buy Me A Coffee — scan to support NoireBox

Scan or click — buymeacoffee.com/samlabbe

License

MIT — see LICENSE.

Release files for noirebox 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for noirebox 0.4.0
File Size Uploaded
noirebox-0.4.0.tar.gz 719.4 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for noirebox 0.4.0
File Interpreter ABI Platform
noirebox-0.4.0-py3-none-any.whl Python 3 none any Details

Total release size: 1.3 MB

Release files / noirebox-0.4.0.tar.gz

Download URL noirebox-0.4.0.tar.gz
Size 719.4 kB
Tags Source
SHA-256 checksum
How to use checksums
64054863b17179526551c69706eac2f57536cbe14bc12c32ca658335bfe78551
BLAKE2b-256 checksum
How to use checksums
2b1d9e505f69d736c4f38c2440ef2bd620b78154d0b76c8d0f95ecf42901d9e5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.7

Release files / noirebox-0.4.0-py3-none-any.whl

Download URL noirebox-0.4.0-py3-none-any.whl
Size 591.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
f3be0787d27f5c7bfb9c527be4373c110c964b3e49cd8779b70b8dfa00a1ea3c
BLAKE2b-256 checksum
How to use checksums
9ac087acff7cbcff5c3acf4b906b2bc8115501cc271a4af077bba7f52d5039a8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.14.7

Release history Release notifications | RSS feed

0.6.0

2 release files

0.5.0

2 release files

This release

0.4.0 This release

2 release files

0.3.0

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page