Full Benchmark Report • Reproduce the Benchmarks • Production Readiness • Measured Limitations

Benchmarks start with source files and report retrieval, delivery, ingestion, and latency separately.

nDCG@10 measures ranking quality in the first ten results; it is not an accuracy percentage. Full methodology, component results, timings, and interpretation are in the benchmark report.

Evidence Pack Contract • Evidence Signals

EvidencePack is the output contract of fitz-sage.

It gives you the relevant sources and the governance signals around them. You can show it directly, pass it to a model, trigger a workflow from it, or store it as an audit artifact.

The source items are the evidence. The signals around them explain how Fitz searched before retrieval and what Pyrrho judged after retrieval.

Pre-retrieval 🔎

Before retrieval, Fitz builds a search plan from deterministic query analysis, managed Qwen query keywords, and optional query intelligence.

Post-retrieval 🛡️

After retrieval, reranking, closure, and compilation, Fitz sends Pyrrho the first three ranked items. An exact INSUFFICIENT verdict adds the next two; SUFFICIENT or DISPUTED stops immediately. These signals tell you whether the result is usable.

This is why fitz-sage is useful as infrastructure: the package returns source evidence plus enough judgment to decide the next action.

Retrieval execution records

When an EvidencePack is not enough to diagnose a result, RetrievalRun records the actual query plan, term origins, candidate stages, compiled ranking, evaluated evidence prefixes, exact Pyrrho outputs, and runtime fingerprints from the same execution.

fitz retrieve "Which test failed?" -c reports --trace run.json
fitz explain run.json

Trace exports redact source bodies by default. Content-bearing traces are an explicit opt-in and enable Pyrrho-only replay over frozen evidence. See Retrieval Execution Records.

Feature docs • Pyrrho on Hugging Face • fitz-gov on Hugging Face

Pyrrho is the local governance model behind fitz-sage. Its default CPU-local ONNX ModernBERT model yafitzdev/pyrrho-v2-nano-g1 is pinned to an immutable Hub revision and cached by Fitz-Sage through the standard Hugging Face cache.

  Query
    │
    ▼
  RetrievalProfile → broad recall → rerank
    │
    ▼
  Ranked evidence prefix (first 3)
    │
    ▼
  Pyrrho authoritative decision
    ├── SUFFICIENT / DISPUTED ───────────────────────→ EvidencePack
    ├── INSUFFICIENT + exhausted ────────────────────→ EvidencePack
    └── INSUFFICIENT + evidence remains → add next 2 ──┐
                ▲                                      │
                └──────────────────────────────────────┘

Fitz-Sage passes every ranked prefix to Pyrrho unchanged. Its managed ONNX adapter applies the model's fixed input and head-decoding contract, maps the resulting verdict into AnswerMode, and returns the stopping prefix plus the exact serialized decisions with the EvidencePack. Applications can answer, retry, show conflict, or request more source material.

[!NOTE] Governance is a source-evidence judgment. Pyrrho is trained to decide whether retrieved evidence is sufficient, disputed, or insufficient, and Fitz records that judgment in the returned metadata.

The model adapter fails closed on known contract violations 🛡️

Fitz-Sage's managed Pyrrho adapter checks the model artifact, label order, ONNX width, token limits, graph parity, and verdict/failure compatibility before or during inference. These checks reduce unsafe failure modes; they are not a substitute for clean-data evaluation or threshold calibration.

No LLM on the governance path ⏱️

Pyrrho is a local encoder forward pass. Governance does not require an external chat model.

Fitz-Sage targets reasonably clean, supported documents. The complete contract and case-level evidence are in docs/LIMITATIONS.md.

governance: pyrrho uses the accepted immutable default. Advanced users may instead configure pyrrho/<absolute-local-path> or an explicitly pinned pyrrho/<owner/repo@40-character-commit>.

CLI

pip install fitz-sage

fitz retrieve "Your question here" --source ./docs

fitz-sage creates a local retrieval config on first run:

1. SQLite storage for collections.
2. Managed local models for semantic query terms, reranking, governance, and optional background enrichment.
3. Pyrrho query planning plus one authoritative evidence decision.

For generated prose from the governed evidence:

fitz answer "..." --endpoint http://localhost:8080/v1 \
                --synthesizer endpoint/gpt-oss-20b
fitz answer "..." --endpoint https://api.together.xyz/v1 \
                --synthesizer endpoint/meta-llama-3.1-70b \
                --api-key-env TOGETHER_API_KEY

Python SDK

import fitz_sage

pack = fitz_sage.evidence("Where is Pyrrho governance implemented?", source="./fitz_sage")

print(pack.mode)
for item in pack.items:
   print(item.file_path, item.address_location)

The SDK provides:

• Module-level evidence() matching fitz retrieve
• Module-level answer() for generated prose from evidence
• Local config creation
• Full provenance tracking
• Governance metadata

For advanced use with multiple collections:

from fitz_sage import fitz

physics = fitz(collection="physics")
pack = physics.evidence("Explain entanglement", source="./physics_papers")

Fully Local (Managed ONNX)

pip install fitz-sage

fitz retrieve "Your question here" --source ./docs

With the default retrieval-only config, reranking, governance, query-time expansion, and optional background enrichment run locally, so fitz retrieve does not send data to an endpoint. Explicitly configured query intelligence, chat tiers, or vision parsing can send query or source content to that endpoint.

Optional synthesis can use vLLM, LM Studio, Ollama in /v1/ mode, TabbyAPI, OpenAI, Together, Groq, Fireworks, OpenRouter, or any endpoint that speaks the OpenAI HTTP protocol.

fitz-sage is a retrieval foundation. It manages indexing, search, reranking, Pyrrho integration, and provenance so products can build on source evidence.

Chatbot Backend 🤖

Connect fitz to Slack, Discord, Teams, or your own UI. The bot can show source-backed evidence, ask for more documents when Pyrrho marks evidence insufficient, or call fitz answer for generated prose.

Example: A support bot retrieves policy sections, shows links to the relevant docs, and only synthesizes when the evidence verdict is sufficient.

Internal Knowledge Base 📖

Point fitz at your wiki, policies, runbooks, and repos. Employees ask natural-language questions and get source units with provenance.

Example: New hires ask "How do I request PTO?" and receive the exact policy section plus the governance verdict.

Continuous Intelligence & Alerting (Watchdog) 🐶

Run scheduled queries over changing folders, logs, reports, or exports. Trigger alerts when the evidence pack contains sufficient sources, disputes, or missing-coverage signals.

Example: A nightly job asks "Were there failed logins from unusual locations?" and sends the evidence pack to the on-call channel.

Web Knowledge Base 🌎

Scrape web pages to disk, point fitz at the folder, and query the resulting corpus with provenance.

Example: A research workflow scrapes reports, stores them locally, and asks comparative or temporal questions across the collected source set.

Codebase Search 🐍 → Code Symbol Extraction • KRAG

Code retrieval:

Tree-sitter parses your codebase into symbols with qualified names, references, and import graphs. Function and class lookup is address-based, and dependency questions can use graph expansion.

Example: A team asks "Where is user authentication handled?" and receives specific functions, files, and symbol addresses rather than generic file snippets.

┌─────────────────────────────────────────────────────────────────┐
│                         fitz-sage                               │
├─────────────────────────────────────────────────────────────────┤
│  User Interfaces                                                │
│  CLI: retrieve | explain | replay | answer | collections | serve│
│  SDK: fitz_sage.evidence(source=...)                            │
│  API: /answer | /evidence | /chat | /collections | /health      │
├─────────────────────────────────────────────────────────────────┤
│  Engine                                                         │
│  FitzKRAG: typed retrieval over code, documents, and tables     │
├─────────────────────────────────────────────────────────────────┤
│  Evidence Contract                                              │
│  EvidencePack: items | mode | reasons | timings | metadata      │
├─────────────────────────────────────────────────────────────────┤
│  Pyrrho                                                         │
│  evidence verdict | failure mode | evidence metadata            │
├─────────────────────────────────────────────────────────────────┤
│  Local CPU Models                                               │
│  ONNX reranker | managed Qwen query/background | Pyrrho         │
├─────────────────────────────────────────────────────────────────┤
│  Storage                                                        │
│  SQLite + FTS5, one .db per collection                          │
├─────────────────────────────────────────────────────────────────┤
│  Optional OpenAI-Compatible Endpoint                            │
│  answer synthesis | query intelligence | vision                 │
└─────────────────────────────────────────────────────────────────┘

fitz retrieve "question" --source ./docs     # Return governed evidence
fitz retrieve "question"                     # Use current folder or existing collection
fitz retrieve "question" --format json    # Evidence with script-friendly controls
fitz answer "question" --synthesizer ...  # Generated prose from evidence
fitz collections                          # List and delete knowledge collections
fitz serve                                # Start REST API server

Config: .fitz/config.yaml in the current workspace - auto-created on first run. Edit it for optional synthesis, query intelligence, vision, or custom model/provider choices.

Simple usage (module-level, matches CLI):

import fitz_sage

pack = fitz_sage.evidence("What is the refund policy?", source="./docs")
print(pack.mode)

Advanced usage (multiple collections):

from fitz_sage import fitz

# Create separate instances for different collections
physics = fitz(collection="physics")
legal = fitz(collection="legal")

# Retrieve evidence from each collection
physics_pack = physics.evidence("Explain entanglement", source="./physics_papers")
legal_pack = legal.evidence("What are the payment terms?", source="./contracts")

Working with evidence:

pack = fitz_sage.evidence("Where is Pyrrho governance implemented?", source="./fitz_sage")

print(pack.mode)  # runtime AnswerMode: SUFFICIENT, DISPUTED, or INSUFFICIENT
print(pack.reasons)

for item in pack.items:
    print(item.file_path, item.address_location, item.line_range)

Start the server:

pip install fitz-sage[api]

# Initialize the workspace and collection once
fitz retrieve "What is indexed?" --source ./docs

fitz serve                    # localhost:8000
fitz serve -p 3000            # custom port
$env:FITZ_API_KEY = "replace-with-a-random-secret"
fitz serve --host 0.0.0.0     # remote access requires the API key

Interactive docs: Visit http://localhost:8000/docs for Swagger UI.

Endpoints:

Example request:

/answer and /chat require a configured synthesizer. /evidence does not.

curl -X POST http://localhost:8000/answer \
  -H "Content-Type: application/json" \
  -d '{"question": "What is the refund policy?", "collection": "default"}'

fitz command not found after install

Your Python Scripts directory is not on PATH. Use python -m fitz_sage.cli.cli, or add the Scripts directory to PATH.

PDF/DOCX/PPTX files are being skipped

The base install reads embedded text from these formats. Image-only files need an OCR-capable parser. parser: glm_ocr expects a local Ollama glm-ocr model; install fitz-sage[docs] only when you explicitly select a Docling parser.

"Connection refused at localhost:8080" error

This applies to optional endpoint-backed synthesis or query intelligence. fitz retrieve "..." returns evidence without an endpoint server. For generated prose: fitz answer "..." --synthesizer openai/gpt-4o.

"Model not found" error

The model name in your config does not match what your server has loaded. Check /v1/models on your server: curl http://localhost:8080/v1/models. Then update synthesizer in .fitz/config.yaml.

First query is slow

First run initializes storage, downloads managed local models if needed, and builds the query-ready index. Later queries reuse the collection.

How do I change my LLM endpoint or model?

Edit .fitz/config.yaml:

synthesizer: endpoint/gpt-oss-20b
chat_base_url: http://localhost:8080/v1

Or override at the CLI:

fitz answer "..." --endpoint http://localhost:8080/v1 --synthesizer endpoint/gpt-oss-20b

How do I use a cloud provider?

Either use the openai preset:

synthesizer: openai/gpt-4o
# OPENAI_API_KEY in env

Or any OpenAI-compatible cloud via the endpoint provider:

synthesizer: endpoint/meta-llama-3.1-70b
chat_base_url: https://api.together.xyz/v1
chat_api_key_env: TOGETHER_API_KEY

See docs/features/platform/openai-compatible-endpoint.md.

How do I reset everything?

Delete the .fitz/ directory in your project root. Next run will initialize a fresh workspace.