complydoc 0.6.4

Know your documents before they reach an LLM.

Works with
LangChain · LlamaIndex · Unstructured · Docling · LlamaParse · Azure Document Intelligence

complydoc is the observability layer for AI ingestion pipelines. It traces every step between your documents and your vector store (loading, cleaning, splitting, embedding, storing) with what each one cost, how long it took and what it passed on, and follows each document through them. It warns of the failures that raise no error, such as a scan that loaded no text, puts different loaders' readings of the same document side by side, and flags the personal or financial identifiers, and instructions hidden for a model, that make it through.

It runs on your machine, and nothing leaves it unless you ask.

Quickstart

uv tool install complydoc        # or: pip install complydoc

Observe an ingestion pipeline as it runs, every loader, splitter, embedding call and vector store a step:

import complydoc as cd

with cd.observe("contracts-ingest"):
    documents = PyMuPDF4LLMLoader("contract.pdf").load()
    chunks = splitter.split_documents(documents)
    vector_store.add_documents(chunks)

Compare loaders on your own documents, page by page:

from langchain_community.document_loaders import PyPDFLoader
from langchain_pymupdf4llm import PyMuPDF4LLMLoader

report = cd.compare_loaders(
    {"PyPDFLoader": PyPDFLoader, "PyMuPDF4LLMLoader": PyMuPDF4LLMLoader},
    paths="./contracts",
    page_images=True,
)
cd.write_json(report, ".complydoc/loaders.json", detail="full")

Or audit a folder from the command line:

complydoc audit ./contracts

Then open it all in the viewer, served from your machine:

complydoc ui

The playground has all of this ready to run.

What it shows

• Pipeline traces: each LangChain or LlamaIndex loader, splitter and embedding call as a step, with its settings, time, tokens, cost and what it passed on.
• Loader diffs: every loader's reading of a document side by side, line by line, with the page it came from.
• Identifiers: personal and financial identifiers from Europe, the Americas, India and Australia, checksum-validated where a checksum exists, and masked until you choose to see them.
• Hidden content: text a reader does not see and a model does, and passages written as instructions to a model.
• Chunks: splitters side by side, drawn over the text, with the sentences and tables they cut.
• Cost: text and vision tokens per page, priced across models and extraction paths.

In CI

complydoc check holds a folder to rules written in YAML and fails when they break. On GitHub, the repository is also an action:

- uses: complydoc/complydoc@v0.6.4
  with:
    path: documents
    policy: policy.yaml

See GitHub Action and Policy files.

Privacy

Outbound connections and DNS lookups are blocked for the whole run, and every report records that they were. Identifiers are masked in every output, the page text included. The only things that send data anywhere are ones you add yourself, such as a hosted parser or the optional typesafe and assistant extras, and they run only with allow_network=True. See Network isolation.

Optional extras

Reading scans and finding names are optional, because they are large. Every report says when they were missing.

uv tool install "complydoc[ocr,multilingual-names]"
complydoc doctor              # what is installed, and the command for anything missing

The name models are downloaded once, before a scan, never during one: complydoc doctor prints the command. Detection accuracy publishes how often each finds and misses names.

Resources

• Documentation
• Playground: runnable examples, with a CI workflow
• Command line and Python API references
• Changelog
• Contributing

License

MIT