cyberdocextractor 0.2.0

Production-grade document-to-Markdown conversion library with hexagonal architecture.

CyberDocExtractor

Production-grade Python library for converting PDF, DOCX, DOC, XLSX, XLS, and CSV documents to Markdown. Multiple precision levels, automatic extractor fallback, LLM-assisted image descriptions, and a strict hexagonal core.

pip install cyberdocextractor==0.1.1

Why another document extractor?

Every upstream library (Docling, PyMuPDF, pdfplumber, python-docx, openpyxl, xlrd, …) has its own model, error semantics, and quirks. CyberDocExtractor wraps them behind a single typed domain model:

• Hexagonal architecture — the domain doesn't know what PyMuPDF is. Boundaries are enforced by import-linter in CI (three contracts, kept on every PR).
• Pluggable extractors — swap, reorder, or add adapters without touching the core. Heavy dependencies are lazy-imported, so installing with no extras still works.
• Automatic fallback — if the preferred extractor fails, the next candidate in the policy chain is tried. A corrupt page doesn't abort a batch.
• Deterministic quality scoring — every conversion reports a 0.0 – 1.0 score, so downstream RAG / ETL can gate on confidence.
• Immutable value objects everywhere — frozen, slotted dataclasses with invariants enforced in __post_init__; metadata maps wrapped in MappingProxyType so templates can't corrupt pipeline state.
• Strict CI — mypy strict, ruff with pydocstyle, import-linter architectural contracts, 90% coverage floor (currently 97%), matrix tests on Linux + macOS × Python 3.11 / 3.12 / 3.13.

Installation

Command	What you get
pip install cyberdocextractor	Core + CSV (stdlib)
pip install 'cyberdocextractor[pdf]'	+ PyMuPDF / pymupdf4llm / pdfplumber (PDF, 3 precision tiers)
pip install 'cyberdocextractor[doc]'	+ Docling (PDF / DOCX / XLSX at HIGHEST_QUALITY; also powers DOCX)
pip install 'cyberdocextractor[docx]'	+ python-docx + defusedxml (needed by the .doc adapter)
pip install 'cyberdocextractor[xlsx]'	+ openpyxl (XLSX) + xlrd (legacy XLS) + defusedxml
pip install 'cyberdocextractor[llm]'	+ httpx + Pillow (LLM image descriptions)
pip install 'cyberdocextractor[cli]'	+ Typer + Rich (cyberdoc command)
pip install 'cyberdocextractor[all]'	Everything above

The .doc adapter additionally requires LibreOffice on your PATH (soffice or libreoffice) — the adapter shells out for the CFBF → DOCX conversion, then parses the result with python-docx.

30-second Python

from pathlib import Path
from cyberdocextractor import PrecisionLevel, build_pipeline, load_document

pipeline = build_pipeline()

doc = load_document(Path("report.pdf"), precision=PrecisionLevel.BALANCED)
report = pipeline.convert(doc)

print(report.markdown.text)
print(f"extractor={report.extractor}")
print(f"attempted={list(report.attempted)}")
print(f"quality={report.markdown.quality_score:.2f}")

30-second CLI

# Single document to stdout (or -o FILE)
cyberdoc convert report.pdf --show-score

# Batch with glob + precision override
cyberdoc batch inputs/ outputs/ --pattern '*.pdf' --level 3

# Which adapters are available on this machine?
cyberdoc status

# Size / MIME / auto-selected precision (no conversion)
cyberdoc info report.pdf

# List bundled Jinja2 templates
cyberdoc templates

cyberdoc version

Command	Purpose
convert	Convert one document. Flags: --level {1,2,3,4}, --template NAME, --mime MIME, --output FILE, --show-score, --no-fallback.
batch	Walk a directory with a glob pattern and write mirrored .md files. Failures are captured, not raised — exit 1 if any file failed.
status	Rich table of extractor availability per MIME type (PDF / DOCX / DOC / XLSX / XLS / CSV).
info	Print size, MIME, classification, and the precision that would be auto-selected.
templates	List available Jinja2 templates.
version	Print the installed package version.

Full CLI reference.

Architecture

Four layers, one set of import rules, enforced by import-linter in CI:

flowchart TB
    CLI["cli — Typer entrypoints"]
    APP["app — ConversionPipeline + stages"]
    INFRA["infra — PyMuPDF, pdfplumber, Docling, openpyxl, xlrd, Jinja2, httpx, LibreOffice"]
    DOMAIN["domain — pure value objects, ports, rules"]

    CLI --> APP
    CLI --> INFRA
    CLI --> DOMAIN
    APP --> DOMAIN
    INFRA --> APP
    INFRA --> DOMAIN

    classDef pure fill:#eef,stroke:#448,stroke-width:2px;
    classDef app fill:#efe,stroke:#484,stroke-width:2px;
    classDef infra fill:#fee,stroke:#844,stroke-width:2px;
    classDef cli fill:#fef,stroke:#848,stroke-width:2px;

    class DOMAIN pure;
    class APP app;
    class INFRA infra;
    class CLI cli;

• Domain — pure value objects (Document, Block, NormalizedDoc, Markdown, ExtractionOutcome, TemplateContext, ConversionReport), enums, Protocol ports, typed errors, pure rules. Zero external dependencies.
• Application — ConversionPipeline + five composable stages, loader + batch helpers, default Clock / Scorer.
• Infrastructure — one lazy-imported adapter per upstream library.
• CLI — Typer commands that parse arguments and delegate.

Three import-linter contracts keep the stack honest on every PR:

1. Hexagonal layers (layered stack).
2. Domain is pure — domain cannot import from app, infra, or cli.
3. App depends only on domain — app cannot import infra or cli.

Conversion pipeline

flowchart LR
    Input[Document] --> Ex[ExtractionStage]
    Ex --> Meta[MetadataStage]
    Meta --> Img[ImageEnrichmentStage]
    Img --> Tab[TableProfilingStage]
    Tab --> Ren[RenderingStage]
    Ren --> Report[ConversionReport]

• ExtractionStage walks the policy-selected chain with automatic fallback; adapters wrap failures in ExtractionOutcome.fail rather than raising.
• MetadataStage, ImageEnrichmentStage, TableProfilingStage are defensive — individual probe / describer / profiler failures are logged and skipped, never allowed to fail the conversion.
• RenderingStage wraps Jinja2 errors in RenderingError and scores the final Markdown.

Full deep-dive: docs/architecture/pipeline.md.

Precision levels

Level	Name	Typical extractor	Trade-off
1	FASTEST	pymupdf-chunked	speed > fidelity; no layout analysis
2	BALANCED (default)	pymupdf4llm	good text + tables at moderate speed
3	TABLE_OPTIMIZED	pdfplumber	best for structured tables
4	HIGHEST_QUALITY	Docling	OCR, images, full layout; slowest

When you pass BALANCED (the default), the policy auto-selects based on document size:

• < 2 MiB → HIGHEST_QUALITY (small files are cheap to process at full fidelity).
• > 20 MiB → FASTEST (archival scans need throughput).
• otherwise → stays at BALANCED.

document.metadata["has_tables"] = True promotes to TABLE_OPTIMIZED at any size. Passing any precision other than BALANCED bypasses the auto-resolution — your choice is respected as-is.

Rationale in ADR 0005; disable via build_pipeline(size_aware_policy=False).

Format support

Format	Extension	Extractor(s)	Required extras
CSV	.csv	csv-stdlib	none
PDF	.pdf	pymupdf-chunked, pymupdf4llm, pdfplumber, docling	[pdf] (one of the first three) or [doc] (Docling)
DOCX	.docx	docling	[doc]
DOC (legacy)	.doc	libreoffice-doc	[docx] plus LibreOffice on PATH
XLSX	.xlsx	openpyxl, docling	[xlsx] or [doc]
XLS (legacy)	.xls	xlrd	[xlsx]

cyberdoc status shows the matrix for the adapters actually available in your environment.

Releases

Version	Highlights
0.1.1	Added XlrdExtractor (legacy .xls) and LibreOfficeDocExtractor (legacy .doc via soffice). CLI status table surfaces DOC and XLS columns.
0.1.0	First public release. Full hexagonal core, six extractors, Jinja2 renderer, heuristic scorer, PDF + Office metadata probes, OpenAI-compatible LLM image describer, Typer CLI.

Changelog: docs/changelog.md. Roadmap (including known limitations and post-1.0 plans): docs/roadmap.md.

Documentation

• Quickstart — every install flow, first conversion, batch, CLI.
• Architecture overview — layer rules, error routing, sequence diagram.
• Pipeline deep-dive — per-stage semantics, testing recipes, extensibility points.
• Domain model — UML + invariant tables per value object.
• Ports reference — every Protocol + its adapters.
• Writing an extractor — step-by-step.
• Writing a template — TemplateContext schema + sandbox notes.
• LLM image descriptions — OpenAI, Azure, Ollama, vLLM recipes.
• Batch conversion — per-file failure handling + parallelism guidance.
• ADRs — seven decisions, each with Context / Decision / Consequences.

Development

Prerequisites: Python 3.11+, uv, just.

git clone https://github.com/CyberdyneCorp/CyberDocExtractor.git
cd CyberDocExtractor
just bootstrap    # uv sync --all-extras --group dev --group docs + pre-commit

just check        # format + lint + mypy + import-linter + tests
just test-unit    # unit tests only
just test-bdd     # pytest-bdd scenarios only
just docs-serve   # mkdocs-material live preview

See CONTRIBUTING.md for the full contribution workflow.

Security

See SECURITY.md for the responsible-disclosure policy. Highlights:

• XML parsing uses defusedxml by default (Office metadata probes).
• Jinja2 environments are sandboxed; autoescape=False because the output format is Markdown.
• Dependencies scanned weekly (Bandit, pip-audit, TruffleHog).

License

MIT — see LICENSE.