paradox2

Fast, simple document extraction — a from-scratch rewrite of paradox-pdf, built around one idea: most PDFs are digital and don't need a GPU.

import paradox2 as pdx

pages = pdx.extract("invoice.pdf")

Why this rewrite exists

paradox-pdf grew into a 2,783-line facade doing routing, backend resolution, and overlay rendering all in one file. paradox2 starts over with a hard rule: digital pages never pay for OCR or vision models. Every PDF page is classified independently — a digital page (has a text layer) goes through a zero-ML fast path (PyMuPDF text + vector-line table detection); a scanned page routes to the GPU OCR pipeline. A single mixed document is handled correctly per-page, automatically.

Install

pip install paradox2

Everything above works with just that — digital PDFs, text, vector-line tables, key-value fields. Heavier features are opt-in extras so the base install stays small:

First-time setup (scanned/OCR support)

For scanned or photographed pages, run paradox2-setup right after installing the base package — it detects whether the machine has an NVIDIA GPU and installs the matching extras and the correct pinned paddlepaddle build for you:

pip install paradox2
paradox2-setup          # asks to confirm the ~2GB download; add -y/--yes to skip the prompt

This installs paradox2[gpu,rapidai,yolo,formats] plus paddlepaddle pinned to 3.2.1 (GPU build from PaddlePaddle's own index if a GPU is detected, plain CPU build otherwise). Pinning matters: paddlepaddle 3.3.1 — pip's unpinned default — has a real PIR/oneDNN inference bug; 3.2.1 is the verified-good version.

If you'd rather install extras manually instead of running the script:

pip install "paradox2[gpu,rapidai,formats]"

Use a clean virtual environment for [gpu]. Installing into a shared/base environment (e.g. Anaconda's base) that already has an older paddleocr or paddlepaddle from a prior project can leave an incompatible version in place — pip does not always resolve this cleanly against pre-existing packages in a polluted environment. python -m venv .venv && source .venv/bin/activate (or conda create -n paradox2 python=3.12) before installing avoids this.

What it does

import paradox2 as pdx
from dataclasses import asdict

# Every page, as a list of PageResult dataclasses (NOT dicts — a page
# doesn't support page["blocks"]; use page.blocks, or asdict(page) /
# the CLI's --format json for a plain-dict/JSON form) — digital pages
# via the fast path, scanned pages via GPU OCR, decided per page.
pages = pdx.extract("document.pdf")
page_dicts = [asdict(p) for p in pages]  # if you want plain dicts

# Just the text
text = pdx.extract_text("document.pdf")

# Just the tables (vector-line detection on digital pages,
# OCR-grid heuristic + optional RapidAI/VLM fallback on scanned pages)
tables = pdx.extract_tables("document.pdf")

# Key-value pairs from an invoice/form-like page (digital only)
fields = pdx.extract_kie("invoice.pdf")

# Any non-PDF format too — same call, same output shape
data = pdx.extract("spreadsheet.xlsx")
data = pdx.extract("scan.docx")

# Before processing a scanned document: check whether it's actually
# processable with what's currently installed, without running OCR.
report = pdx.can_process("document.pdf")
# {"n_pages": 12, "n_scanned_pages": 12, "needs_ocr": True,
#  "can_process": False, "problems": [...]}

page_workers > 1 (multi-page process-pool parallelism) uses multiprocessing's spawn start method — a caller script MUST guard its top-level code with if __name__ == "__main__":, or the worker processes die on import and every page silently falls back to sequential processing (you'll see a RuntimeWarning when this happens).

Each worker loads its own full OCR model set (~2.4GB RSS measured) — page_workers is automatically capped against both available CPU cores (respecting taskset/cgroup/Docker --cpus limits, not just total system cores) and available RAM, so a request for more workers than the machine can actually hold gets a smaller worker count instead of an OOM kill.

Selecting pages, features, and output format

pdx.extract("report.pdf", pages="1-3,7")           # specific pages, same list[PageResult] shape
pdx.extract("report.pdf", output_format="markdown") # str, not list[PageResult]
pdx.extract("report.pdf", fields=True)               # turn on key-value extraction inline, still list[PageResult]

feature= changes the return type, unlike the calls above - real documentation gap found 2026-09-24 (this used to say "still full extract() under the hood" without mentioning that): the full pipeline does still run internally, but the RETURN VALUE is projected down to list[dict] (or a single value for a scalar feature), not list[PageResult]. Code written against page.blocks breaks with AttributeError: 'dict' object has no attribute 'blocks' if feature= is added later without updating the calling code.

pdx.extract("report.pdf", feature="tables")          # list[dict], not list[PageResult]
pdx.extract("report.pdf", feature="text")             # list[str]

extract() normally returns list[PageResult] (dataclasses - see the first code example above for how to get plain dicts via dataclasses. asdict); only feature= and output_format= change that.

Bundling options with ExtractConfig

Real documentation gap found 2026-09-24: config= was never mentioned anywhere in this README, and it's the only way to override the render DPI for scanned pages:

from paradox2.config import ExtractConfig

cfg = ExtractConfig(dpi=300, scan_text_threshold=80, backend="cpu")
pdx.extract("scan.pdf", config=cfg)

Direct keyword arguments (backend=, language=, etc.) always win over config= for the options both accept - config= is for bundling several overrides together once, not a replacement for the keyword arguments shown throughout this README.

Optional specialists (all off by default, all lazy-loaded)

pdx.extract("form.pdf", handwriting=True)   # TrOCR on handwritten blocks
pdx.extract("contract.pdf", signatures=True) # Conditional-DETR signature boxes
pdx.extract("paper.pdf", formulas=True)      # inline math -> LaTeX

None of these import their heavy dependencies unless the flag is set — import paradox2 alone never touches torch, paddle, or transformers.

The scanned-table router

Scanned pages route tables through up to five tiers, each opt-in past the first (real gap found and fixed in the README 2026-09-24: this used to describe only 3 of the 5, and described RapidAI as the standing mid-tier when PP-StructureV3 has actually taken that slot by default since 2026-09-24):

OCR-grid heuristic (free, always on)
        |  rejects merged/borderless/dense tables by design,
        |  or under-reads a real table (low_confidence flag)
        v
YOLO26 bbox (PARADOX2_YOLO_DETECT=1) -- crop the page to the detected
        |  table region before handing it to the tiers below. A
        |  borderline-confidence detection (<0.85) skips the mid-tier
        |  entirely and goes straight to the VLM tier - VLM tolerates
        |  a loose/imprecise crop better than a structure model does.
        |  A table covering >55% of the page area also skips straight
        |  to VLM (large tables measured harder in this session).
        v
PP-StructureV3 (PARADOX2_PPSTRUCTURE_TABLES=1, the default occupant of
        |  this slot) OR RapidAI (PARADOX2_RAPIDAI_TABLES=1, used only
        |  when PP-StructureV3 is disabled or has no GPU/CPU headroom -
        |  see paradox2.doctor()/resource_detect for that check). Either
        |  one's result is cross-checked against OCR line density in the
        |  same crop - implausibly few OR implausibly many rows re-route
        |  to the VLM tier
        v
VLM fallback (PARADOX2_VLM_TABLES=1)
        |  PaddleOCR-VL single-pass, reserved for tables that look
        |  genuinely complex - not run on everything the mid-tier touches
        v
Azure Document Intelligence (PARADOX2_AZURE_DI_TABLES=1 AND
        |  AZURE_DI_ENDPOINT/AZURE_DI_KEY both set - double opt-in,
        |  never called by default). PAID, external API - only reaches
        |  this tier if every free tier above still flags the result as
        |  implausible. Measured 86.5% mean accuracy across 12 tables,
        |  97.4% on the one 32-row table where the three local engines
        |  scored 0.9-4.7% - but total failure on the 3 simplest tables,
        |  hence last resort rather than first choice.
        v
   best available result

Timeouts for each network/GPU-bound tier are configurable (PARADOX2_{RAPIDAI,VLM,YOLO,PPSTRUCTURE}_TIMEOUT_S, PARADOX2_PPSTRUCTURE_CPU_TIMEOUT_S), as is the CPU-core floor below which PP-StructureV3 is skipped entirely (PARADOX2_PPSTRUCTURE_MIN_CPU_CORES) and the per-machine state cache directory (PARADOX2_STATE_DIR) — see .env.example for every variable with its default and rationale.

Every tier is measured end-to-end (paradox2.extract(), not the isolated engine wrapper) on the same 20 real full-page tables (OmniDocBench), not simulated. This table predates PP-StructureV3 becoming the default mid-tier occupant (2026-09-24) - it measures the RapidAI-as-mid-tier chain only; the 35%/80% numbers are NOT what the current default (PARADOX2_PPSTRUCTURE_TABLES=1) produces, and haven't been re-measured against it yet:

The isolated RapidAI engine alone scores 65% exact on tight ground-truth crops — the gap to the router's end-to-end number is the YOLO26 crop's margin versus a perfect crop, not a RapidAI accuracy problem. Cropping to the heuristic's own bbox instead of an independent detector actively hurts: that bbox only spans the rows the heuristic already found, so it bakes its own under-read into the pixels before the next tier ever sees them. A same-crop self-consistency check (RapidAI's row count vs. OCR line density) can't catch a loose crop itself, since both numbers inflate together on the same imprecise crop — that's what the YOLO-confidence trigger (<0.85) is for instead.

The heuristic never gets replaced by a worse result — a lower tier's output is kept unless a higher tier actually produces something.

export PARADOX2_YOLO_DETECT=1
export PARADOX2_RAPIDAI_TABLES=1
export PARADOX2_VLM_TABLES=1

OCR language (scanned pages)

language= (default "latin") selects the recognition model - real gap found 2026-09-24: 5 aliases mapping to only 2 real PP-OCRv5 models were never documented anywhere.

Any other string is passed straight through to PaddleOCR as a model name (flexible, but undocumented beyond this) - an unknown one raises a clear ValueError: No engine bindings registered for model '...'.

Health check

from paradox2 import doctor
doctor()  # {"cuda": True/False, "pymupdf": "1.24.x", "ok": True/False, "problems": [...], ...}

paradox2 doctor          # full report
paradox2 doctor --fix    # prints only the pip install command(s) needed, exit 1 if any

First OCR call downloads model weights (PP-OCRv5, no progress bar) — a first extract() on a scanned page that appears to hang for 10-30s on a slow connection is this, not a crash. Backend stderr noise (onnxruntime/ TensorFlow/cuDNN registration warnings) is upstream noise from paddleocr's own dependencies, not paradox2 — there is currently no flag to suppress it.

Design principles

• Digital-first: a PDF with a text layer never pays for OCR, vision layout models, or GPU inference — verified per-page, not per-document.
• Facade stays thin: paradox2/api.py is dispatch only; business logic lives in pipeline/, tables/, engines/, formats/. If the facade creeps toward 100+ lines, that's a signal something leaked out of place.
• Specialists are lazy and swallow their own failures: an optional engine (handwriting, signatures, formulas, RapidAI, VLM) that fails to load or errors mid-call returns []/None — it never takes down the base extraction path.
• No simulated benchmarks: every accuracy/speed number in this README and in the codebase's docstrings comes from a real run against real documents, with the script path noted alongside it.

Status

Early-stage rewrite (Alpha) - the API and table router are still stabilizing. Development docs referenced in some docstrings (docs/PLAN.md, docs/AGENT_SYNC.md, docs/TASKS.md) are internal working notes, not shipped with the package - real gap found 2026-09-24: those references are dead ends for anyone who only has the installed package, not the source repo. See the repository for current source and issues.

Production use: three conditions

Verified 2026-09-24 by a from-scratch install + real 12-page notarial document benchmark (rotation, low-res, Spanish accents, real Office/ODF formats). The core path (PDF → text/tables, digital or scanned, backend auto-detected) is solid at this version: correct text, correct rotation handling, security holes closed. Three conditions still apply before running it unattended:

1. Pin the exact version. paradox2==0.2.10, upgrade on your own schedule, not pip's default resolution. This package ships multiple releases per day while output_format's return shape, vector-page routing, and the table-escalation threshold are still moving.
2. Recycle the worker process periodically - this is an architecture requirement, not a temporary workaround. The RAM growth is PaddleOCR's own, upstream, known memory leak (confirmed against PaddleOCR's own issue tracker: #7823 and #11639 - "the process won't give allocated memory back, ever"), not something a future paradox2 release will make go away. Measured baseline here: ~2.4GB/worker plus roughly 100MB/page. Treat it the way Celery's own worker_max_tasks_per_child treats this exact class of problem: cap pages-per-worker (30-50 is a reasonable starting point at these numbers, keeping a worker under ~6GB) and let the orchestrator recycle the process on that count, not on a memory-pressure signal you hope never fires. The persistent worker (paradox2.service.server) now has this built in: set PARADOX2_SERVICE_MAX_PAGES=N and the process exits cleanly (code 0) once it has served N pages, for your process supervisor (systemd Restart=always, Docker --restart=always) to restart it fresh. Unset by default - existing deployments that already recycle at their own orchestration layer keep the exact prior behavior.
3. Leave the specialist engines off (handwriting=, classify=, detect_language=, formulas=) unless you have verified them on your own documents. Measured on the same real document: the handwriting discriminator flagged ~100% of printed blocks as handwritten, language detection called Spanish text English, and the document classifier mistagged a notarial power-of-attorney as "finance". The signature detector was the one specialist that scored correctly. These are real, open accuracy gaps, not configuration mistakes - see the open items tracked in the repository issues.

Not yet ready for a client-facing service with an SLA: there is no automated before/after benchmark gate on this repo, so each new version is unverified until you run one against your own reference documents. Build a golden set (20-30 real documents with expected output) and compare each pinned version's output against it in your own CI before upgrading - this and worker recycling are the two gaps between this package and a real production deployment, not raw accuracy or security.