razorpdf — Hybrid Geometric PDF Parser for RAG & MCP

Ultra-fast, local-first PDF parser: PDFs in, clean LLM-ready Markdown plus grounded RAG chunks out.

Deterministic Rust geometry handles the easy 90%+ of document layouts in microseconds per page (16,720 pages/sec on synthetic benchmarks; 339 pages/sec across real arXiv papers and tax invoices). Only genuinely messy sub-page regions ($S(R) \ge 0.45$) ever get cropped to WebP and sent to a Vision Language Model (VLM).

Try it in 30 seconds

pip install razorpdf          # Python 3.10+, prebuilt wheels, no Rust needed
cargo install razorpdf        # CLI + MCP server from source

import razorpdf
doc = razorpdf.parse_file("paper.pdf")   # -> {"markdown": ..., "chunks": [...]}

Prefer a binary? Grab one from Releases (Linux shown):

curl -L -o razorpdf.tar.gz https://github.com/shridhargsabat/razorpdf/releases/download/v0.1.4/razorpdf-v0.1.4-x86_64-unknown-linux-gnu.tar.gz
tar xzf razorpdf.tar.gz && ./razorpdf parse paper.pdf -o paper.md --json paper.json

Or run the container (builds once, then parse + MCP server included):

git clone https://github.com/shridhargsabat/razorpdf.git && cd razorpdf
docker compose run --rm parse   # samples/invoice.pdf -> ./out/invoice.{md,json}

Prefer a prebuilt image? Each release publishes to GHCR:

docker run --rm -v ./samples:/work:ro ghcr.io/shridhargsabat/razorpdf:latest \
  parse /work/invoice.pdf

No Docker either? Try it in your browser:

Why razorpdf?

Most PDF pipelines force an expensive tradeoff: flat text extractors mangle multi-column papers and borderless tables, while full-page vision/OCR pipelines burn seconds and thousands of image tokens on simple text pages. razorpdf combines both worlds:

Why sub-page VLM wins on cost

Full-page vision pipelines pay image tokens for every page. razorpdf pays zero for geometric pages and only a small crop for the rest — measured on the wild suite:

At typical frontier-VLM image pricing (~$2.50/1M input tokens, illustrative), that is ~$0.0053 per full-res page vs ~$0.0011 per crop — and geometry itself runs in ~60 µs/page with no model at all.

What You Get Out

Pass any PDF via CLI, Python, or MCP and receive both structured Markdown and grounded RAG chunks with exact bounding boxes, heading hierarchies, token counts, and provenance flags:

{
  "total_pages": 15,
  "markdown": "# Attention Is All You Need\n\n...",
  "chunks": [
    {
      "chunk_id": "p5-005",
      "page_index": 5,
      "bbox": {"x0": 124.5, "y0": 137.2, "x1": 468.0, "y1": 181.4},
      "breadcrumb": ["Attention Is All You Need", "3 Model Architecture"],
      "markdown": "| Self-Attention | O(n2 ·d) | O(1) | O(1) |\n| --- | ...",
      "token_estimate": 57,
      "parsed_by_vlm": false
    }
  ]
}

Architecture

PDF bytes
    │
    ▼
┌─────────┐   lopdf (pure Rust, rayon page-parallel)
│ EXTRACT │─── TextSpan + VectorSegment + image bboxes (top-left coords)
└─────────┘
    │
    ▼
┌──────────┐  line merge -> XY-cut reading order -> vector table grids
│ GEOMETRY │── multi-column safe, zero model calls
└──────────┘
    │
    ▼
┌────────┐  S(R) = min(1, 0.40·poverlap + 0.35·ptables + 0.25·pvectors)
│ TRIAGE │── fast path (< 0.45) or padded sub-page VLM region
└────────┘
    │
    │ WebP crop of BBox only (never full pages)
    ▼
┌─────┐  OpenAI / Gemini / Ollama / any OpenAI-compatible endpoint
│ VLM │── bounded concurrency, per-region failure tolerance
└─────┘
    │  │
    ▼  ▼
┌──────────┐ reading-order Markdown + chunks with
│ ASSEMBLE │── `{x0,y0,x1,y1}` bboxes + breadcrumb trails
└──────────┘

• Fast path is pure geometry: Line merging, recursive XY-cut, and R-tree vector-table reconstruction run in parallel across cores with zero model calls.
• VLM fallback is sub-page: Only the failing BBox is cropped to WebP and sent — a messy borderless table costs one small image crop, not a full-page render.
• Chunks are grounded: Every chunk carries its chunk_id, page_index, bbox ({x0, y0, x1, y1} in PDF points), heading breadcrumb trail, token_estimate, and the parsed_by_vlm provenance flag.
• Offline by default: No network calls run in tests or on the fast path. Without --vlm-url or --mock-vlm, messy regions keep their raw geometric text so content is never dropped silently.

Quickstart

1. CLI (Rust)

cargo install razorpdf

# Parse a PDF to Markdown (stdout or file) plus grounded JSON chunks
razorpdf parse paper.pdf -o paper.md --json paper.json

# Inspect per-page spans, vectors, lines, tables, S(R) triage scores, and fallback regions
razorpdf inspect paper.pdf

# Route messy sub-page regions to local Ollama (or any OpenAI-compatible VLM endpoint)
razorpdf parse scan.pdf --vlm-url http://localhost:11434/v1/chat/completions

# Run an offline demo of the VLM fallback path using the deterministic mock provider
razorpdf parse messy.pdf --mock-vlm

From source: git clone https://github.com/shridhargsabat/razorpdf.git && cd razorpdf && cargo build --release -p razorpdf.

2. Python Bindings

razorpdf provides native Rust bindings via PyO3/maturin and releases the Python GIL during parsing so thread pools scale cleanly across CPU cores.

pip install razorpdf   # prebuilt abi3 wheels for Linux/macOS/Windows, Python 3.10+

import razorpdf

# Full grounded document (dict with 'markdown' and 'chunks')
doc = razorpdf.parse_file("paper.pdf")
print(doc["markdown"][:500])
print(doc["chunks"][0]["bbox"], doc["chunks"][0]["breadcrumb"])

# Fast Markdown-only extraction (str)
md = razorpdf.to_markdown("paper.pdf")

# Route high-entropy regions to a local or cloud OpenAI-compatible VLM
doc = razorpdf.parse_file(
    "scan.pdf",
    vlm_url="http://localhost:11434/v1/chat/completions"
)

From source: cd crates/razorpdf-py && pip install maturin && maturin develop --release.

3. MCP Server (Claude Code / OpenCode / Hermes)

razorpdf includes a built-in Model Context Protocol (MCP) JSON-RPC 2.0 server over stdio exposing three tools: parse_pdf, inspect_pdf_layout, and extract_pdf_tables.

Claude Code:

claude mcp add razorpdf /path/to/razorpdf/target/release/razorpdf mcp

OpenCode (opencode.json):

{
  "mcp": {
    "razorpdf": {
      "type": "local",
      "command": ["/path/to/razorpdf/target/release/razorpdf", "mcp"]
    }
  }
}

Generic MCP / Hermes (stdio configuration):

{
  "name": "razorpdf",
  "transport": "stdio",
  "command": "/path/to/razorpdf/target/release/razorpdf",
  "args": ["mcp"]
}

Example round-trip over stdio:

echo '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"parse_pdf","arguments":{"path":"paper.pdf"}}}' | ./target/release/razorpdf mcp

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "content": [
      {
        "type": "text",
        "text": "{\"total_pages\":1,\"markdown\":\"# Sample Document\\n...\",\"chunks\":[{\"chunk_id\":\"p0-000\",\"page_index\":0,\"bbox\":{\"x0\":72.0,\"y0\":72.0,\"x1\":540.0,\"y1\":120.0},\"breadcrumb\":[\"Sample Document\"],\"parsed_by_vlm\":false}]}"
      }
    ]
  }
}

4. More ways to try

• Gradio demo / HF Space: demo/ is a Space-ready app (upload a PDF, get Markdown + chunks + bbox view) — see demo/README.md.
• Notebook: docs/notebook/razorpdf_quickstart.ipynb (Colab badge above).
• RAG frameworks: docs/integrations.md — LangChain loader (tested, in demo/) + LlamaIndex snippet.
• VLM providers: docs/vlm-providers.md — Ollama, OpenAI, Gemini, and any OpenAI-compatible endpoint.

CLI Reference

Examples

Bordered invoice table recovered exactly (green box), text around it untouched:

Rulings-free table under a rotated watermark — the watermark stays isolated in its own chunk and never merges into rows:

Machine-readable outputs live in examples/ (Markdown + grounded JSON for both invoices and Attention Table 1).

Benchmarks

Synthetic Fast-Path Throughput

Measured with razorpdf bench (400-page mixed two-column, single-column, and bordered-table paper, fast path, no VLM; median of 3 timed runs after 3 warmups) on an AMD EPYC 9354P (4 vCPU):

Reproduce: cargo run --release -p razorpdf -- bench Criterion suite: cargo bench -p razorpdf-core --bench throughput

Wild Head-to-Head (Real PDFs, Python API)

scripts/bench_wild.py benchmarks razorpdf.parse_file (release bindings) against flat extractors (pypdf, pdfminer.six, pymupdf) and fellow Rust parsers (rasgapdf, oxidize-pdf) on tests/wild/*.pdf (median of 3 runs after 1 warmup on the same EPYC 9354P machine; razorpdf 0.1.1, pypdf 6.21.4, pdfminer.six 20260107, pymupdf 2.0, rasgapdf 0.3.3, oxidize-pdf 0.23.1):

* rasgapdf 0.3.3 is genuinely the fastest flat extractor on the files it reads — but it returned empty output for both generated invoices (verified with path and bytes input), so its TOTAL is flattered by two instant empties. Compare per-file, and see fidelity/structure below.

Content Fidelity & Structure Check

Five backends recover the marker phrases, but only razorpdf reconstructs the tables as structured Markdown grids (the 3-row GST invoice grid and Attention's Tables 1–3 plus author block) rather than flat text.

Model Pipelines (Docling)

Heavy vision/layout pipelines trade latency for structure. On the two single-page files (docling 2.145.0, models cached, same machine):

Docling matches razorpdf on the GST invoice (5/5 markers, 3 table rows) — at ~3,400× the latency. That's the gap the hybrid design closes: geometry gives you the structure in microseconds, and the VLM bill only arrives for regions geometry can't solve.

Reproduce: .venv/bin/python scripts/bench_wild.py (after maturin develop --release). The corpus is gitignored — fetch it first per tests/wild/README.md; the script degrades gracefully on a partial corpus.

Triage Math

For each candidate region $R$, crates/razorpdf-core/src/triage/scorer.rs computes a visual complexity score $S(R)$:

$$S(R) = \min(1.0, 0.40 \cdot p_{\text{overlap}} + 0.35 \cdot p_{\text{tab_entropy}} + 0.25 \cdot p_{\text{vector_noise}})$$

• $p_{\text{overlap}}$: Share of spans touching a bounding-box overlap above 0.25 (measured by span involvement so the ratio does not dilute on dense pages).
• $p_{\text{tab_entropy}}$: Set to 1.0 when $\ge 4$ consecutive $\ge 3$-island rows fail column bucketing (jagged column counts or $x_0$ standard deviation above tolerance).
• $p_{\text{vector_noise}}$: Density of diagonal vector segments intersecting text spans.
• Routing threshold: Regions with $S(R) \ge 0.45$, scanned pages without text spans, and diagram clusters ($\ge 4$ vector segments) are padded, cropped to WebP, and sent to the configured VLM endpoint.

Project Structure

razorpdf/
├── crates/
│   ├── razorpdf-core/            # Engine: extract -> geometry -> triage -> VLM -> emit
│   │   ├── benches/throughput.rs # Criterion pages/sec benchmark suite
│   │   └── examples/dump_fixtures.rs
│   ├── razorpdf/             # Binary: parse / inspect / bench / mcp
│   └── razorpdf-py/              # PyO3 bindings: parse_file / to_markdown (+ GIL release)
├── scripts/
│   └── bench_wild.py             # Real-world PDF benchmark vs pypdf, pdfminer.six, pymupdf
└── tests/wild/                   # Wild-PDF corpus (gitignored, refetchable — see README.md)

Roadmap

• Completed (Phases 1–6): Core geometric extraction, recursive XY-cut layout engine, R-tree vector table reconstruction, $S(R)$ triage scorer, sub-page WebP VLM client, MCP stdio server, and PyO3 bindings.
• Planned:
  • pdfium render backend (--features pdfium)
  • Exact token counting via vendored tiktoken ranks
  • Automatic region splitting for oversized blocks

Contributing

Contributions, bug reports, and benchmark additions are welcome! Feel free to open an issue or submit a pull request on GitHub.

License

Licensed under either of:

• Apache License, Version 2.0 (LICENSE-APACHE or http://www.apache.org/licenses/LICENSE-2.0)
• MIT license (LICENSE or http://opensource.org/licenses/MIT)

at your option.