Paper Siphon

Extract clean Markdown from academic PDFs - like drinking through a straw.

Academic papers come with artifacts: awkward page breaks, mangled tables, or even line numbers. Paper Siphon filters them out, leaving you with clean, readable Markdown.

paper-siphon paper.pdf

That's it. Your paper is now paper.md.

Features

• Smart whitespace - Collapses excessive blank lines, normalizes spacing
• Table preservation - Keeps your data tables intact and formatted
• Formula support - Optional enrichment for mathematical expressions
• Line number removal - Automatically strips the margin numbers (when present)
• VLM backend - --vlm uses GLM-OCR on Apple Silicon (marker elsewhere) for complex layouts, heavy math, and broken encodings — see the benchmark
• Self-healing - Auto-escalates to the VLM backend when the fast default output looks garbled or drops equations

Installation

# With uv (recommended)
uv pip install paper-siphon

# With pip
pip install paper-siphon

The --vlm backend (GLM-OCR on Apple Silicon, marker elsewhere) needs no extra install — it runs in an isolated environment provisioned on demand, and only requires uv on your PATH (already true if you use uvx). The first --vlm run downloads the model (~2 GB), then caches it.

Usage

Quick start (no install)

uvx paper-siphon paper.pdf                # Run directly with uvx

Basic

paper-siphon paper.pdf                    # Creates paper.md
paper-siphon paper.pdf -o notes.md        # Custom output path

From URL (including arXiv)

paper-siphon https://arxiv.org/pdf/1706.03762.pdf

Tip: For arXiv papers, just change /abs/ to /pdf/ in the URL:

https://arxiv.org/abs/1706.03762  →  https://arxiv.org/pdf/1706.03762.pdf

(That's "Attention Is All You Need" - the Transformer paper)

Advanced

paper-siphon --vlm paper.pdf              # Force the VLM backend (GLM-OCR / marker)
paper-siphon --no-escalate paper.pdf      # Don't auto-retry with the VLM backend
paper-siphon --enrich-formula paper.pdf   # Formula enrichment on the default pipeline
paper-siphon --no-mlx --vlm paper.pdf     # Force the marker backend even on a Mac
paper-siphon -v paper.pdf                 # Verbose logging

By default, the fast pipeline runs first and paper-siphon automatically re-runs with the VLM backend if that output looks garbled (a font-decoding failure) or dropped its equations. Use --vlm to force the VLM backend from the start, or --no-escalate to keep the fast output as-is.

How It Works

Paper Siphon uses Docling for PDF parsing, then applies post-processing to clean up common academic paper artifacts:

1. PDF parsing - Extracts structure, text, and tables (fast Docling pipeline)
2. Quality check - Detects font-decoding failures (garbled glyphs) and dropped equations; escalates to the VLM backend when needed
3. Line number filtering - Removes standalone 1-4 digit numbers (common in journal formats)
4. Whitespace normalization - Collapses multiple blank lines

The VLM backend (--vlm or auto-escalation) runs GLM-OCR on Apple Silicon and marker elsewhere, in an isolated environment so its dependencies never interfere with the fast default pipeline. See bench/README.md for how these backends were chosen.

Options

Backend benchmark

Which PDF→Markdown backend should Paper Siphon use? bench/ holds a reproducible benchmark that compares candidate backends on 14 public academic PDFs using a blinded dual-LLM-judge evaluation (plus reference-free metrics and Apple-Silicon speed).

Headline result (overall /10, mean of two blinded judges, 8-backend re-run): LightOnOCR-2 8.18 > GLM-OCR/MLX 7.43 > Marker 7.21 > Docling standard 4.87 (current default). LightOnOCR-2 wins ~95% of blind head-to-heads — the best quality on prose and math — but at ~77 s/page it is roughly 11× slower than GLM-OCR/MLX, which is why GLM-OCR stays the practical pick for --vlm (~7 s/page on an M4 Max, MIT weights, 0.9B model that fits in 8 GB). Leaderboard-topping candidates (PaddleOCR-VL, olmOCR-2, Nanonets, MinerU, Chandra) collapse or prove non-viable on Apple Silicon via unofficial MLX/MPS paths. See bench/README.md for the full tables, methodology, and caveats.

Development

# Clone and install
git clone https://github.com/mrshu/paper-siphon.git
cd paper-siphon
uv sync --dev

# Run tests
uv run pytest

# Run tests with coverage
uv run pytest --cov=paper_siphon

License

MIT

Stop wrestling with PDFs. Just siphon the good stuff.