Skip to main content

Paper Siphon

Extract clean Markdown from academic PDFs - like drinking through a straw.

Academic papers come with artifacts: awkward page breaks, mangled tables, or even line numbers. Paper Siphon filters them out, leaving you with clean, readable Markdown.

paper-siphon paper.pdf

That's it. Your paper is now paper.md.


Features

  • Smart whitespace - Collapses excessive blank lines, normalizes spacing
  • Table preservation - Keeps your data tables intact and formatted
  • Formula support - Optional enrichment for mathematical expressions
  • Line number removal - Automatically strips the margin numbers (when present)
  • VLM backend - --vlm uses Nemotron float32 on Apple Silicon (marker elsewhere) for complex layouts, heavy math, and broken encodings. See the benchmark
  • Self-healing - Auto-escalates to the VLM backend when the fast default output looks garbled or drops equations

Installation

# With uv (recommended)
uv pip install paper-siphon

# With pip
pip install paper-siphon

The VLM backends run in isolated environments provisioned on demand. They require uv on your PATH. The Apple Silicon backends also need Git to fetch the pinned MLX-VLM revision. The first run installs the runtime and downloads model weights, then caches them. Nemotron float32 weights are about 2.5 GB, Nemotron BF16 about 1.3 GB, and Xiaomi about 1.7 GB. Runtime packages require additional disk space.

Usage

Quick start (no install)

uvx paper-siphon paper.pdf                # Run directly with uvx

Basic

paper-siphon paper.pdf                    # Creates paper.md
paper-siphon paper.pdf -o notes.md        # Custom output path

From URL (including arXiv)

paper-siphon https://arxiv.org/pdf/1706.03762.pdf

Tip: For arXiv papers, just change /abs/ to /pdf/ in the URL:

https://arxiv.org/abs/1706.03762  →  https://arxiv.org/pdf/1706.03762.pdf

(That's "Attention Is All You Need" - the Transformer paper)

Advanced

paper-siphon --vlm paper.pdf              # Force Nemotron float32 / marker
paper-siphon --vlm --vlm-backend xiaomi paper.pdf  # Local formula-focused model
paper-siphon --vlm --vlm-backend nemotron-bf16 paper.pdf  # Faster local VLM
paper-siphon --no-escalate paper.pdf      # Don't auto-retry with the VLM backend
paper-siphon --enrich-formula paper.pdf   # Formula enrichment on the default pipeline
paper-siphon --no-mlx --vlm paper.pdf     # Force the marker backend even on a Mac
paper-siphon -v paper.pdf                 # Verbose logging

By default, the fast pipeline runs first and paper-siphon automatically re-runs with the VLM backend if that output looks garbled (a font-decoding failure) or dropped its equations. Use --vlm to force the VLM backend from the start, or --no-escalate to keep the fast output as-is.

How It Works

Paper Siphon uses Docling for PDF parsing, then applies post-processing to clean up common academic paper artifacts:

  1. PDF parsing - Extracts structure, text, and tables (fast Docling pipeline)
  2. Quality check - Detects font-decoding failures (garbled glyphs) and dropped equations and escalates to the VLM backend when needed
  3. Line number filtering - Removes standalone 1-4 digit numbers (common in journal formats)
  4. Whitespace normalization - Collapses multiple blank lines

The VLM backend uses Nemotron 3.5 float32 on Apple Silicon and marker elsewhere. --vlm-backend selects the model for forced VLM extraction and automatic retries. Xiaomi MLX BF16 is the formula-focused choice, and Nemotron BF16 is faster with some additional formula errors. GLM remains available as --vlm-backend glm-ocr. Explicit MLX choices require Apple Silicon. --no-mlx uses marker.

Automatic escalation reruns the complete document. Xiaomi can duplicate bibliography entries, and all tested local models can assign values to the wrong table columns. Check important formulas and table values against the source. See the measured comparison for the results. The isolated runtimes keep their dependencies separate from Docling.

Options

Flag Description
-o, --output Output file path (default: input with .md extension)
--vlm Force the selected VLM (Nemotron float32 on Apple Silicon, marker elsewhere)
--vlm-backend auto, nemotron, nemotron-bf16, xiaomi, glm-ocr, or marker (default: auto)
--mlx/--no-mlx Enable MLX on Apple Silicon. --no-mlx forces marker
--escalate/--no-escalate Auto-retry with the VLM backend on garbled/math-dropping output (default: on)
--enrich-formula Enable formula enrichment on the default pipeline (slow, CPU-bound)
-v, --verbose Enable debug logging

Backend benchmark

The combined model comparison covers all measured configurations, including both Nemotron precisions and Xiaomi on MLX and MPS.

The page-aligned comparison evaluates Docling, the current GLM integration, and GPT-6.1 Sol on 16 pages from eight papers. It compares complete transcriptions with matching rendered pages and includes a separate three-page LightOnOCR-3 test. Sol scored highest in this sample, and Docling was fastest. See the reproduction guide for commands and saved evidence. Follow-up experiments cover Nemotron 3.5 and Xiaomi-OCR-0 on the same 16 pages. Version 0.5.0 keeps Docling first and uses Nemotron float32 for Apple Silicon VLM retries.

The older 14-PDF comparison remains available as historical results. An audit found truncated candidate inputs and weak verdict validation in that benchmark. Its scores should not determine model selection without a new comparison.

Development

# Clone and install
git clone https://github.com/mrshu/paper-siphon.git
cd paper-siphon
uv sync --dev

# Run tests
uv run pytest

# Run tests with coverage
uv run pytest --cov=paper_siphon

License

MIT


Stop wrestling with PDFs. Just siphon the good stuff.

Metadata

Release files for paper-siphon 0.5.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for paper-siphon 0.5.0
File Size Uploaded Provenance Actions
paper_siphon-0.5.0.tar.gz 366.0 kB Without provenance Details for paper_siphon-0.5.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for paper-siphon 0.5.0
File Interpreter ABI Platform Actions
paper_siphon-0.5.0-py3-none-any.whl Python 3 none any Details for paper_siphon-0.5.0-py3-none-any.whl

Total release size: 394.6 kB

Release files / paper_siphon-0.5.0.tar.gz

Download URL paper_siphon-0.5.0.tar.gz
Size 366.0 kB
Tags Source
SHA-256 checksum
How to use checksums
6bc5a9cb7f775a76d6ce1f00e49e1987713cf9fdb2e8ff26f08d99c1624855be
BLAKE2b-256 checksum
How to use checksums
981aac761318e3f913568b98041acedd3bbcf82d3f60bc16e052344ed789e555
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / paper_siphon-0.5.0-py3-none-any.whl

Download URL paper_siphon-0.5.0-py3-none-any.whl
Size 28.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b3fb68acc79a601d49a09f38da51e3b407a47938f1e64c9f77a3a0681a25b1d2
BLAKE2b-256 checksum
How to use checksums
23b8c95444a4f57d62d5b033e7cdf5aba53390beb1740cf068c4d84d64100a84
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.13.5 {"installer":{"name":"uv","version":"0.13.5","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for paper-siphon 0.5.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page