rustypaper

Structure-aware conversion of born-digital scientific PDFs to Markdown, Typst and JSON. Fast, CPU-only, no models.

The good open-source converters (Marker, MinerU, Docling, Nougat) are Python stacks that want a GPU; GROBID is CPU-only and fast but is a JVM service that emits TEI and ignores maths. The Rust crates that exist are generic text extractors with no notion of a paper. This aims at the gap: structure-aware, maths-aware, CPU-only, single binary.

Status: feature complete. Converts one- and two-column papers to Markdown, Typst, plain text or JSON, with reading order, figures, tables, mathematics and references.

Install

cargo install rustypaper          # the command-line tool

rustypaper = "0.1"            # the library

pip install rustypaper        # the Python bindings

Nothing else to install. The default build reads PDFs with rustium-pdf, a pure-Rust interpreter, so there is no native library to fetch, point an environment variable at, or match versions with. The CLI is a single static binary and the wheel is self-contained.

pdfium — the Chromium PDF engine, behind FFI — remains available as an opt-in feature. It is the reference the pure-Rust backend is measured against, and an escape hatch for documents rustium cannot yet read. Asking for it means supplying it:

cargo build --release --no-default-features --features pdfium
scripts/fetch-pdfium.sh     # or set PDFIUM_DYNAMIC_LIB_PATH yourself

Getting started

scripts/fetch-corpus.sh     # evaluation corpus of arXiv papers, not committed
scripts/build.sh            # cargo build --release, plus installing the Python extension

# Convert.
./target/release/rustypaper convert corpus/resnet.pdf
./target/release/rustypaper convert corpus/resnet.pdf --format typst --assets figures/
./target/release/rustypaper convert corpus/*.pdf --out out/        # batch
./target/release/rustypaper convert paper.pdf --caveman=hard       # -25% words for LLM ingestion

# Diagnostics.
./target/release/rustypaper probe corpus/resnet.pdf --pages   # counts, fonts, detected gutters
./target/release/rustypaper text  corpus/resnet.pdf --geometry # reconstructed lines
./target/release/rustypaper dump  corpus/resnet.pdf --page 0 --pretty

probe prints per-page counts, the font histogram and the detected gutters. Gutters are the first thing to check when a two-column paper comes out interleaved; the font histogram is the first thing to check when text comes out wrong.

Design

Read docs/ARCHITECTURE.md for the pipeline, the coordinate convention, and the eight things M0 and M1 discovered the hard way — most importantly that pdfium-render's thread_safe feature does no locking at all.

The short version: the backend produces a PageRaw of glyphs, paths and images, and every later stage is a pass over an IR. The Document JSON is the contract; Markdown, Typst and plain text are renderings of it.
speed 3.8–7.0 ms/page, single process
corpus 10 papers: ML, pure maths, physics, biology, statistics
memory 17–31 MB peak for a whole paper
footprint a 2.3 MB static binary, no native library, no models

Unicode repair turned out not to be needed — glyph-name fallback already resolves TeX ligatures, so the tables the plan budgeted for were dropped. De-hyphenation could not work the way the plan assumed either: soft line-break hyphens are removed before the text reaches the pipeline, so there is no hyphen to key off. Words split across a break are rejoined using the document's own vocabulary instead, which needs no word list and knows the paper's jargon.

Maths is reconstructed geometrically, MaxTract-style, from exact glyph identities and positions rather than by OCR — a born-digital PDF hands you perfect character information, so image-to- LaTeX models are solving a problem this pipeline does not have. Equations carry a confidence score and fall back to a rendered crop rather than emitting confident-looking nonsense.

Scanned documents are explicitly out of scope: extract fails with Error::Scanned rather than pretending.

Milestones

status
M0 Backend, PageRaw, CLI, corpus, thread-safety spike done
M1 Lines/words, furniture removal, columns, reading order → Markdown done
M2 Figures, captions, footnotes, lists, de-hyphenation done
M3 Tables done
M4 Maths detection and reconstruction done
M5 References and citation linking done
M6 Typst emitter, performance pass, batch mode done

Python

The core is Rust; the tooling around it is Python, because evaluation, corpus management and comparison against other converters are scripting jobs.

scripts/build.sh
PYTHONPATH=python python3 -c "
import rustypaper
print(rustypaper.to_markdown('corpus/resnet.pdf')[:80])
doc = rustypaper.to_document('corpus/resnet.pdf')   # the document model as a dict
"

ScannedDocument is raised for image-only PDFs, so callers can route those to an OCR pipeline instead. Conversion releases the GIL.

Evaluation

Quality is measured, not eyeballed. Papers submitted to arXiv as TeX source come with the prose their PDF was rendered from, which is free ground truth for exactly this document class — for the subset of papers that have it. PDF-only submissions have none, and are reported as skipped rather than scored.

cd eval && PYTHONPATH=.:../python python3 -m rustypaper_eval

Current scores across the nine scorable papers:

metric value what it says
prose bigram recall 0.894 prose comes out right, in the right order
equation recall 0.375 most display equations are not found
equation fidelity 0.557 those that are found are roughly half right
tables 57 found / 44 in source badly distributed: 0 on one paper, over-detected on another

The maths numbers are the honest state of the differentiator, and they are the project's weakest point — see docs/ARCHITECTURE.md. See eval/README.md for what the metrics mean and why plain edit distance is the wrong primary measure here.

Testing

cargo test                                    # unit, robustness and corpus tests; corpus tests
                                              # skip if corpus/ is empty
python3 -m unittest discover -s eval/tests    # the eval harness's own tests
cd eval && PYTHONPATH=.:../python python3 -m rustypaper_eval --baseline baseline.json

Integration tests live in rustypaper/tests/corpus.rs and run against real papers. They skip rather than fail when the corpus is absent, so a fresh clone is green.

Licence

MIT OR Apache-2.0.

That covers everything a default build contains, the published crate and the published wheels included: the dependency tree is Rust, and rustium-pdf is MIT OR Apache-2.0 as well.

The pdfium feature is the exception, and only for whoever turns it on. pdfium is BSD-3-Clause and vendors further third-party code (FreeType, libjpeg-turbo, zlib, ICU and others) under their own terms. After running scripts/fetch-pdfium.sh those notices are in vendor/pdfium/licenses/, and vendor/pdfium/LICENSE-packaging is the MIT licence of pdfium-binaries, which builds and packages it.

If you redistribute a binary that bundles libpdfium.so, ship vendor/pdfium/licenses/ with it. Nothing released from this repository does.

Project links

Data verified by PyPI on Aug 2, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.2.1)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for pgarrett from gravatar.com pgarrett

GitHub Statistics

Data verified by PyPI on Aug 2, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT OR Apache-2.0

Requires

Python >=3.9

Tags

pdf markdown typst science papers extraction

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rustypaper-0.1.1.tar.gz (121.0 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

rustypaper-0.1.1-cp39-abi3-win_amd64.whl (1.0 MB view details)

Uploaded CPython 3.9+Windows x86-64

rustypaper-0.1.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ x86-64

rustypaper-0.1.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.1 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64

rustypaper-0.1.1-cp39-abi3-macosx_11_0_arm64.whl (1.0 MB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

rustypaper-0.1.1-cp39-abi3-macosx_10_12_x86_64.whl (1.1 MB view details)

Uploaded CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file rustypaper-0.1.1.tar.gz.

File metadata

  • Download URL: rustypaper-0.1.1.tar.gz
  • Upload date:
  • Size: 121.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for rustypaper-0.1.1.tar.gz
Algorithm Hash digest
SHA256 8701cbea6c12f8fb24fc88b27f18ac1236e82725ab3382ce66314efcad74a73f
MD5 e9f2511b507e1ee67b8b71aa369ef5a1
BLAKE2b-256 e874270d39feaee532f88e06d8d0c16dee31cdda690b84aeaa5fb147ed87a472

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.1.1.tar.gz:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.1.1-cp39-abi3-win_amd64.whl.

File metadata

  • Download URL: rustypaper-0.1.1-cp39-abi3-win_amd64.whl
  • Upload date:
  • Size: 1.0 MB
  • Tags: CPython 3.9+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for rustypaper-0.1.1-cp39-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 de7e2431b98d3deeb662dab16f5120e047619cc8f6232f92b52b087aee20bc13
MD5 559760383cc669076a0e7fdbfbd376a6
BLAKE2b-256 df73a8c9e58a08986445d3526e7326b3bccdcf010cec8f34b872f8ccf03de897

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.1.1-cp39-abi3-win_amd64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.1.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rustypaper-0.1.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f69e53a40cd345a89d31ee5966ab3bf9e4435fc5fc1ce9291c1c8ca8e402c356
MD5 73a5dccf55d67892f1147fc789b288cb
BLAKE2b-256 d8a43128c96cdf40231cc78f0077dc85288fbfb920424bbe6f087fa0026dc458

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.1.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.1.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for rustypaper-0.1.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 b1a902a8967a122526c2656db08681262bd2cabb6994280ac263ba7920e776af
MD5 3924c31047974e2364a828f8fd21377a
BLAKE2b-256 6b9f8f43df2fc171478d35e120d2179582cac2851caa9433a2c4826b0dfe78f1

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.1.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.1.1-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rustypaper-0.1.1-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 c13039e4e2b74e83cb822b5ce18c0b161faa7ae5701c67d6af0cf8f8f66b0494
MD5 acafbbe1d0e9ad2dd28b48d5b8ad1184
BLAKE2b-256 d5510d5e7d0222b4fe8b08e754024dadd54561991fa82ea2356deaf4c00bc307

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.1.1-cp39-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.1.1-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for rustypaper-0.1.1-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 410497e0704215ff3664446f1e8b5eb0dede649f1fa97df193081b62c7c88993
MD5 8bf5c39e836ede0d096e7cda3fd9f861
BLAKE2b-256 91ae167e41faf8a2cddec94d934f88df1785c82738ec00cb8859414cd1e71ccf

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.1.1-cp39-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.1

6 files

0.2.0

6 files

This release

0.1.1 This release

6 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page