rustypaper

Structure-aware conversion of born-digital scientific PDFs to Markdown, Typst, JSON and plain text. Headings, reading order, figures, tables, equations and references, on a CPU, with no models and no native libraries.

The good open-source converters (Marker, MinerU, Docling, Nougat) are Python stacks that want a GPU; GROBID is CPU-only and fast but is a JVM service that emits TEI and ignores maths. The Rust crates that exist are generic text extractors with no notion of a paper. This aims at the gap: structure-aware, maths-aware, CPU-only, single binary.

Status: feature complete. Converts one- and two-column papers to Markdown, Typst, plain text or JSON, with reading order, figures, tables, mathematics and references.

Install

cargo install rustypaper          # the command-line tool

rustypaper = "0.2"            # the library

pip install rustypaper        # the Python bindings

Nothing else to install. PDFs are read with rustium-pdf, a pure-Rust interpreter, so there is no native library to fetch, point an environment variable at, or match versions with. ldd on the binary shows libc, libm and libgcc and nothing else; the wheel is the extension module and the Python package around it, with no C library travelling beside it.

Getting started

scripts/fetch-corpus.sh     # evaluation corpus of arXiv papers, not committed
scripts/build.sh            # cargo build --release, plus installing the Python extension

# Convert.
./target/release/rustypaper convert corpus/resnet.pdf
./target/release/rustypaper convert corpus/resnet.pdf --format typst --assets figures/
./target/release/rustypaper convert corpus/*.pdf --out out/        # batch
./target/release/rustypaper convert paper.pdf --caveman=hard       # -24% words for LLM ingestion

# Diagnostics.
./target/release/rustypaper probe corpus/resnet.pdf --pages   # counts, fonts, detected gutters
./target/release/rustypaper text  corpus/resnet.pdf --geometry # reconstructed lines
./target/release/rustypaper dump  corpus/resnet.pdf --page 0 --pretty

probe prints per-page counts, the font histogram and the detected gutters. Gutters are the first thing to check when a two-column paper comes out interleaved; the font histogram is the first thing to check when text comes out wrong.

Design

Read docs/ARCHITECTURE.md for the pipeline, the coordinate convention, the PageSource boundary, and the things the corpus taught this converter the hard way.

The short version: the backend produces a PageRaw of glyphs, paths and images, and every later stage is a pass over an IR. The Document JSON is the contract; Markdown, Typst and plain text are renderings of it.
speed 2.5–8.7 ms/page, single process
corpus 16 papers in 10 template families: ML, pure maths, physics, biology, medicine, statistics
memory 13–30 MB peak for a whole paper
footprint a 3.2 MB binary, no native library, no models

Unicode repair turned out not to be needed — glyph-name fallback already resolves TeX ligatures, so the tables the plan budgeted for were dropped. De-hyphenation needed a second mechanism the plan did not anticipate: the page is read as written, so a soft line-break hyphen is usually there and is preferred when it is, but where a document leaves none the words split across the break are rejoined using the document's own vocabulary, which needs no word list and knows the paper's jargon.

Maths is reconstructed geometrically, MaxTract-style, from exact glyph identities and positions rather than by OCR — a born-digital PDF hands you perfect character information, so image-to- LaTeX models are solving a problem this pipeline does not have. Equations carry a confidence score and fall back to a rendered crop rather than emitting confident-looking nonsense.

Scanned documents are explicitly out of scope: extract fails with Error::Scanned rather than pretending.

Milestones

status
M0 Backend, PageRaw, CLI, corpus, thread-safety spike done
M1 Lines/words, furniture removal, columns, reading order → Markdown done
M2 Figures, captions, footnotes, lists, de-hyphenation done
M3 Tables done
M4 Maths detection and reconstruction done
M5 References and citation linking done
M6 Typst emitter, performance pass, batch mode done

Python

The core is Rust; the tooling around it is Python, because evaluation, corpus management and comparison against other converters are scripting jobs.

scripts/build.sh
PYTHONPATH=python python3 -c "
import rustypaper
print(rustypaper.to_markdown('corpus/resnet.pdf')[:80])
doc = rustypaper.to_document('corpus/resnet.pdf')   # the document model as a dict
for section in doc['sections']:                     # the outline, with block ranges
    print(section['level'], section['title'], section['start'], section['end'])

markdown, doc = rustypaper.convert('corpus/resnet.pdf')   # both, from one pipeline run
print(rustypaper.to_typst('corpus/resnet.pdf')[:80])
print(rustypaper.to_text('corpus/resnet.pdf')[:80])
"

sections is the document's outline: each entry has a title (None for the front matter that precedes every heading), a level, a half-open start/end range into blocks that includes its nested children, and the pages it spans. A consumer that wants the methods section can slice the blocks rather than re-deriving structure from the Markdown.

ScannedDocument is raised for image-only PDFs, so callers can route those to an OCR pipeline instead. Conversion releases the GIL, so several threads convert in parallel.

Evaluation

Quality is measured, not eyeballed. Papers submitted to arXiv as TeX source come with the prose their PDF was rendered from, which is free ground truth for exactly this document class — for the subset of papers that have it. PDF-only submissions have none, and are reported as skipped rather than scored.

cd eval && PYTHONPATH=.:../python python3 -m rustypaper_eval

Current scores across the fifteen scorable papers:

metric value what it says
prose bigram recall 0.900 prose comes out right, in the right order
equation recall 0.565 most display equations are found; templates vary widely
equation fidelity 0.677 the found ones are mostly right, not wholly right
tables 62 found / 90 in source five papers complete; ImageNet's 9/26 is the largest gap
references 971 found / 1174 in source eight bibliographies complete; topological's 163/368 is a column gap, not a parsing one
sections 206 matched / 264 in source the deficit is subsections set bold at body size

The maths numbers are the honest state of the differentiator, and they are the project's weakest point — see docs/ARCHITECTURE.md. See eval/README.md for what the metrics mean and why plain edit distance is the wrong primary measure here.

Testing

cargo test                                    # unit, robustness and corpus tests; corpus tests
                                              # skip if corpus/ is empty
python3 -m unittest discover -s eval/tests    # the eval harness's own tests
PYTHONPATH=python pytest python/tests -q      # the Python surface, against the corpus
cd eval && PYTHONPATH=.:../python python3 -m rustypaper_eval --baseline baseline.json

Integration tests live in rustypaper/tests/corpus.rs and run against real papers. They skip rather than fail when the corpus is absent, so a fresh clone is green.

Licence

MIT OR Apache-2.0.

That covers everything a build contains, the published crate and the published wheels included: the dependency tree is Rust, and rustium-pdf is MIT OR Apache-2.0 as well.

Project links

Data verified by PyPI on Sep 5, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for pgarrett from gravatar.com pgarrett

GitHub Statistics

Data verified by PyPI on Sep 5, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT OR Apache-2.0

Requires

Python >=3.9

Tags

pdf markdown typst science papers extraction

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rustypaper-0.2.1.tar.gz (158.3 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

rustypaper-0.2.1-cp39-abi3-win_amd64.whl (1.1 MB view details)

Uploaded CPython 3.9+Windows x86-64

rustypaper-0.2.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ x86-64

rustypaper-0.2.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64

rustypaper-0.2.1-cp39-abi3-macosx_11_0_arm64.whl (1.1 MB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

rustypaper-0.2.1-cp39-abi3-macosx_10_12_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file rustypaper-0.2.1.tar.gz.

File metadata

  • Download URL: rustypaper-0.2.1.tar.gz
  • Upload date:
  • Size: 158.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for rustypaper-0.2.1.tar.gz
Algorithm Hash digest
SHA256 ccd3dbf6fd10597e48a41cf1fd0e499ad1954c00cc8341878df26f1c6fe54699
MD5 160849617499e9bfcab5cb9386f9b8e4
BLAKE2b-256 92d61990bda6f01ff1a1397fda3f3b50cab71b62a10c0c7b3161b54e560d5f09

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.1.tar.gz:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.2.1-cp39-abi3-win_amd64.whl.

File metadata

  • Download URL: rustypaper-0.2.1-cp39-abi3-win_amd64.whl
  • Upload date:
  • Size: 1.1 MB
  • Tags: CPython 3.9+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for rustypaper-0.2.1-cp39-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 95fa74f259cb84a0cdffaf272ab1905d85002b73a0b6a36c76de2b4ca2db0b26
MD5 25723af63bcaaede26015534fefb20a3
BLAKE2b-256 0ed28fed79ae82c65e10d1aa3818626de3dbd21aad43a93d50d0ce79e03ff354

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.1-cp39-abi3-win_amd64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.2.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rustypaper-0.2.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 9bd63b430d3c6256c16e116708ac39eda3cbae0299a4f5ad52d37c28f39f74c4
MD5 9ddc9b088a56ba7619de69b6d76e33eb
BLAKE2b-256 5734d0bc1a42fb60774d38144ddbc2d723f88e2e1b105e0d8bf5848c5e6588d8

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.1-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.2.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for rustypaper-0.2.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 14b7d758641de995d5133d8df97c9241c07e9cf107e23f9199fd78d6751b256f
MD5 df8684681c2fe017056044dbdd6c30c4
BLAKE2b-256 38e351e371f5d6ecaf4f41f73257b59324a2a11853b6c6ad934c43a365d03b9a

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.1-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.2.1-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rustypaper-0.2.1-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 30f991da6d289b8d64ee482a5e6b6c7f1e35296298a67372ee6c559d6d1d298a
MD5 829babf935fa0476f76044c3b819895f
BLAKE2b-256 fa5f09845585533f92e21d9048656b7597607eb84428f40f04783c3de703fdda

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.1-cp39-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.2.1-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for rustypaper-0.2.1-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 18043c396704f6b7f4206b895213d64202f7b73c55298a0834f132260922395d
MD5 f01abf685851b42d58de58c7c93f8298
BLAKE2b-256 57096435d6526ca3bde61c04e293f9b45ddf4ffa4acbd2ef183822c60b89f089

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.1-cp39-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

0.2.1 This release

6 files

0.2.0

6 files

0.1.1

6 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page