rustypaper

Structure-aware conversion of born-digital scientific PDFs to Markdown, Typst, JSON and plain text. Headings, reading order, figures, tables, equations and references, on a CPU, with no models and no native libraries.

The good open-source converters (Marker, MinerU, Docling, Nougat) are Python stacks that want a GPU; GROBID is CPU-only and fast but is a JVM service that emits TEI and ignores maths. The Rust crates that exist are generic text extractors with no notion of a paper. This aims at the gap: structure-aware, maths-aware, CPU-only, single binary.

Status: feature complete. Converts one- and two-column papers to Markdown, Typst, plain text or JSON, with reading order, figures, tables, mathematics and references.

Install

cargo install rustypaper          # the command-line tool

rustypaper = "0.1"            # the library

pip install rustypaper        # the Python bindings

Nothing else to install. PDFs are read with rustium-pdf, a pure-Rust interpreter, so there is no native library to fetch, point an environment variable at, or match versions with. ldd on the binary shows libc, libm and libgcc and nothing else; the wheel is the extension module and the Python package around it, with no C library travelling beside it.

Getting started

scripts/fetch-corpus.sh     # evaluation corpus of arXiv papers, not committed
scripts/build.sh            # cargo build --release, plus installing the Python extension

# Convert.
./target/release/rustypaper convert corpus/resnet.pdf
./target/release/rustypaper convert corpus/resnet.pdf --format typst --assets figures/
./target/release/rustypaper convert corpus/*.pdf --out out/        # batch
./target/release/rustypaper convert paper.pdf --caveman=hard       # -24% words for LLM ingestion

# Diagnostics.
./target/release/rustypaper probe corpus/resnet.pdf --pages   # counts, fonts, detected gutters
./target/release/rustypaper text  corpus/resnet.pdf --geometry # reconstructed lines
./target/release/rustypaper dump  corpus/resnet.pdf --page 0 --pretty

probe prints per-page counts, the font histogram and the detected gutters. Gutters are the first thing to check when a two-column paper comes out interleaved; the font histogram is the first thing to check when text comes out wrong.

Design

Read docs/ARCHITECTURE.md for the pipeline, the coordinate convention, the PageSource boundary, and the things the corpus taught this converter the hard way.

The short version: the backend produces a PageRaw of glyphs, paths and images, and every later stage is a pass over an IR. The Document JSON is the contract; Markdown, Typst and plain text are renderings of it.
speed 2.5–8.7 ms/page, single process
corpus 16 papers in 10 template families: ML, pure maths, physics, biology, medicine, statistics
memory 13–30 MB peak for a whole paper
footprint a 3.2 MB binary, no native library, no models

Unicode repair turned out not to be needed — glyph-name fallback already resolves TeX ligatures, so the tables the plan budgeted for were dropped. De-hyphenation needed a second mechanism the plan did not anticipate: the page is read as written, so a soft line-break hyphen is usually there and is preferred when it is, but where a document leaves none the words split across the break are rejoined using the document's own vocabulary, which needs no word list and knows the paper's jargon.

Maths is reconstructed geometrically, MaxTract-style, from exact glyph identities and positions rather than by OCR — a born-digital PDF hands you perfect character information, so image-to- LaTeX models are solving a problem this pipeline does not have. Equations carry a confidence score and fall back to a rendered crop rather than emitting confident-looking nonsense.

Scanned documents are explicitly out of scope: extract fails with Error::Scanned rather than pretending.

Milestones

status
M0 Backend, PageRaw, CLI, corpus, thread-safety spike done
M1 Lines/words, furniture removal, columns, reading order → Markdown done
M2 Figures, captions, footnotes, lists, de-hyphenation done
M3 Tables done
M4 Maths detection and reconstruction done
M5 References and citation linking done
M6 Typst emitter, performance pass, batch mode done

Python

The core is Rust; the tooling around it is Python, because evaluation, corpus management and comparison against other converters are scripting jobs.

scripts/build.sh
PYTHONPATH=python python3 -c "
import rustypaper
print(rustypaper.to_markdown('corpus/resnet.pdf')[:80])
doc = rustypaper.to_document('corpus/resnet.pdf')   # the document model as a dict
for section in doc['sections']:                     # the outline, with block ranges
    print(section['level'], section['title'], section['start'], section['end'])

markdown, doc = rustypaper.convert('corpus/resnet.pdf')   # both, from one pipeline run
print(rustypaper.to_typst('corpus/resnet.pdf')[:80])
print(rustypaper.to_text('corpus/resnet.pdf')[:80])
"

sections is the document's outline: each entry has a title (None for the front matter that precedes every heading), a level, a half-open start/end range into blocks that includes its nested children, and the pages it spans. A consumer that wants the methods section can slice the blocks rather than re-deriving structure from the Markdown.

ScannedDocument is raised for image-only PDFs, so callers can route those to an OCR pipeline instead. Conversion releases the GIL, so several threads convert in parallel.

Evaluation

Quality is measured, not eyeballed. Papers submitted to arXiv as TeX source come with the prose their PDF was rendered from, which is free ground truth for exactly this document class — for the subset of papers that have it. PDF-only submissions have none, and are reported as skipped rather than scored.

cd eval && PYTHONPATH=.:../python python3 -m rustypaper_eval

Current scores across the fifteen scorable papers:

metric value what it says
prose bigram recall 0.900 prose comes out right, in the right order
equation recall 0.565 most display equations are found; templates vary widely
equation fidelity 0.677 the found ones are mostly right, not wholly right
tables 62 found / 90 in source five papers complete; ImageNet's 9/26 is the largest gap
references 971 found / 1174 in source eight bibliographies complete; topological's 163/368 is a column gap, not a parsing one
sections 206 matched / 264 in source the deficit is subsections set bold at body size

The maths numbers are the honest state of the differentiator, and they are the project's weakest point — see docs/ARCHITECTURE.md. See eval/README.md for what the metrics mean and why plain edit distance is the wrong primary measure here.

Testing

cargo test                                    # unit, robustness and corpus tests; corpus tests
                                              # skip if corpus/ is empty
python3 -m unittest discover -s eval/tests    # the eval harness's own tests
PYTHONPATH=python pytest python/tests -q      # the Python surface, against the corpus
cd eval && PYTHONPATH=.:../python python3 -m rustypaper_eval --baseline baseline.json

Integration tests live in rustypaper/tests/corpus.rs and run against real papers. They skip rather than fail when the corpus is absent, so a fresh clone is green.

Licence

MIT OR Apache-2.0.

That covers everything a build contains, the published crate and the published wheels included: the dependency tree is Rust, and rustium-pdf is MIT OR Apache-2.0 as well.

Project links

Data verified by PyPI on Aug 13, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.2.1)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for pgarrett from gravatar.com pgarrett

GitHub Statistics

Data verified by PyPI on Aug 13, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License

MIT OR Apache-2.0

Requires

Python >=3.9

Tags

pdf markdown typst science papers extraction

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

rustypaper-0.2.0.tar.gz (157.8 kB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

rustypaper-0.2.0-cp39-abi3-win_amd64.whl (1.1 MB view details)

Uploaded CPython 3.9+Windows x86-64

rustypaper-0.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ x86-64

rustypaper-0.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl (1.2 MB view details)

Uploaded CPython 3.9+manylinux: glibc 2.17+ ARM64

rustypaper-0.2.0-cp39-abi3-macosx_11_0_arm64.whl (1.1 MB view details)

Uploaded CPython 3.9+macOS 11.0+ ARM64

rustypaper-0.2.0-cp39-abi3-macosx_10_12_x86_64.whl (1.2 MB view details)

Uploaded CPython 3.9+macOS 10.12+ x86-64

File details

Details for the file rustypaper-0.2.0.tar.gz.

File metadata

  • Download URL: rustypaper-0.2.0.tar.gz
  • Upload date:
  • Size: 157.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for rustypaper-0.2.0.tar.gz
Algorithm Hash digest
SHA256 b1be8dc81e673f0e2889717760e813cfc13c5f1fc198955f4bd233410c464db9
MD5 8783dd754b07c91c8f4d8cd6d5a69f70
BLAKE2b-256 42aff29b3c4de845caed9eb2243baaad8720c564ce76dd0b15c65542b4df3120

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.0.tar.gz:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.2.0-cp39-abi3-win_amd64.whl.

File metadata

  • Download URL: rustypaper-0.2.0-cp39-abi3-win_amd64.whl
  • Upload date:
  • Size: 1.1 MB
  • Tags: CPython 3.9+, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for rustypaper-0.2.0-cp39-abi3-win_amd64.whl
Algorithm Hash digest
SHA256 3f1c68c3bd6ad0b4bd05a420d1f6d31eb87ebb0636b0590177a424edadaa68a5
MD5 74fda05f8d496662bdfd7bfe652241c6
BLAKE2b-256 df8675ddc81a29791188f6c2bd418361055226ed43a4706960fcbc334eed7ff7

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.0-cp39-abi3-win_amd64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

File metadata

File hashes

Hashes for rustypaper-0.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Algorithm Hash digest
SHA256 f6285619177744bede73f7848e212ec48da28bfd5d009f771c76533367780816
MD5 cc67c242cf52392a25ef4615192efc99
BLAKE2b-256 d4f412555c00323c073621431aa89b7d66da3011ff5643c3d2f2a9e9919c0314

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl.

File metadata

File hashes

Hashes for rustypaper-0.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
Algorithm Hash digest
SHA256 ee57ca167f61d7aa7e816c7d837b935fd75e7d6890836dfffd43c7a88b8e15fa
MD5 92a5427e6d9ce44255c07aac9e8e1c2d
BLAKE2b-256 fa872c986cff7170c253ac96fa5ae0d18f81843ef40cefa94c99a354d83da3cd

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.0-cp39-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.2.0-cp39-abi3-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for rustypaper-0.2.0-cp39-abi3-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 77d1ba53ffd80847cf462e2d881e99eb1847e0495747fa2e1e9da8bde7e2f7cd
MD5 21fe604c31d19b1516fd32288c6db6cb
BLAKE2b-256 57ab55c501bfb93eb0f27de03cdac235f349e526b990d9bccdb0f58dc04f0700

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.0-cp39-abi3-macosx_11_0_arm64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file rustypaper-0.2.0-cp39-abi3-macosx_10_12_x86_64.whl.

File metadata

File hashes

Hashes for rustypaper-0.2.0-cp39-abi3-macosx_10_12_x86_64.whl
Algorithm Hash digest
SHA256 61587b794e919e7bd56ea6db0d50fb451865fe3a09daba9177758391f6c1497d
MD5 c9497c2a83eb37641edf2445ed6829f1
BLAKE2b-256 0da66a71430c6f00a8dd68dcd1f078bdf55fcda770e557187262844e635f7bfc

See more details on using hashes here.

Provenance

The following attestation bundles were made for rustypaper-0.2.0-cp39-abi3-macosx_10_12_x86_64.whl:

Publisher: release.yml on pgarrett-scripps/rustypaper

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.2.1

6 files

This release

0.2.0 This release

6 files

0.1.1

6 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page