Skip to main content

paperextract

Check Coverage Docs PyPI Python Ruff basedpyright License: Apache-2.0

Turn a folder of scientific PDFs into a searchable, citable, machine-readable library, locally, without losing the equations, tables or figures.

paperextract is built for technical literature, where a misread exponent or a shifted table cell matters. It runs state-of-the-art document models on your own machine or cluster, keeps every result traceable to the page it came from, and tells you what it could not verify instead of guessing.

paper.pdf  ──►  Travers_2019_HighEnergyPulse/
                ├── paper.md          Markdown with LaTeX equations, tables, figures
                ├── document.json     every block with its page, position and source
                ├── figures/ tables/  crops, captions, tables as HTML, JSON and CSV
                ├── citation.bib      identity validated against Crossref or DataCite
                ├── validation.json   what could not be verified
                └── original/         your PDF, byte for byte

Features

  • Equations, tables and figures as data. Display and inline math as LaTeX, table cells exactly as printed, figures with their panels and captions, supplements linked to the main text.
  • Nothing invented. Values are never repaired or filled in; original files are preserved with hashes; every doubt is recorded as a finding.
  • Real bibliographic identity. DOIs, titles and authors checked against Crossref, DataCite and arXiv, with BibTeX, clear names, and document versions (preprint, accepted manuscript, published) kept apart.
  • Duplicates caught before extraction. Identical files, copies with the same text and shared DOIs are found before any model runs.
  • A library you can search and share. Full-text search, lookup by DOI or file, library.bib, portable folders with relative links, and an agent skill so AI assistants can find and cite your papers.
  • Figure descriptions. Optional, clearly labelled machine descriptions of every figure from a local vision-language model or the Claude API.
  • From laptop to cluster. MinerU, Docling or Marker in isolated environments on a Mac or Linux; GPU batches on Slurm clusters, resumable and published in one step. 162 papers took under an hour on four GPUs.

Install

Requires uv and macOS on Apple silicon or Linux on x86_64.

git clone https://github.com/jtravs/paperextract.git && cd paperextract
uv sync --locked
uv sync --directory workers/mineru --locked       # add --extra cuda on an NVIDIA GPU
uv run paperextract models fetch mineru           # or mineru-cuda; pinned and verified

Use

uv run paperextract paper.pdf --library ~/papers          # one paper
uv run paperextract batch incoming/ --library ~/papers    # a folder, duplicates handled
uv run paperextract search "soliton self-compression" --library ~/papers
uv run paperextract describe --all --library ~/papers     # optional figure descriptions
uv run paperextract attach Travers_2019_HighEnergyPulse si-data.xlsx --library ~/papers

Documentation

The manual is at paperextract.readthedocs.io:

To build it locally, run uv run poe docs and open docs/_build/html/index.html.

Status

paperextract is alpha software. The pipeline is complete and has been run on about 170 papers on macOS and on NVIDIA A40 GPUs; accuracy has not yet been measured against a reviewed reference set. See the changelog.

Contributing

uv sync --locked
uv run poe check      # lint, types, docstrings, spelling, tests, manual

The default test suite is offline and needs no models. See AGENTS.md for conventions and tooling for the checks.

Licence

Apache License 2.0; see LICENSE and NOTICE. paperextract distributes no model weights: each backend's models keep their own licences (summarized in model licences), and papers keep their publishers' copyright.

Metadata

Release files for paperextract 0.3.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for paperextract 0.3.2
File Size Uploaded Provenance Actions
paperextract-0.3.2.tar.gz 849.0 kB Without provenance Details for paperextract-0.3.2.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for paperextract 0.3.2
File Interpreter ABI Platform Actions
paperextract-0.3.2-py3-none-any.whl Python 3 none any Details for paperextract-0.3.2-py3-none-any.whl

Total release size: 1.1 MB

Release files / paperextract-0.3.2.tar.gz

Download URL paperextract-0.3.2.tar.gz
Size 849.0 kB
Tags Source
SHA-256 checksum
How to use checksums
7540390540ff85c6dd5428bed188602666da771f0933d585304fed97182a05f5
BLAKE2b-256 checksum
How to use checksums
c5bb266d15d430aec14af96165e78e67f345261a10b1501ea2c99fc42ce33ee4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.12.6 {"installer":{"name":"uv","version":"0.12.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / paperextract-0.3.2-py3-none-any.whl

Download URL paperextract-0.3.2-py3-none-any.whl
Size 294.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
cbf2fc0d4a9a48d949e419d13b4d6c256d3e001abb5e78defca6cba5a0693e21
BLAKE2b-256 checksum
How to use checksums
ce09932c0c1be94225a1e9d0e820939a73cedcb0b6cbb53b2ea10c9d4cca99b5
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.12.6 {"installer":{"name":"uv","version":"0.12.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for paperextract 0.3.2

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page