Skip to main content

paperextract

Check Coverage Docs PyPI Python Ruff basedpyright License: Apache-2.0

Turn a folder of scientific PDFs into a searchable, citable, machine-readable library, locally, without losing the equations, tables or figures.

paperextract is built for technical literature, where a misread exponent or a shifted table cell matters. It runs state-of-the-art document models on your own machine or cluster, keeps every result traceable to the page it came from, and tells you what it could not verify instead of guessing.

paper.pdf  ──►  Travers_2019_HighEnergyPulse/
                ├── paper.md          Markdown with LaTeX equations, tables, figures
                ├── document.json     every block with its page, position and source
                ├── figures/ tables/  crops, captions, tables as HTML, JSON and CSV
                ├── citation.bib      identity validated against Crossref or DataCite
                ├── validation.json   what could not be verified
                └── original/         your PDF, byte for byte

Features

  • Equations, tables and figures as data. Display and inline math as LaTeX, table cells exactly as printed, figures with their panels and captions, supplements linked to the main text.
  • Nothing invented. Values are never repaired or filled in; original files are preserved with hashes; every doubt is recorded as a finding.
  • Real bibliographic identity. DOIs, titles and authors checked against Crossref, DataCite and arXiv, with BibTeX, clear names, and document versions (preprint, accepted manuscript, published) kept apart.
  • Duplicates caught before extraction. Identical files, copies with the same text and shared DOIs are found before any model runs.
  • A library you can search and share. Full-text search, lookup by DOI or file, library.bib, portable folders with relative links, and an agent skill so AI assistants can find and cite your papers.
  • Figure descriptions. Optional, clearly labelled machine descriptions of every figure from a local vision-language model or the Claude API.
  • From laptop to cluster. MinerU, Docling or Marker in isolated environments on a Mac or Linux; GPU batches on Slurm clusters, resumable and published in one step. 162 papers took under an hour on four GPUs.

Install

Requires uv and macOS on Apple silicon or Linux on x86_64.

git clone https://github.com/jtravs/paperextract.git && cd paperextract
uv sync --locked
uv sync --directory workers/mineru --locked       # add --extra cuda on an NVIDIA GPU
uv run paperextract models fetch mineru           # or mineru-cuda; pinned and verified

Use

uv run paperextract paper.pdf --library ~/papers          # one paper
uv run paperextract batch incoming/ --library ~/papers    # a folder, duplicates handled
uv run paperextract search "soliton self-compression" --library ~/papers
uv run paperextract describe --all --library ~/papers     # optional figure descriptions
uv run paperextract attach Travers_2019_HighEnergyPulse si-data.xlsx --library ~/papers

Documentation

The manual is at paperextract.readthedocs.io:

To build it locally, run uv run poe docs and open docs/_build/html/index.html.

Status

paperextract is alpha software. The pipeline is complete and has been run on about 170 papers on macOS and on NVIDIA A40 GPUs; accuracy has not yet been measured against a reviewed reference set. See the changelog.

Contributing

uv sync --locked
uv run poe check      # lint, types, docstrings, spelling, tests, manual

The default test suite is offline and needs no models. See AGENTS.md for conventions and tooling for the checks.

Licence

Apache License 2.0; see LICENSE and NOTICE. paperextract distributes no model weights: each backend's models keep their own licences (summarized in model licences), and papers keep their publishers' copyright.

Metadata

Release files for paperextract 0.3.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for paperextract 0.3.1
File Size Uploaded Provenance Actions
paperextract-0.3.1.tar.gz 848.7 kB Without provenance Details for paperextract-0.3.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for paperextract 0.3.1
File Interpreter ABI Platform Actions
paperextract-0.3.1-py3-none-any.whl Python 3 none any Details for paperextract-0.3.1-py3-none-any.whl

Total release size: 1.1 MB

Release files / paperextract-0.3.1.tar.gz

Download URL paperextract-0.3.1.tar.gz
Size 848.7 kB
Tags Source
SHA-256 checksum
How to use checksums
2e132b22967a7e7e75a8a86d641ad6b52a35b0ad2f0f2f816a62647851aef6e8
BLAKE2b-256 checksum
How to use checksums
e6a7cfa364110b8c942bbefb43af9b09a3492f9aab58d057485fa878e8aabf57
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.12.6 {"installer":{"name":"uv","version":"0.12.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / paperextract-0.3.1-py3-none-any.whl

Download URL paperextract-0.3.1-py3-none-any.whl
Size 294.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
4dc35997b1c81f3ca5e11efd54c7c96391da13e766f0e8692136e6f605ea087e
BLAKE2b-256 checksum
How to use checksums
5a33ec44df040de5245acad979077809ee84f26e50dda67e174bbd2a9b6bad6f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via uv/0.12.6 {"installer":{"name":"uv","version":"0.12.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for paperextract 0.3.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page