paperextract
Turn a folder of scientific PDFs into a searchable, citable, machine-readable library, locally, without losing the equations, tables or figures.
paperextract is built for technical literature, where a misread exponent or a shifted table cell matters. It runs state-of-the-art document models on your own machine or cluster, keeps every result traceable to the page it came from, and tells you what it could not verify instead of guessing.
paper.pdf ──► Travers_2019_HighEnergyPulse/
├── paper.md Markdown with LaTeX equations, tables, figures
├── document.json every block with its page, position and source
├── figures/ tables/ crops, captions, tables as HTML, JSON and CSV
├── citation.bib identity validated against Crossref or DataCite
├── validation.json what could not be verified
└── original/ your PDF, byte for byte
Features
- Equations, tables and figures as data. Display and inline math as LaTeX, table cells exactly as printed, figures with their panels and captions, supplements linked to the main text.
- Nothing invented. Values are never repaired or filled in; original files are preserved with hashes; every doubt is recorded as a finding.
- Real bibliographic identity. DOIs, titles and authors checked against Crossref, DataCite and arXiv, with BibTeX, clear names, and document versions (preprint, accepted manuscript, published) kept apart.
- Duplicates caught before extraction. Identical files, copies with the same text and shared DOIs are found before any model runs.
- A library you can search and share. Full-text search, lookup by DOI or
file,
library.bib, portable folders with relative links, and an agent skill so AI assistants can find and cite your papers. - Figure descriptions. Optional, clearly labelled machine descriptions of every figure from a local vision-language model or the Claude API.
- From laptop to cluster. MinerU, Docling or Marker in isolated environments on a Mac or Linux; GPU batches on Slurm clusters, resumable and published in one step. 162 papers took under an hour on four GPUs.
Install
Requires uv and macOS on Apple silicon or Linux on x86_64.
git clone https://github.com/jtravs/paperextract.git && cd paperextract
uv sync --locked
uv sync --directory workers/mineru --locked # add --extra cuda on an NVIDIA GPU
uv run paperextract models fetch mineru # or mineru-cuda; pinned and verified
Use
uv run paperextract paper.pdf --library ~/papers # one paper
uv run paperextract batch incoming/ --library ~/papers # a folder, duplicates handled
uv run paperextract search "soliton self-compression" --library ~/papers
uv run paperextract describe --all --library ~/papers # optional figure descriptions
uv run paperextract attach Travers_2019_HighEnergyPulse si-data.xlsx --library ~/papers
Documentation
The manual is at paperextract.readthedocs.io:
- Getting started: installation, a first extraction and what it produces
- Worked example: a real paper followed from PDF to library, with the frozen input and output in
examples/ - Command line: every command and setting
- Running on a Slurm cluster: GPU batches
- Figure descriptions
- Known limitations: read before relying on a result
- Model licences: the terms of each backend's models
To build it locally, run uv run poe docs and open
docs/_build/html/index.html.
Status
paperextract is alpha software. The pipeline is complete and has been run on about 170 papers on macOS and on NVIDIA A40 GPUs; accuracy has not yet been measured against a reviewed reference set. See the changelog.
Contributing
uv sync --locked
uv run poe check # lint, types, docstrings, spelling, tests, manual
The default test suite is offline and needs no models. See AGENTS.md for conventions and tooling for the checks.
Licence
Apache License 2.0; see LICENSE and NOTICE. paperextract distributes no model weights: each backend's models keep their own licences (summarized in model licences), and papers keep their publishers' copyright.
Metadata
Release files for paperextract 0.3.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| paperextract-0.3.1.tar.gz | 848.7 kB | Without provenance | Details for paperextract-0.3.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| paperextract-0.3.1-py3-none-any.whl | Python 3 | none | any | Details for paperextract-0.3.1-py3-none-any.whl |
Total release size: 1.1 MB
Release files / paperextract-0.3.1.tar.gz
| Download URL | paperextract-0.3.1.tar.gz |
|---|---|
| Size | 848.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
2e132b22967a7e7e75a8a86d641ad6b52a35b0ad2f0f2f816a62647851aef6e8
|
|
BLAKE2b-256 checksum How to use checksums |
e6a7cfa364110b8c942bbefb43af9b09a3492f9aab58d057485fa878e8aabf57
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.6 {"installer":{"name":"uv","version":"0.12.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / paperextract-0.3.1-py3-none-any.whl
| Download URL | paperextract-0.3.1-py3-none-any.whl |
|---|---|
| Size | 294.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
4dc35997b1c81f3ca5e11efd54c7c96391da13e766f0e8692136e6f605ea087e
|
|
BLAKE2b-256 checksum How to use checksums |
5a33ec44df040de5245acad979077809ee84f26e50dda67e174bbd2a9b6bad6f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.6 {"installer":{"name":"uv","version":"0.12.6","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for paperextract 0.3.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.