Skip to main content

paper-harvest

Collect arXiv papers into a unified article directory: PDF, TeX source, optional Markdown, and a structured manifest.

Python 3.9+, stdlib only at runtime. Two ways to use it:

Audience How
Terminal / automation uvx paper-harvest (PyPI)
Cursor / Agent Install the skill zip

Quick start

uvx paper-harvest 'https://arxiv.org/abs/2403.18074' --json

From a local checkout:

uvx --from . paper-harvest 'https://arxiv.org/abs/2403.18074' --json

Install permanently:

uv tool install paper-harvest
paper-harvest 'https://arxiv.org/abs/2403.18074' --json

Input: arXiv URLs only (arxiv.org, abs / pdf / src).

Output

<article_id>/
  pdf/<article_id>.pdf
  tex/
    arXiv-<id>.tar.gz
    source/
  md/
    <article_id>.md
    assets/
    mineru_result.json
  harvest_result.json

Each step reports ok, skipped, or failed in harvest_result.json. Failed steps do not remove earlier artifacts.

CLI flags

Flag Effect
--target-dir <dir> Output root (default: .)
--skip-tex Skip TeX source download
--skip-md Skip Markdown extraction
--force Overwrite existing md/
--json Print structured JSON to stdout
--timeout <sec> Download timeout (default: 120)

Markdown (MinerU)

Markdown is optional and uses MinerU.

  1. With token → precision API (better quality, larger PDFs)
  2. Without token → lightweight API (PDFs ≤ 10 MB)
  3. Otherwise → md is skipped or failed; PDF and TeX are kept

Configure a token via any one of:

export MINERU_API_TOKEN='your-token'
echo 'your-token' > ~/.mineru_token && chmod 600 ~/.mineru_token

Also checked: ./.mineru_token, ~/.config/mineru/token. Never commit token files.

Re-run Markdown only after adding a token:

uvx paper-harvest '<same-arxiv-url>' --skip-tex --force --json

Cursor Skill

The skill is Agent instructions only; execution still goes through the CLI.

Install — download paper-harvest-<version>-skill.zip from releases:

unzip paper-harvest-0.2.0-skill.zip -d ~/.cursor/skills/

Build locally:

uv build
python tools/build_skill_zip.py
# → dist/paper-harvest-0.2.0-skill.zip

Skill sources live in skill/ (SKILL.md, references/, scripts/harvest.sh).

Project layout

paper-harvest/
  pyproject.toml
  src/paper_harvest/     # CLI + core
  skill/                   # Agent skill (shipped as zip)
  docs/spec.md             # product specification
  tools/build_skill_zip.py
  tests/

Development

uv sync --extra dev
uv run pytest
uv build
uvx --from . paper-harvest --help

Publish

  1. Bump version in pyproject.toml and src/paper_harvest/__init__.py
  2. uv run pytest && uv build && python tools/build_skill_zip.py
  3. Tag v0.x.y, push, attach dist/*-skill.zip to GitHub Release
  4. uv publish for PyPI (uvx paper-harvest)

Docs

  • docs/spec.md — product / engineering specification
  • skill/references/ — Agent-facing reference (manifest schema, MinerU setup)

Metadata

Release files for paper-harvest 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for paper-harvest 0.2.0
File Size Uploaded Provenance Actions
paper_harvest-0.2.0.tar.gz 13.4 kB With provenance Details for paper_harvest-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for paper-harvest 0.2.0
File Interpreter ABI Platform Actions
paper_harvest-0.2.0-py3-none-any.whl Python 3 none any Details for paper_harvest-0.2.0-py3-none-any.whl

Total release size: 28.4 kB

Release files / paper_harvest-0.2.0.tar.gz

Download URL paper_harvest-0.2.0.tar.gz
Size 13.4 kB
Tags Source
SHA-256 checksum
How to use checksums
dd3013aba8eed14fa22701689d032b7ba72b17043b32eff728659ab936b6e1d4
BLAKE2b-256 checksum
How to use checksums
d82fbc5ea35ba7c0b9eec491a200320b61b554799c125347584a79ce13a57166
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 30, 2026.

Transparency log

Release files / paper_harvest-0.2.0-py3-none-any.whl

Download URL paper_harvest-0.2.0-py3-none-any.whl
Size 15.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
0c7f047b9a4eea19bc9d964c7f177e6a2ad049221ecd7327a950289eec85ef85
BLAKE2b-256 checksum
How to use checksums
7fc9a5e845c0439038e64aac18865b09486072c70a634dadfa140c1cfcb477ae
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 30, 2026.

Transparency log

Security for paper-harvest 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for paper_harvest-0.2.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 30, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.2.0 This release

2 release files Full provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page