PaperMinerToolkit banner

Tests Coverage PyPI version Python 3.11 or newer Documentation

PaperMinerToolkit

PaperMinerToolkit builds scientific-paper corpora and extracts structured, recipe-defined data with configurable text and vision models. It searches Elsevier/Scopus, CORE, OpenAlex, PubMed, arXiv, medRxiv, bioRxiv, and chemRxiv; supplements paper metadata from Crossref, OpenAlex, PubMed, arXiv, medRxiv, bioRxiv, and chemRxiv, and imports an author's works from Crossref; downloads abstracts, full text, and PDFs; supports persistent regex and LDA topic filters; and stores source content and pipeline state in SQLite.

The complete user guide, CLI reference, Python API, HPC instructions, and rendered notebooks live in the documentation source. The repository is ready for Read the Docs; a hosted link will be added after the project is imported.

Installation

PaperMinerToolkit requires Python 3.11 or newer:

git clone https://github.com/SMTG-Bham/PaperMinerToolkit.git
cd PaperMinerToolkit
python -m pip install -e .

Quickstart

Configure a text model and any search/download credentials you need, then run a small workflow:

pmt config model text --provider openai --model YOUR_TEXT_MODEL

pmt search "lithium solid electrolyte" papers.db \
  --source openalex --count 25
pmt enrich papers.db
pmt download papers.db --format abstract

pmt scrape papers.db sse \
  --mode abstract \
  --output temp_scraped_materials.csv

pmt store papers.db \
  temp_scraped_materials.csv \
  materials.csv \
  sse \
  --assume-yes

Use pmt corpus stats papers.db to inspect stored content and pmt status papers.db to inspect pipeline progress.

Documentation

Install and build the Sphinx site locally:

python -m pip install -e '.[docs]'
make -C docs html

Open docs/_build/html/index.html. Notebook templates for OpenAI, Anthropic, local Qwen/vLLM, LDA model selection, temporal trends, and hybrid filtering are under docs/examples/.

Testing

python -m pip install -e '.[test]'
ruff check paperminertoolkit tests
pytest

PaperMinerToolkit is currently alpha software. Keep the corpus database, recipe, model configuration, intermediate extraction CSV, and final results together so a workflow can be reviewed and reproduced.

Project links

Data verified by PyPI on Aug 26, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for osoulas from gravatar.com osoulas

Credits

Author: Oskar Soulas

GitHub Statistics

Data verified by PyPI on Aug 26, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

License expression

MIT
View SPDX License List

Requires

Python >=3.11

Provides Extra

test docs

Tags

research scientific-papers materials text-mining llm

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

paperminertoolkit-1.0.0.tar.gz (379.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

paperminertoolkit-1.0.0-py3-none-any.whl (235.8 kB view details)

Uploaded Python 3

File details

Details for the file paperminertoolkit-1.0.0.tar.gz.

File metadata

  • Download URL: paperminertoolkit-1.0.0.tar.gz
  • Upload date:
  • Size: 379.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for paperminertoolkit-1.0.0.tar.gz
Algorithm Hash digest
SHA256 b515ce4cfa3fff92fc12a4f3df276166835767e9321513d1e52e71f2b12a4cc0
MD5 370410a2c0f127afca46d91e4b4d2850
BLAKE2b-256 ad438d531748b2b8c36b99c3282c8177089c677614b04dc2076ef7b2dbf98c9f

See more details on using hashes here.

Provenance

The following attestation bundles were made for paperminertoolkit-1.0.0.tar.gz:

Publisher: publish.yml on SMTG-Bham/PaperMinerToolkit

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file paperminertoolkit-1.0.0-py3-none-any.whl.

File metadata

File hashes

Hashes for paperminertoolkit-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 1c9c4adf4ef53ec06b47f114d01c2a61ef7e634d523c9b7270ba0a9f63760777
MD5 0c40306d26e5dde10391eec76d3dd869
BLAKE2b-256 e3abc266d45a5595180f2096991589386aac0de9a8dd96c2ecdb6a57bf39db51

See more details on using hashes here.

Provenance

The following attestation bundles were made for paperminertoolkit-1.0.0-py3-none-any.whl:

Publisher: publish.yml on SMTG-Bham/PaperMinerToolkit

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

1.0.0 This release

2 files

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page