Data Gatherer
Data Gatherer is a Python library for automatically extracting dataset references from scientific publications. It processes full-text articles—whether in HTML or XML format—and uses both rule-based and LLM-based methods to identify and structure dataset citations.
What It Does
Parses scientific articles from open-access sources like PubMed Central (PMC).
Extracts dataset mentions from structured sections (e.g., Data Availability, Supplementary Material).
Supports two main strategies:
Retrieve-Then-Read (RTR): First retrieves relevant sections using hand-crafted rules, then applies LLMs.
Full-Document Read (FDR): Applies LLMs to the full text without section filtering.
Outputs structured results in JSON format.
Includes support for known repositories (e.g., GEO, PRIDE, MassIVE) via a configurable ontology.
Use Cases
Helping data curators and librarians identify datasets cited in publications.
Supporting meta-analysis and secondary data discovery.
Enabling dataset indexing and retrieval across the open-access literature.
Citation
If you use Data Gatherer in your research or project, please cite our paper:
@inproceedings{marini_data_2025,
title = {Data {Gatherer}: {LLM}-{Powered} {Dataset} {Reference} {Extraction} from {Scientific} {Literature}},
url = {https://aclanthology.org/2025.sdp-1.10},
doi = {10.18653/v1/2025.sdp-1.10},
booktitle = {Proceedings of the {Fifth} {Workshop} on {Scholarly} {Document} {Processing} ({SDP} 2025)},
publisher = {Association for Computational Linguistics},
author = {Marini, Pietro and Santos, Aécio and Contaxis, Nicole and Freire, Juliana},
year = {2025},
pages = {114--123},
}
Metadata
Release files for data-gatherer 0.2.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| data_gatherer-0.2.2.tar.gz | 175.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| data_gatherer-0.2.2-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 382.4 kB
Release files / data_gatherer-0.2.2.tar.gz
| Download URL | data_gatherer-0.2.2.tar.gz |
|---|---|
| Size | 175.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3020bccab9f167d4a10e8ed638592af8bd77813ec8a49c4eb164d129ba70ca17
|
|
BLAKE2b-256 checksum How to use checksums |
e61122b21c0735cc439338c2f133ee42ebcc02deba438feb7178e35a82526bf2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on May 26, 2026.
Transparency logRelease files / data_gatherer-0.2.2-py3-none-any.whl
| Download URL | data_gatherer-0.2.2-py3-none-any.whl |
|---|---|
| Size | 206.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b9e3563bd5cef644b1216bb179188d564e844dcdd2622e42336ddf2d918a16cd
|
|
BLAKE2b-256 checksum How to use checksums |
4d13e0b8b9bd11b7bba79d0e5a39d6b6d09284e0ea76e54f78f33836fe631ef8
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on May 26, 2026.
Transparency log