oaipmh-scythe: A Scythe for harvesting OAI-PMH repositories

Welcome to oaipmh-scythe, an updated and modernized version of the original sickle, now with additional features and ongoing maintenance.

CI
Package
Meta

oaipmh-scythe is a lightweight OAI-PMH client library written in Python. It has been designed for conveniently retrieving data from OAI interfaces the Pythonic way:

from oaipmh_scythe import Scythe

with Scythe("https://zenodo.org/oai2d") as scythe:
    records = scythe.list_records()
    next(records)
# <Record oai:zenodo.org:4574771>

Features

• Easy harvesting of OAI-compliant interfaces
• Support for all six OAI verbs
• Convenient object representations of OAI items (records, headers, sets, ...)
• Automatic de-serialization of Dublin Core-encoded metadata payloads to Python dictionaries
• Option for ignoring deleted items

Requirements

Python >= 3.10

oaipmh-scythe is built with:

• httpx2 for issuing HTTP requests
• lxml for parsing XML responses

Installation

You can install oaipmh-scythe from PyPI using uv:

uv add oaipmh-scythe

or with pip:

python -m pip install oaipmh-scythe

Documentation

The documentation is made with Zensical and is hosted by GitHub Pages.

Similar Projects

There are a couple of similar projects available on PyPI and GitHub, e.g. via the topics oai-pmh and oai-pmh-client. Among them are these implementations in Python:

Project	Description	Last commit
sickle	oaipmh-scythe is a fork of sickle
pyoai	sickle was inspired by pyoai
oaipmh	oaipmh is a fork of pyoai
pyoaiharvester	oai-pmh harvester CLI
ddblabs-ometha	oai-pmh harvester with CLI and TUI
oai-harvest	uses pyoai internally
oai-pmh-harvester	uses sickle internally

There are also similar projects available in Java, PHP and Rust.

Acknowledgments

This is a fork of sickle which was originally written by Mathias Loesch.

License

oaipmh-scythe is distributed under the terms of the BSD license.