commonmeta-rs 0.9.51

PyO3 bindings for commonmeta-rs's list/batch functions (VRAIX dump fetch+cache, Parquet I/O, batched archives)

commonmeta-rs (Python bindings)

PyO3 bindings exposing commonmeta-rs's list-command batch functions to Python: VRAIX daily-dump fetch with local caching, lossless Parquet read/write, and batched archive rendering. Records cross the FFI boundary as plain JSON-shaped dicts — the same shape commonmeta-py's own readers and writers already produce and consume.

Install

uv pip install commonmeta-rs

Requires CPython 3.14 or newer. The wheels are built against the stable ABI (abi3-py314), so one wheel per platform also covers later 3.x releases. commonmeta-py itself keeps a 3.9 floor; only this backend needs 3.14.

Development

The interpreter must be 3.14+, matching requires-python; an older venv fails to build rather than falling back.

uv venv --python 3.14 .venv
uv pip install --python .venv/bin/python maturin pytest
source .venv/bin/activate
maturin develop
pytest tests/

Release

Releases are published by hand. PyPI trusted publishing only issues OIDC identities to GitHub, GitLab, Google Cloud and ActiveState, and this repo lives on Codeberg, so CI cannot authenticate to PyPI — the workflow builds and tests wheels only.

The version comes from [workspace.package] in the root Cargo.toml and is shared with the commonmeta crate, so a wheel names the release it wraps. Bump it there, not here.

Build one wheel per platform plus the sdist, then publish. The macOS wheel builds natively; the Linux manylinux wheel builds in the official maturin container (which bundles the manylinux toolchain), so it needs no cross linker on the host — just a running Docker. All commands are run from python/.

rm -rf dist

# macOS (native — arm64 on Apple silicon)
maturin build --release --out dist

# Linux (manylinux2014 x86_64) via Docker. Mount the repo ROOT, not python/:
# the binding depends on the parent crate through `commonmeta = { path = ".." }`.
# --platform linux/amd64 runs the x86_64 image under emulation on Apple silicon
# (slow but reliable). Pinning --target keeps the Linux artifacts out of the
# host's target/release, so they don't collide with the native build.
docker run --rm --platform linux/amd64 \
  -v "$(git rev-parse --show-toplevel)":/io -w /io/python \
  ghcr.io/pyo3/maturin:latest \
  build --release --out dist --target x86_64-unknown-linux-gnu

# Source distribution (lets other platforms build from source as a fallback)
maturin sdist --out dist

uv publish --token "$PYPI_TOKEN" dist/*

Publishing from one machine uploads only the wheels present in dist/; any platform without a wheel falls back to building from the sdist. To ship binaries for more platforms (Windows, Intel macOS, Linux aarch64), add the matching maturin build --target … invocations or collect the wheels-* artifacts from the CI workflow into dist/ first.

API

• fetch_vraix(source, date, input_path=None, limit=None, offset=0, cache_ttl_days=30) -> list[dict] Fetch records from a VRAIX daily dump (source is "crossref" or "datacite"). With input_path, reads a local SQLite file directly (no network); otherwise downloads {source}-{date}.sqlite3.zst from metadata.vraix.org, caching it locally for cache_ttl_days.
• write_parquet(records: list[dict]) -> bytes Write records as a single lossless Parquet file.
• read_parquet(data: bytes) -> list[dict] Read records back from Parquet bytes written by write_parquet.
• write_archive(records: list[dict], to: str, base_name: str, batch_size=100_000) -> list[tuple[str, bytes]] Render records to to format, split into batches of at most batch_size records, returned as (entry_name, bytes) pairs ready to pack into a zip/tar archive.
• convert(from_: str, to: str, input: str) -> bytes Convert a single record between formats.