oasnap-reader
Read a full OpenAlex snapshot and convert it to a reduced dataset for analysis.
The reduced dataset filters out paratext records and extracts bibliographic fields, topic classifications, authorship, and citation data into a compact format suitable for large-scale analysis.
Full conversion time depends on available hardware. The full conversion with >2000 files took ~2 hours with 24 workers and sufficient RAM.
Install
pip install oasnap-reader
Usage
from pathlib import Path
from oasnap_reader.reader import ReadGZ
reader = ReadGZ(
in_path=Path("/data/openalex/works"),
out_path=Path("/data/reduced"),
)
reader.read_all()
in_path should point to the root of the OpenAlex works snapshot directory.
Output is one gzip-compressed JSONL file per input file, written to out_path.
Documentation
See the full documentation including usage options and API reference at the project docs site.
Development
git clone https://gitlab.gwdg.de/mpigea/dt/oasnap-reader
cd oasnap-reader
uv sync --dev
1 maintainer
Metadata
1 maintainer
Release files for oasnap-reader 0.3.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| oasnap_reader-0.3.5.tar.gz | 9.7 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| oasnap_reader-0.3.5-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 21.1 kB
Release files / oasnap_reader-0.3.5.tar.gz
| Download URL | oasnap_reader-0.3.5.tar.gz |
|---|---|
| Size | 9.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
29413ccf7fdf64caba16daa54ae449a0eaf733acb0c2fecea65741a2ccc9e89b
|
|
BLAKE2b-256 checksum How to use checksums |
d3db06fd5939df2545b62887cf6cb4c99606f8f9ef5eb9184c072df77958cbfa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.17 {"installer":{"name":"uv","version":"0.9.17","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / oasnap_reader-0.3.5-py3-none-any.whl
| Download URL | oasnap_reader-0.3.5-py3-none-any.whl |
|---|---|
| Size | 11.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
7a0b4c5d48d43b7485d7fe76c78e75d4f9d5a03db2010f506fb297e7198b5fe0
|
|
BLAKE2b-256 checksum How to use checksums |
db06c4f1c0bace6e4289fe73d2d9ef2d8466ee2ec783ed7097a3c3e31d9a9672
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.17 {"installer":{"name":"uv","version":"0.9.17","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
