Zippy (ZDS) Python Package
High-performance, HuggingFace-compatible dataset storage format.
Installation
pip install zippy-zds
# With optional dependencies
pip install zippy-zds[pandas]
pip install zippy-zds[all]
Quick Start
from zippy import ZDSStore, ZDataset, ZIterableDataset
# Create a store
store = ZDSStore.open("./my_dataset", collection="train")
# Add documents
store.put("doc1", {"text": "Hello world", "label": 1})
store.put("doc2", {"text": "Goodbye world", "label": 0})
# Map-style dataset (random access)
dataset = store.to_dataset()
print(dataset[0]) # {"text": "Hello world", "label": 1}
print(len(dataset)) # 2
# Iterable dataset (streaming)
iterable = store.to_iterable_dataset()
for doc in iterable:
print(doc)
# With shuffle buffer
for doc in iterable.shuffle(buffer_size=1000):
print(doc)
Multi-Collection Management with ZDSStore
from zippy import ZDSStore
# Omit the collection argument to get a root-capable store handle
store = ZDSStore.open("./ml_data", native=True)
# Open as many collections as you need from the same handle
train = store.collection("train")
test = store.collection("test")
validation = store.collection("validation")
train.put("doc1", {"split": "train"})
test.put("doc1", {"split": "test"})
validation.put("doc1", {"split": "validation"})
print(store.list_collections())
# Context manager closes the shared root when you're done
with ZDSStore.open("./ml_data", "train") as store:
store.put("doc2", {"split": "train"})
> ℹ️ `ZDSRoot` still exists under the hood (and is exposed via `store.root`)
> for advanced scenarios that need explicit read/write modes or manual
> locking, but most workflows can just rely on the unified
> `ZDSStore.open()` entry point shown above.
>
> ⚠️ Closing the root invalidates every reader/writer for that path. Only call `store.root.close()` during shutdown/cleanup.
print(store.list_collections()) # ['test', 'train', 'validation']
DataFrame Integration
from zippy import read_zds, to_zds
# Load as DataFrame (requires pandas)
df = read_zds("./my_dataset", collection="train")
# Export DataFrame to ZDS
to_zds(df, "./output", collection="exported")
HuggingFace Compatibility
ZDS datasets are designed to work seamlessly with HuggingFace training loops:
from zippy import ZIterableDataset
dataset = ZIterableDataset.from_store("./my_dataset", collection="train")
# Works with DataLoader
from torch.utils.data import DataLoader
loader = DataLoader(dataset, batch_size=32)
1 maintainer
Metadata
1 maintainer
Release files for zippy-data 0.1.2
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| zippy_data-0.1.2.tar.gz | 80.0 kB | Details |
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| zippy_data-0.1.2-cp313-cp313-win_amd64.whl | CPython 3.13 | CPython 3.13 | Windows x86-64 | Details |
| zippy_data-0.1.2-cp313-cp313-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl | CPython 3.13 | CPython 3.13 | macOS 11.0+ ARM64, macOS 10.12+ universal2 (ARM64, x86-64), macOS 10.12+ x86-64 | Details |
| zippy_data-0.1.2-cp312-cp312-win_amd64.whl | CPython 3.12 | CPython 3.12 | Windows x86-64 | Details |
| zippy_data-0.1.2-cp312-cp312-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl | CPython 3.12 | CPython 3.12 | macOS 11.0+ ARM64, macOS 10.12+ universal2 (ARM64, x86-64), macOS 10.12+ x86-64 | Details |
| zippy_data-0.1.2-cp311-cp311-win_amd64.whl | CPython 3.11 | CPython 3.11 | Windows x86-64 | Details |
| zippy_data-0.1.2-cp311-cp311-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl | CPython 3.11 | CPython 3.11 | macOS 10.12+ x86-64, macOS 11.0+ ARM64, macOS 10.12+ universal2 (ARM64, x86-64) | Details |
| zippy_data-0.1.2-cp310-cp310-win_amd64.whl | CPython 3.10 | CPython 3.10 | Windows x86-64 | Details |
| zippy_data-0.1.2-cp310-cp310-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl | CPython 3.10 | CPython 3.10 | macOS 10.12+ universal2 (ARM64, x86-64), macOS 10.12+ x86-64, macOS 11.0+ ARM64 | Details |
| zippy_data-0.1.2-cp39-cp39-win_amd64.whl | CPython 3.9 | CPython 3.9 | Windows x86-64 | Details |
| zippy_data-0.1.2-cp39-cp39-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl | CPython 3.9 | CPython 3.9 | macOS 11.0+ ARM64, macOS 10.12+ universal2 (ARM64, x86-64), macOS 10.12+ x86-64 | Details |
| zippy_data-0.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.8 | CPython 3.8 | Linux glibc 2.17+ x86-64 | Details |
Total release size: 9.5 MB
Release files / zippy_data-0.1.2.tar.gz
| Download URL | zippy_data-0.1.2.tar.gz |
|---|---|
| Size | 80.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
cced48e87cd5dcf505b3c9554ae7b7a7ca68c10e116f7d07b628036d4672d82c
|
|
BLAKE2b-256 checksum How to use checksums |
e4e55a00c3a392bc4b3543eb80e635cd6e4b7601082c14c0faa3c38c2e3bfc84
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp313-cp313-win_amd64.whl
| Download URL | zippy_data-0.1.2-cp313-cp313-win_amd64.whl |
|---|---|
| Size | 522.6 kB |
| Tags | CPython 3.13 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
f9708973ab81f22c615dcf09719ae006a5e196edab4d59db83bef26e9db56711
|
|
BLAKE2b-256 checksum How to use checksums |
08e52fa19914f184000ad6aaf886d8e8ca9993ca7c13d3c478e90894910ede92
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp313-cp313-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl
| Download URL | zippy_data-0.1.2-cp313-cp313-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl |
|---|---|
| Size | 1.2 MB |
| Tags | CPython 3.13 macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
ee49d53e63f5b5d43becdd391e41f77e0b686a35d3628b9050202c9d2094962b
|
|
BLAKE2b-256 checksum How to use checksums |
ee3070c6cbabec1faf58ace53c399488b27b3f064db3e68087fe1199dbb2eb28
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp312-cp312-win_amd64.whl
| Download URL | zippy_data-0.1.2-cp312-cp312-win_amd64.whl |
|---|---|
| Size | 523.6 kB |
| Tags | CPython 3.12 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
af0dc85564fd40c4b0919582f615463633b2d212465d8763c80bc7e9a088e89b
|
|
BLAKE2b-256 checksum How to use checksums |
989a9efdb5803e196fa5dd742f732e06e52b427a1b250475bf5eaf5b5cf3030f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp312-cp312-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl
| Download URL | zippy_data-0.1.2-cp312-cp312-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl |
|---|---|
| Size | 1.2 MB |
| Tags | CPython 3.12 macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
4f2ea6a1eb19e802142100e9c41fb7cef21d5e01fd8a698407c328a5ac9e3202
|
|
BLAKE2b-256 checksum How to use checksums |
9eecaa598233fb0037a9408aa6f4e2a71e550508356c053f5457e5d61c3b7704
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp311-cp311-win_amd64.whl
| Download URL | zippy_data-0.1.2-cp311-cp311-win_amd64.whl |
|---|---|
| Size | 523.2 kB |
| Tags | CPython 3.11 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
01eca4a17fe565a3d8d22471a3edc60ca1288a4ad043fa648f2ee681397a3c8b
|
|
BLAKE2b-256 checksum How to use checksums |
587309416f2cd859ed5d212c71787cb7b6c108131c95e2e0600d046a9017edb2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp311-cp311-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl
| Download URL | zippy_data-0.1.2-cp311-cp311-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl |
|---|---|
| Size | 1.2 MB |
| Tags | CPython 3.11 macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
1d8a6567069d9492ff438825802fded7778e2b636f4c6e2fe9d8d8ef88eeb3d4
|
|
BLAKE2b-256 checksum How to use checksums |
cb173ab61faf596f3a9c49cfe383185b47d57836694ab97429caeecf533fd9d4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp310-cp310-win_amd64.whl
| Download URL | zippy_data-0.1.2-cp310-cp310-win_amd64.whl |
|---|---|
| Size | 522.5 kB |
| Tags | CPython 3.10 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
875b0732c62ee7880e30dd250686735349f1fe23895ee431cbdf91772ed423c2
|
|
BLAKE2b-256 checksum How to use checksums |
24dd106e9f06fa9c0adbb5849c9e00cfd75d9a5722bb9124ace0d226142c1834
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp310-cp310-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl
| Download URL | zippy_data-0.1.2-cp310-cp310-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl |
|---|---|
| Size | 1.2 MB |
| Tags | CPython 3.10 macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
674cd636dddef55932ae0d6f44170dfe6880bb2d6330d82dcc181cf373fdd03d
|
|
BLAKE2b-256 checksum How to use checksums |
6431158ff3b9e4b5f5af5066a31e6e8c5ef0e3eedb3a26438619c8d17214eb0e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp39-cp39-win_amd64.whl
| Download URL | zippy_data-0.1.2-cp39-cp39-win_amd64.whl |
|---|---|
| Size | 523.2 kB |
| Tags | CPython 3.9 Windows x86-64 |
|
SHA-256 checksum How to use checksums |
06c066745c8f052da71242a553cfea4e4177ac1a32f2a42bd2f07986cb75fc2b
|
|
BLAKE2b-256 checksum How to use checksums |
81c55c72e1893579b1acc0093b3057fe33d29e87411f42b238256a01cc106a83
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp39-cp39-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl
| Download URL | zippy_data-0.1.2-cp39-cp39-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl |
|---|---|
| Size | 1.2 MB |
| Tags | CPython 3.9 macOS 10.12+ universal2 (ARM64, x86-64) macOS 10.12+ x86-64 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
90f9f6dab88a2749d2732febee0cf4a3fc46641ccb330224b5e1e3ff72086b50
|
|
BLAKE2b-256 checksum How to use checksums |
b2862936a5154ec182627566eab8148ffbf98fe76d682fa49e442858a6ba7929
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency logRelease files / zippy_data-0.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | zippy_data-0.1.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 698.2 kB |
| Tags | CPython 3.8 Linux glibc 2.17+ x86-64 |
|
SHA-256 checksum How to use checksums |
f7f86368d3317b2c06a9a137cf4763e6f6b463b706c846bbfc3e79bb4aca1efb
|
|
BLAKE2b-256 checksum How to use checksums |
200c1c9be670ee97dc624ceda46ee754608b402767ec7cae0da9dbacaff746df
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Dec 15, 2025.
Transparency log
