This release is a pre-release and may not be stable for production use.
GRIST 🌾
Generative Retrieval ID Semantic Transforms for reproducible data pipelines.
GRIST is a focused Python library for bridging raw research datasets and generative retrieval models. It enriches datasets with Semantic Identifiers, guarantees deterministic preprocessing, and provides helpers for publishing results to public hubs like HuggingFace and Kaggle. It is designed to work smoothly with existing data pipeline tooling, including Grain.
Why GRIST
In Generative Retrieval (GR) research, reproducibility is everything. GRIST treats a dataset not as a static file, but as a deterministic factory. Every transformation, from text cleaning to model-based ID generation, is designed to be perfectly reproducible.
Features
- Pipeline-native: Fits into existing data pipeline tooling without new paradigms to learn.
- Semantic ID injection: Built-in MapTransform classes for UUIDs, hashes, or model-generated codes.
- Inference-ready: Wrap any pre-trained model (HuggingFace, JAX, PyTorch) as an ID generator.
- Publishing helpers: Tools to facilitate uploads to HuggingFace or Kaggle.
Installation
uv add grist
Quick Start
TODO: Quick start example for the planned public API.
Concepts
- Semantic Identifiers: Stable, model-aware IDs that augment dataset samples for generative retrieval.
- Deterministic pipelines: Transform semantics guarantee repeatable preprocessing.
- Dataset configs: Optional, reusable configuration files for well-known datasets.
Why the Name
In milling, grist is the grain separated from its chaff and ready to be ground. This library prepares your "raw grain" (datasets) into a refined format ready for the "mill" of generative retrieval models.
1 maintainer
Metadata
1 maintainer
Release files for grist 0.1.0.dev0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| grist-0.1.0.dev0-py3-none-any.whl | Python 3 | none | any | Details for grist-0.1.0.dev0-py3-none-any.whl |
Release files / grist-0.1.0.dev0-py3-none-any.whl
| Download URL | grist-0.1.0.dev0-py3-none-any.whl |
|---|---|
| Size | 2.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
f5cd832d7c0908b0b2885d3ad41b241af137310c955a4c8abcfeb6ba16a32b44
|
|
BLAKE2b-256 checksum How to use checksums |
0e90f7355867df0045e7b5e491f96bc8709c38db7e7594418508259d108b6f17
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.9.26 {"installer":{"name":"uv","version":"0.9.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Security for grist 0.1.0.dev0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
