🌾 SQL Grain
SQLite databases as Grain data sources.
sql-grain lets you prototype ML data pipelines using SQL queries before committing to a production data format. Define your training examples with expressive SQL—joins, window functions, filtering—and iterate quickly without preprocessing. When you're ready to scale, convert to ArrayRecord or similar formats; sql-grain is not designed for large-scale training.
from sqlgrain import Sqlite3DataSource
import grain
source = Sqlite3DataSource(
"data.db",
key_query="SELECT id FROM users",
record_query="SELECT item FROM purchases WHERE user_id = :id ORDER BY timestamp",
)
dataset = grain.MapDataset.source(source).shuffle().batch(32)
Converting to ArrayRecord
Once you're ready to run larger experiments, convert the dataset to ArrayRecords using the to_array_record function which serializes records in msgpack format with native support for NumPy arrays.
from sqlgrain import to_array_record
to_array_record(source, "output/", shard_every=1000)
Grain's ArrayRecordDataSource reads the raw bytes, and we need to integrate a decoder into the data pipeline. SQL Grain's decode_record takes care of that, matching the behavior of to_array_record.
from sqlgrain import from_array_record, decode_record
ar_source, metadata = from_array_record("output/")
dataset = grain.MapDataset.source(ar_source).map(decode_record).batch(32)
to_array_record is agnostic to the type of the records, and you can also serialize datasets, e.g., for pre-batching.
Metadata
Release files for sql-grain 0.2.3
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| sql_grain-0.2.3.tar.gz | 9.0 kB | With provenance | Details for sql_grain-0.2.3.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| sql_grain-0.2.3-py3-none-any.whl | Python 3 | none | any | Details for sql_grain-0.2.3-py3-none-any.whl |
Total release size: 16.8 kB
Release files / sql_grain-0.2.3.tar.gz
| Download URL | sql_grain-0.2.3.tar.gz |
|---|---|
| Size | 9.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
a2b965b5a3d56106162ec8262397a34f44627221130282ee7baa8cf761babbf9
|
|
BLAKE2b-256 checksum How to use checksums |
da8857470eb68470c1778b27daa1c5d97d70122521b009fa1ad71d90c653628e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jan 9, 2026.
Transparency logRelease files / sql_grain-0.2.3-py3-none-any.whl
| Download URL | sql_grain-0.2.3-py3-none-any.whl |
|---|---|
| Size | 7.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
48130e28fdb6e1fe564656b8a4f9eec6e8d0c4ffa0ce6477efd1364b029a9c95
|
|
BLAKE2b-256 checksum How to use checksums |
42624eb46786da7b9ac0656ba937a57d390906398edb98874202811b695e508f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/6.1.0 CPython/3.13.7
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jan 9, 2026.
Transparency logSecurity for sql-grain 0.2.3
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for sql_grain-0.2.3.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Jan 9, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.