prep
prep is an agent-friendly CLI for converting LLM/ML datasets into common
standard training and evaluation schemas backed by Hugging Face datasets.
Installation
uv tool install prep-cli
# pip install prep-cli # or pip
prep --help
Alternatively, you can install from source:
# as tool
uv tool install 'prep-cli @ https://github.com/Moenupa/prep.git'
prep --help
# as folder
git clone https://github.com/Moenupa/prep.git
uv sync --dev
uv run prep --help
Usage
# export UI=1 # if you want interactive access
prep TARGET_FORMAT PIPELINE_ID [SPLIT] [SOURCE[@SUBSET][:SPLIT]] [OPTIONS]
# `auto`: generic conversion pipeline, requires SPLIT and SOURCE:
prep sft auto train path/to/data.jsonl --no-save
# a registered dataset-specific formatter pipeline:
prep verl geo3k test --save
# list registered pipelines and local output status:
ppls
Documentation
The full usage reference lives in skills/prep, maintained for both humans and coding agents:
1 maintainer
Metadata
1 maintainer
Release files for prep-cli 0.3.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| prep_cli-0.3.0.tar.gz | 21.1 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| prep_cli-0.3.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 52.5 kB
Release files / prep_cli-0.3.0.tar.gz
| Download URL | prep_cli-0.3.0.tar.gz |
|---|---|
| Size | 21.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f3cea71ef2267b76469818aacb988f04230ec0a8602264474fe88e7f96c72130
|
|
BLAKE2b-256 checksum How to use checksums |
9efbb8a6d1f57f4f7e3d986504b2019d3b15be09cfb37365b8bd2569cacca2db
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.8 {"installer":{"name":"uv","version":"0.12.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / prep_cli-0.3.0-py3-none-any.whl
| Download URL | prep_cli-0.3.0-py3-none-any.whl |
|---|---|
| Size | 31.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
db77a7004f7962136db295d8e2f1e9c7c52594bba6e9e941d28f09efa852b92f
|
|
BLAKE2b-256 checksum How to use checksums |
164386acc0ff79ea6716465232c05fbcfa8a91f445cc62bc14778287d0bff171
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
uv/0.12.8 {"installer":{"name":"uv","version":"0.12.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
