miniverl 0.7.1

PyPI · Stable docs · Development docs · 中文

miniVERL is a local, inspectable single-GPU alignment and distillation runtime. It runs native SFT, DPO, KD and strict OPD recipes, preserves assistant-only loss masks and policy-version provenance, and exchanges standard HF/PEFT/Parquet artifacts through a fail-closed bridge to one pinned verl profile.

PyPI v0.7.1 is stable; main is development. miniVERL is independent from verl. It does not claim arbitrary verl YAML execution, distributed execution, or full algorithmic compatibility.

Install and verify in about a minute

python -m pip install "miniverl[train]"
miniverl doctor
miniverl demo --fast --output runs/quickstart
miniverl inspect runs/quickstart/trajectories.jsonl
miniverl evidence validate alignment-external-v1

The deterministic demo downloads no model and produces typed trajectories, a checksummed teacher cache, manifest and report. The evidence command reads self-contained package data; it works from a wheel without a Git checkout. For schemas and inspection without the ML stack, install miniverl alone.

Supported hardware and runtime boundary

miniVERL runs one local process on CPU or one NVIDIA CUDA GPU. The CUDA path is device-name agnostic, but fit depends on model pair, context, kernels and VRAM. Install the matching CUDA-enabled PyTorch build first, then miniverl[train,cuda]; that extra does not select a CUDA PyTorch wheel. Ray, FSDP, Megatron, PPO, GRPO and distributed launch are outside the runtime. See the single-GPU guide.

verl compatibility summary

The bridge targets official verl v0.8.0 at commit 7aed6b23. Its verified boundary is checksummed standard artifacts plus pinned config-parse and model/data-load smoke—not native checkpoint parity or a completed verl job. Imports fail closed when dataset, environment, teacher, objective or schedule semantics are unresolved; they never substitute calculator tasks or invent an unqualified teacher.

Current exports remain launchable: false: the base snapshot is absent, the reward scaffold fails closed and required mappings remain placeholders. The entry point is launch.template.sh; readiness, parse/load evidence, launchability, distributed execution and semantic parity are separate facts. Read the bridge contract.

One measured systems result

On one RTX 4080 with Qwen3-0.6B and eight fixed SQLite trajectories, physical batch 4 increased dual-model update throughput from 2.369 to 3.866 trajectories/s. Shared-backbone batch 4 used 2.227 GiB peak reserved memory versus 3.035 GiB for dual model while running 10.1% slower. All 12 preregistered equivalence comparisons passed. This is one workload on one machine, not a promise for other GPUs.

Consumer Runtime v1 methods and caveats

Three paths

Research notes and preserved negative evidence

v0.7 External Alignment Gate

The preregistered external study stopped before teacher or method training. Both declared starting-policy lineages scored 0/64 retained JSONNav utility for every candidate against the unchanged 20% floor.

miniverl pilot --builtin-study alignment-external-v1 --json

The result is do_not_continue_this_study and insufficient_evidence, not a recommendation among SFT/DPO/KD/OPD. Granite Guardian values are unqualified selection diagnostics; Granite, PairRM and teacher qualification and the reserved final test did not run. Study and limitations.

Earlier measured alignment case study

Alignment Lab v1 began from an SFT checkpoint already at 100% policy compliance and 100% retained tool utility in all three seeds. No continuation improved the ceiling; continued SFT and both OPD variants retained measured regressions. The two sandbox safety checks tied at zero while utility still regressed. IFEval, XSTest, HarmBench and RewardBench were not executed, and “preference win rate” is a deterministic Minipolicy paired outcome, not human preference. Seed-level evidence.

• RecoveryBench v1: frozen-student KD beat slower fresh-state OPD on the preregistered primary view; the verifier gate remained insufficient_evidence.
• Calculator benchmark: both negative controls completed normally at 0%; the ambiguous historical protocol-v1 prompt prevents attributing failure solely to intrinsic teacher behavior.
• Limitations, math, reproducibility and compatibility policy.

New runs establish tokenizer compatibility through structural identity. The legacy behavioral fingerprint is only a migration fallback, not identity proof.

Develop

git clone https://github.com/DaoyuanLi2816/mini-verl.git
cd mini-verl
python -m pip install -e ".[dev]"
pytest -q -m "not gpu and not network"

Apache-2.0 licensed. See CONTRIBUTING.md, SECURITY.md, the changelog and citation. Project records: default GPU recipe, frozen calculator result and license.