Deterministic and verifiable LLM inference
About
detLLM verifies reproducibility for LLM inference and produces a minimal repro pack when outputs diverge. It measures run-to-run variance and batch-size variance, and reports results with explicit, capability-gated guarantees (only claimed when the backend can actually enforce them).
Demo
Quickstart
pip install detllm
detllm check --backend hf --model <model_id> \
--prompt "Choose one: A or B. Answer with a single letter." \
--tier 1 --runs 5 --batch-size 1
Note: some shells (like zsh) require quotes when installing extras, e.g. pip install 'detllm[test,hf]'.
Verification
See docs/verification.md for the full local verification procedure and expected outputs.
Tiers
- Tier 0: artifacts + deterministic diff/report (no equality guarantees)
- Tier 1: repeatability across runs for a fixed batch size
- Tier 2: Tier 1 + score/logprob equality (capability-gated)
Tier 1 guarantees repeatability only for a fixed batch size; batch invariance is measured separately.
Tier 2 scores are captured when the backend supports stable score/logprob output. See docs/verification.md for how to verify scores appear in traces.
Artifacts (minimal repro pack)
Each run writes an artifacts/<run_id>/ folder:
env.jsonrun_config.jsondeterminism_applied.jsontrace.jsonlreport.json+report.txtdiffs/first_divergence.json
Python API
from detllm import check, run
run(
backend="hf",
model="distilgpt2",
prompts=["Hello"],
tier=1,
out_dir="artifacts/run1",
)
report = check(
backend="hf",
model="distilgpt2",
prompts=["Hello"],
runs=3,
batch_size=1,
out_dir="artifacts/check1",
)
print(report.status, report.category)
CLI
detllm envdetllm rundetllm checkdetllm diffdetllm report
Known limitations
- GPU determinism is conditional; results can change across drivers/kernels.
- Batch invariance is not guaranteed; it is measured separately.
- Strict guarantees depend on backend capabilities.
- Distributed/multiprocess inference is out of scope for now.
Docs
- docs/verification.md
- docs/determinism_boundary.md
- docs/troubleshooting.md
- docs/vllm.md
- docs/python_api.md
- docs/versioning.md
Versioning
See docs/versioning.md for compatibility guarantees.
Metadata
Release files for detllm 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| detllm-0.1.1.tar.gz | 23.9 kB | Without provenance | Details for detllm-0.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| detllm-0.1.1-py3-none-any.whl | Python 3 | none | any | Details for detllm-0.1.1-py3-none-any.whl |
Total release size: 51.9 kB
Release files / detllm-0.1.1.tar.gz
| Download URL | detllm-0.1.1.tar.gz |
|---|---|
| Size | 23.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
931dc26bf674aa2ada03c161e8cfd7a0badd83fef77879344ef98e62a0dace38
|
|
BLAKE2b-256 checksum How to use checksums |
026d46fbcdf7a9a8e0acd7c8588e4d3fed23489526aa2748897714244a9e58f7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.6
|
Release files / detllm-0.1.1-py3-none-any.whl
| Download URL | detllm-0.1.1-py3-none-any.whl |
|---|---|
| Size | 27.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9cf2602a3993c0eed7b28504a43338705e4efc11a03142c5f0b0558ea588e2b5
|
|
BLAKE2b-256 checksum How to use checksums |
7ca030faeb62058c969910d2849a0db933195354906e9fc2603d2508895ed781
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.6
|
Security for detllm 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.