chaincheck-llm
Validate LLM responses with composable, production-ready checks.
pip install chaincheck-llm
The Problem
LLMs return unpredictable output. Sometimes they leak PII. Sometimes they refuse to answer. Sometimes they hallucinate. You need a way to validate responses before they reach your users.
chaincheck-llm gives you a composable validation pipeline — run any combination of checks on any LLM response.
Quick Start
from chaincheck import validate
from chaincheck.checks import (
check_not_empty,
check_no_pii,
check_no_toxicity,
check_min_length,
)
result = validate(
response=llm_output,
checks=[
check_not_empty,
check_no_pii,
check_no_toxicity,
lambda r: check_min_length(r, min_chars=50),
]
)
print(result.passed) # True / False
print(result.score) # 0.0 - 1.0
print(result.summary())
Available Checks
Format Checks (free, no LLM needed)
| Check | Description |
|---|---|
check_not_empty |
Response is not empty |
check_min_length(r, min_chars) |
Response meets minimum length |
check_max_length(r, max_chars) |
Response within maximum length |
check_json_valid |
Response is valid JSON |
check_contains(r, keywords) |
Response contains required keywords |
check_not_contains(r, forbidden) |
Response has no forbidden words |
check_regex(r, pattern) |
Response matches regex pattern |
check_no_placeholder |
No unfilled {variable} placeholders |
Content Checks (free, rule-based)
| Check | Description |
|---|---|
check_no_pii |
No email, phone, SSN, credit card |
check_no_toxicity |
No toxic/harmful language |
check_no_hallucination_markers |
No uncertainty disclaimers |
check_no_refusal |
LLM did not refuse to answer |
check_language(r, expected) |
Response is in expected language |
LLM-as-Judge Checks (uses tokens)
| Check | Description |
|---|---|
check_relevance(r, question, ...) |
Response is relevant to question |
check_factual_consistency(r, context, ...) |
Response consistent with context |
check_tone(r, expected_tone, ...) |
Response matches expected tone |
Composing Checks
from chaincheck import validate
from chaincheck.checks import (
check_not_empty, check_no_pii, check_no_toxicity,
check_relevance, check_min_length,
)
result = validate(
response=llm_output,
checks=[
# Fast rule-based checks first
check_not_empty,
check_no_pii,
check_no_toxicity,
lambda r: check_min_length(r, min_chars=100),
# LLM judge last (costs tokens)
lambda r: check_relevance(
r,
question="What is machine learning?",
provider="openai",
model="gpt-4o-mini",
api_key="sk-...",
),
],
fail_fast=True, # stop after first failure
)
if not result.passed:
for check in result.failed_checks:
print(f"FAILED: {check.name} — {check.message}")
ValidationResult
result.passed # True if all checks passed
result.score # Average score (0.0 - 1.0)
result.checks # List of CheckResult
result.passed_checks # Checks that passed
result.failed_checks # Checks that failed
result.summary() # Dict with full breakdown
License
MIT
Metadata
Release files for chaincheck-llm 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| chaincheck_llm-0.1.0.tar.gz | 12.0 kB | Without provenance | Details for chaincheck_llm-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| chaincheck_llm-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for chaincheck_llm-0.1.0-py3-none-any.whl |
Total release size: 22.4 kB
Release files / chaincheck_llm-0.1.0.tar.gz
| Download URL | chaincheck_llm-0.1.0.tar.gz |
|---|---|
| Size | 12.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f5076ac1e3ff6703a9654849fb944827a2acf7e5e5a4daa79d282812f34d9d79
|
|
BLAKE2b-256 checksum How to use checksums |
cb1ed3592153b815c60e6e2fa612b157d39a3691a8cbcc2bac614722b11ce68e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.14.0
|
Release files / chaincheck_llm-0.1.0-py3-none-any.whl
| Download URL | chaincheck_llm-0.1.0-py3-none-any.whl |
|---|---|
| Size | 10.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
913b14b8799c3bdb5567b12bc1d9634bafd04560e4d7d287933d0dc0d0af1c98
|
|
BLAKE2b-256 checksum How to use checksums |
82a85110127ad5aad39cdada7ffc16154239effa77805674baf9af9b46526c1f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.14.0
|
Security for chaincheck-llm 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.