Skip to main content

LLM Output Validation, Linting, and Assertion Layer

Project description

llm-contracts

PyPI version Python License Tests Build Status

🛡️ "ESLint + Pytest" for AI responses — Catch LLM mistakes before they reach production.
Schema validation, content linting, and professional reports for any LLM.


📦 Install

pip install llm-contracts

⚡ Why llm-contracts?

LLMs are fluent, confident, and totally wrong just often enough to break your app.

  • Air Canada's chatbot promised non-existent bereavement fares → Legal action
  • CNET's AI published financial advice with wrong interest rates → Public corrections
  • ChatGPT lawyer submitted fake legal citations in court → Professional sanctions

llm-contracts validates every AI response before it causes problems.


🚀 Quick Start

CLI

# Validate AI output against schema
llm-validate output.json --schema schema.yaml

# Generate professional reports
llm-validate output.json --schema schema.yaml --html-report report.html

Python API

from llm_contracts import contracts

# Validate output against schema
result = contracts.validate(data, "schema.yaml")

if not result.is_valid:
    print("AI failed validation:")
    for error in result.errors:
        print(f"  - {error}")

# Generate reports
contracts.generate_report(result, "report.html", "schema.yaml", format="html")

✅ Key Features

  • Schema Validation — Ensure proper JSON/YAML structure and data types
  • Content Linting — Check keywords, tone, word count, patterns, quality rules
  • Professional Reports — Beautiful HTML and Markdown validation reports
  • Framework Agnostic — Works with OpenAI, Anthropic, local models, any LLM
  • CLI + Python API — Use in scripts or integrate into applications
  • Zero Vendor Lock-in — Pure validation logic, no external API calls required

📋 Example Schema

schema:
  title:
    type: str
    min_length: 10
  description:
    type: str
    min_length: 100

rules:
  - keyword_must_include: ["quality", "premium"]
  - keyword_must_not_include: ["cheap", "defective"]
  - no_placeholder_text: "\\[YOUR_TEXT_HERE\\]"
  - word_count_min: 100
  - word_count_max: 500

Result: Every AI response gets validated before reaching your users. No more silent failures.


📚 Documentation & Links


🤝 Contributors

Created by Mohamed Jama - Built for developers shipping AI features in production.

Major Contributors:

  • Abdirahman Attila - Frontend web interface, comprehensive documentation website, enhanced testing suite, and Jekyll/GitHub Pages setup

We welcome contributions! See CONTRIBUTING.md for guidelines.


🏷 License

MIT © Mohamed Jama - see LICENSE file for details.


Stop hoping your AI gets it right. Start knowing it does.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

llm_contracts-1.0.0.tar.gz (36.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

llm_contracts-1.0.0-py3-none-any.whl (20.4 kB view details)

Uploaded Python 3

File details

Details for the file llm_contracts-1.0.0.tar.gz.

File metadata

  • Download URL: llm_contracts-1.0.0.tar.gz
  • Upload date:
  • Size: 36.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.18

File hashes

Hashes for llm_contracts-1.0.0.tar.gz
Algorithm Hash digest
SHA256 867e7b6c559619554e20400deadf09bb8ff046b23b876f44f4a2a0403eca5cc5
MD5 ca6797ec72e6e9f35aff462b59894d54
BLAKE2b-256 71b781c694be887392b0cba0f7b1c0534a6a15cc9da380cadd4462703a9a1d81

See more details on using hashes here.

File details

Details for the file llm_contracts-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: llm_contracts-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 20.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.18

File hashes

Hashes for llm_contracts-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 1041f6cb3d8f71154eefabc711559bb3940958662f67bda2e32fee9d484415ea
MD5 b98e60a3288fb372e5dc87012bce79cf
BLAKE2b-256 5f43e43013078e936f4b94aa1f31716a08695617347471c7af255166baf98a2f

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page