🧠 ModelRed Python SDK
Official Python SDK for ModelRed.ai —
Run large-scale security assessments and risk analysis on any LLM or AI system.
🚀 Overview
ModelRed helps you red-team, benchmark, and secure your language models and AI systems.
This SDK provides a simple interface to:
- ✅ List your registered models
- ✅ Browse owned & imported probe packs
- ✅ Run security assessments programmatically
- ✅ Retrieve assessment details & scores
- ✅ Handle robust error types for production reliability
The SDK is typed, retry-aware, and designed for both sync and async workflows.
🧩 Installation
pip install modelred
Requires Python 3.8+
🔑 Authentication
Generate your API key (mr_...) from the ModelRed web app:
Keep it private — your key determines the organization context.
⚡ Quickstart
Synchronous example
from modelred import ModelRed
client = ModelRed(api_key="mr_...")
# 1. List your models
models = client.list_models()
model_id = models["data"][0]["id"]
# 2. List your owned probe packs
packs = client.list_owned_probes()["data"]
probe_pack_ids = [p["id"] for p in packs[:2]]
# 3. Create a new assessment
resp = client.create_assessment(
model_id=model_id,
probe_pack_ids=probe_pack_ids,
)
print(resp)
Asynchronous example
import asyncio
from modelred import AsyncModelRed
async def main():
async with AsyncModelRed(api_key="mr_...") as client:
models = await client.list_models()
model_id = models["data"][0]["id"]
packs = await client.list_owned_probes()
probe_pack_ids = [p["id"] for p in packs["data"][:2]]
resp = await client.create_assessment(
model_id=model_id,
probe_pack_ids=probe_pack_ids,
)
print(resp)
asyncio.run(main())
📘 Common Methods
| Category | Method | Description |
|---|---|---|
| Models | list_models() |
List your registered models |
| Probe Packs | list_owned_probes() / list_imported_probes() |
Browse your probe packs |
| Assessments | create_assessment() |
Run a test suite on a model |
list_assessments() |
View past runs | |
get_assessment(id) |
Fetch full details | |
cancel_assessment(id) |
Cancel (UI-only, raises NotAllowedForApiKey) |
🧱 Error Handling
Every API error is a specific exception:
from modelred.errors import Unauthorized, LimitExceeded, NotAllowedForApiKey
try:
resp = client.create_assessment(...)
except Unauthorized:
print("Invalid or expired API key")
except LimitExceeded as e:
print("Plan limit hit:", e.message)
except NotAllowedForApiKey:
print("This action must be done from the web UI")
| Exception | Meaning |
|---|---|
Unauthorized |
401 — bad or missing key |
Forbidden, NotAllowedForApiKey |
403 — disallowed action |
LimitExceeded |
403 — plan or usage limit |
NotFound |
404 — resource not found |
Conflict |
409 — concurrent/duplicate |
ValidationFailed |
400/422 — bad request |
RateLimited |
429 — retry with backoff |
ServerError |
5xx — internal server issue |
🧪 Testing (optional)
You can test locally using the included mock mode:
MODELRED_TEST_MODE=mock python test_runner.py
This uses httpx.MockTransport to simulate API responses — no network required.
To test live, set your real environment variables:
export MODELRED_API_KEY="mr_..."
python test_runner.py
🔗 Links
- 🌐 Website
- 💡 App Dashboard
- 📄 Docs (MDX)
- 🧰 GitHub
📜 License
MIT License © 2025 ModelRed.ai
Developed with ❤️ by the ModelRed Engineering Team
1 maintainer
Metadata
1 maintainer
Release files for modelred 0.1.40
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| modelred-0.1.40.tar.gz | 187.7 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| modelred-0.1.40-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 199.3 kB
Release files / modelred-0.1.40.tar.gz
| Download URL | modelred-0.1.40.tar.gz |
|---|---|
| Size | 187.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
f86ae028dd8764f7e45dbe877be66687aa1d8e86233c52736ae204b2feab8cc8
|
|
BLAKE2b-256 checksum How to use checksums |
d021fd52e435c6166f5f6616e03d75d68cdb97e469ed46a7a4626ec2e97033fe
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.6.17
|
Release files / modelred-0.1.40-py3-none-any.whl
| Download URL | modelred-0.1.40-py3-none-any.whl |
|---|---|
| Size | 11.6 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5edad48f9811611595d519adc197a11701fe2252161648f0a0d77a43a44e6b77
|
|
BLAKE2b-256 checksum How to use checksums |
7ded7b6833235227cd89049fa7f07d0219f90cf357921056a29e5e71938895ef
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.6.17
|
Security for modelred 0.1.40
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
