title: SimpleAudit Studio emoji: 🔍 colorFrom: blue colorTo: indigo sdk: docker app_port: 7860 pinned: false

SimpleAudit Studio

A self-hostable platform for running reproducible AI model audits using the SimpleAudit engine (Target → Auditor → Judge). Every audit captures frozen, versioned inputs so historical results stay interpretable years later.

🚀 Quick Start

uvx simpleaudit-studio

uvx installs the simpleaudit-studio package and runs it in an isolated environment. No Docker, no Postgres, no manual setup. Opens at http://localhost:8000 (login: studio / admin123). A mock model server is pre-seeded so you can start exploring audit results immediately.

🌐 Or skip the setup entirely — try the live demo: sushantgautam-simpleaudit-studio.hf.space

Want the latest unreleased code from this repo instead of the PyPI release? Same one-liner, pointed at git:

uvx --from "git+https://github.com/SushantGautam/SimpleAuditStudio" spin

🤖 Using Real Models

The pre-seeded connections point at a built-in mock server. To run real audits, update your model connections on the Models page to any OpenAI-compatible endpoint. A few common options:

Set the API key in the connection form (leave blank for local servers that don't require auth), then start a run from New Experiment.

🛠️ Local Development

For contributors working on the codebase, use uv — no manual venv steps, no Makefile. Two verbs cover everything:

• uv sync — create/update .venv from pyproject.toml + uv.lock
• uv run <cmd> — run any command inside .venv (auto-syncs first if the lockfile changed)

Setup (one time)

git clone https://github.com/SushantGautam/SimpleAuditStudio
cd SimpleAuditStudio
cp .env.local.example .env          # minimal local config (SQLite + mock models)

uv sync --extra dev --extra postgres   # create .venv + install all deps
uv run manage.py setup_local           # migrate + create admin (studio / BOOTSTRAP_PASSWORD from .env) + seed data

setup_local chains the three idempotent first-run steps (migrate → bootstrap admin → seed scenario packs & model connections) into one command. Re-run it any time — it skips what already exists.

Daily work

uv run manage.py dev_server --embedded   # easiest: web UI + API + audit worker, zero-Docker → http://localhost:8000

Other options:

uv run manage.py runserver               # web UI + API only (no worker)
uv run manage.py dev_server              # web UI + worker, uses .env (edit POSTGRES_*/HATCHET_* to point at your infra)
SIMPLEAUDIT_LOCAL_SQLITE=1 uv run manage.py test infra   # tests

🐳 Self-Hosting

For teams or multi-user setups, use Docker Compose:

git clone https://github.com/SushantGautam/SimpleAuditStudio
cd SimpleAuditStudio
cp .env.example .env
# edit POSTGRES_PASSWORD and BOOTSTRAP_PASSWORD at minimum
docker compose up -d

Services: Web UI (:8000), PostgreSQL, Hatchet queue (:8888), Worker. Optional profile: --profile mock (mock model API).

See docs/deployment.md for production hardening, backups, and upgrades.

✨ What You Can Do

• Build versioned scenario sets and register OpenAI-compatible models
• Run experiments across models, scenario versions and settings, with a review step before launch
• Watch runs live, including per-repetition results
• Monitor models for drift on a schedule (interval or cron)
• Filter, customise and compare runs on an interactive dashboard

🏗️ Architecture

Browser → Django (gunicorn :8000 in Compose; runserver :7860 in the HF Space / uvx demo)
              ↓
         PostgreSQL 16 (domain + queue DBs)
              ↓
         Hatchet Server (durable job queue)
              ↓
         Worker (CPU pool) → SimpleAudit Engine
              Target → Auditor → Judge

📚 Documentation

• Architecture — system design, service boundaries, deployment topology
• Domain Model — scenarios, scenario sets, audit runs, immutability invariants
• Deployment — Docker Compose, production hardening, upgrades

🤝 Contributing

Architecture decisions are documented in docs/architecture.md.