verity-audit 1.4.1

Verity - AI Governance Through Enforcement (API + CLI + live audit SDK)

Verity - AI Governance Through Enforcement

Complete AI model audit system with CI/CD integration, evidence storage, and regulator-ready exports.

Quick Start

Install Dependencies

# Install root dependencies (concurrently)
npm install

# Install website and dashboard dependencies
npm run install:all

Run Everything

# Run website, dashboard, and API together
npm run dev:all

This will start:

• Website: http://localhost:3000
• Dashboard: http://localhost:3001
• API: http://localhost:8000

Run Individual Services

# Just the website
npm run dev:website

# Just the dashboard
npm run dev:dashboard

# Just the API
npm run dev:api

Project Structure

ai-audit-ci/
├── audit/              # CLI audit tool
│   ├── cli.py         # Main CLI interface
│   ├── metrics.py      # Fairness metrics
│   ├── evidence.py    # Report generation
│   └── export.py      # PDF/JSON exports
├── api/                # FastAPI backend
│   ├── main.py        # API server
│   ├── database.py    # Database models
│   ├── routes/        # API routes
│   └── manage.py      # Management CLI
├── website/            # Marketing website (React)
│   └── src/
│       ├── pages/     # Website pages
│       └── components/
└── dashboard/          # Dashboard app (React)
    └── src/
        ├── pages/     # Dashboard pages
        └── components/

Features

Core Audit Tool

• ✅ Fairness metrics (demographic parity, equal opportunity, disparate impact)
• ✅ SHA-256 hashing for all artifacts
• ✅ Immutable audit reports
• ✅ GitHub Actions integration
• ✅ Regulator-ready PDF/JSON exports

API & Evidence Store

• ✅ User authentication
• ✅ Organization & project management
• ✅ API key management
• ✅ Append-only audit storage
• ✅ Commit context tracking
• ✅ Policy versioning

Dashboard

• ✅ Project overview with statistics
• ✅ Audit timeline visualization
• ✅ Trend charts
• ✅ Policy management
• ✅ Regulator portal (Enterprise)

Website

• ✅ Dark, professional design
• ✅ Responsive layout
• ✅ Animated interactions
• ✅ Enterprise messaging

Development

Prerequisites

• Python 3.11+
• Node.js 18+
• npm or yarn

Setup

1. Install Python dependencies:

cd ai-audit-ci
pip install -r requirements.txt

2. Install Node dependencies:

npm run install:all

3. Initialize database:

cd api
python -m api.main

4. Run all services:

npm run dev:all

Usage

Run an Audit

cd ai-audit-ci
python -m audit.cli run --config examples/tabular/config.yml

Upload Audit Evidence

Set environment variables:

export VERITY_UPLOAD_TOKEN=your_project_upload_token
export VERITY_API_URL=https://verityai-production.up.railway.app

Then run with upload:

python -m audit.cli run --config config.yml --upload

GitHub Actions

Add to .github/workflows/audit.yml:

name: AI Governance Audit (Verity)
on: [pull_request]
jobs:
  audit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: |
          pip install verity-audit
          verity run --config config.yml --upload
        env:
          VERITY_UPLOAD_TOKEN: ${{ secrets.VERITY_UPLOAD_TOKEN }}
          DATASET_URL: ${{ secrets.DATASET_URL }} # For datasets not in repo
          GITHUB_SHA: ${{ github.sha }}
          GITHUB_REF_NAME: ${{ github.ref_name }}
          GITHUB_REPOSITORY: ${{ github.repository }}

Note: For datasets stored outside the repository (S3, GCS, etc.), see CI/CD Dataset Patterns for configuration options.

Live audit SDK (production LLM calls)

Live audits are automatic via swap-in clients only: install the package, set env vars, replace the vendor client with a Verity wrapper. There is no separate “send audit” call — prompts and completions are captured on a non-blocking side channel to POST /v1/live-audits.

pip install "verity-audit[live]"

Use your project upload token (same as CLI uploads). You can set either name:

export VERITY_API_KEY=your_project_upload_token
# or: export VERITY_UPLOAD_TOKEN=...
export VERITY_API_URL=https://verityai-production.up.railway.app

Public facade (verity package)

from verity import VerityOpenAI

client = VerityOpenAI(
    openai_api_key="...",
    verity_api_key="...",  # or rely on VERITY_API_KEY / VERITY_UPLOAD_TOKEN in the environment
)
resp = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Hello"}],
)

Async: from verity import VerityAsyncOpenAI. Anthropic: VerityAnthropic / VerityAsyncAnthropic. Gemini (google.genai): VerityGemini.

Lower-level module (verity_live) — same behavior with api_key + optional verity_upload_token / verity_api_url (or env vars above).

API Endpoints

• POST /v1/audits - Upload audit evidence
• GET /v1/audits/{audit_id} - Retrieve audit
• POST /auth/signup - Create account
• POST /auth/login - Sign in
• GET /dashboard/orgs/{org_id}/projects - List projects
• GET /dashboard/projects/{project_id}/audits - Get audit history
• GET /dashboard/projects/{project_id}/trends - Get trend data

See api/README.md for full API documentation.

License

Proprietary - All rights reserved