veritas-ml 0.1.8

AI Bias Detection and Ethics Compliance Agent

Veritas - AI Bias Detection and Ethics Compliance

A Python package for detecting bias in ML models and checking compliance with AI ethics laws using RAG-powered recommendations.

Features

• Automatic bias detection using IBM AIF360 toolkit
• AI ethics compliance checking via LangGraph ReAct agent
• RAG-powered law lookup from bundled AI ethics knowledge base
• sklearn-compatible wrappers for drop-in integration
• Auto-initialization - knowledge base ingests automatically on first import

Installation

# From PyPI
pip install veritas-ml

# From source
pip install -e .

# Set API key for LLM provider
export OPENROUTER_API_KEY="your-key-here"
# OR
export GROQ_API_KEY="your-key-here"

Quick Start

Create bias_params.json beside your main Python file, or in the directory where you run Python:

{
  "protected_attributes": ["gender"]
}

Veritas will read this automatically, so your application code does not need to repeat sensitive/protected attribute names.

Pattern 1: Drop-in sklearn Replacement

from veritas import VeritasClassifier
from sklearn.linear_model import LogisticRegression

# Wrap any sklearn classifier
clf = VeritasClassifier(
    base_estimator=LogisticRegression(),
    run_audit=True,
    generate_report=True
)

# Fit with sensitive features
clf.fit(X_train, y_train, sensitive_features=sensitive_train)

# Get reports
print(clf.get_bias_report())         # AIF360 metrics
print(clf.get_compliance_report())   # Mitigation recommendations

Pattern 2: Manual Audit After Training

from veritas import BiasDetector, ComplianceChecker

# Train your model normally
model.fit(X_train, y_train)

# Audit separately
detector = BiasDetector()
bias_report = detector.audit(model, X_test, y_test, sensitive_test)

# Check compliance
checker = ComplianceChecker()
compliance_report = checker.check(bias_report)

Pattern 3: Full End-to-End Audit

from veritas import ComplianceChecker

checker = ComplianceChecker()
result = checker.full_audit(
    X_train=X_train, y_train=y_train,
    X_test=X_test, y_test=y_test,
    sensitive_train=sensitive_train,
    sensitive_test=sensitive_test,
)

print(result["compliance_report"])

Publishing to PyPI

This section documents how to release new versions of veritas-ml to the Python Package Index.

Prerequisites

1. PyPI Account: Create an account at https://pypi.org/
2. API Token: Generate at https://pypi.org/manage/account/#api-tokens
3. Token Access: Ensure token has scope for veritas-ml project

Set Up Your Token

Add to veritas/.env:

PYPI_API_TOKEN=pypi-AgEIcHlwaS5vcmcC...

Or set environment variable:

export PYPI_API_TOKEN="your-token-here"

Using the Upload Script (Recommended)

# Navigate to veritas directory
cd veritas

# Option 1: Test first, then live (recommended)
python scripts/upload_pypi.py --test

# Option 2: Direct live upload
python scripts/upload_pypi.py

# Option 3: Build only (for verification)
python scripts/upload_pypi.py --build

Manual Upload (Alternative)

# Install build tools
pip install build twine

# Clean previous builds
rm -rf dist/ build/ src/*egg-info/

# Build
python -m build

# Upload to TestPyPI first (recommended)
twine upload --repository testppi dist/*
# Verify: https://test.pypi.org/project/veritas-ml/
# Install: pip install --index-url https://test.pypi.org/simple veritas-ml

# Upload to live PyPI
twine upload --repository pypi dist/*

Version Bumping

Before uploading, increment version in pyproject.toml:

[project]
version = "0.1.1"  # Increment as needed

Troubleshooting

Error	Solution
name too similar	Change package name in pyproject.toml
403 Forbidden	Check API token permissions
[mutex.cc : 452]	TensorFlow and PyArrow are included by default (see Dependencies)

Dependencies

LLM Providers

Veritas loads settings in this order:

1. Explicit constructor overrides (e.g. ComplianceChecker(llm_provider="groq"))
2. Environment variables (auto-loaded from veritas/.env)
3. defaults.json (veritas/defaults.json or repository-root defaults.json)
4. ~/.veritas/config.json

Environment variables:

# OpenRouter
export LLM_PROVIDER=openrouter
export OPENROUTER_API_KEY=your-key

# Ollama (local)
export LLM_PROVIDER=ollama
export OLLAMA_BASE_URL=http://localhost:11434

# Groq
export LLM_PROVIDER=groq
export GROQ_API_KEY=your-key

Or set defaults.json:

{
  "provider": "ollama",
  "llm": "qwen3.5:4b"
}

You can still use ~/.veritas/config.json:

{
  "provider": "openrouter",
  "llm": "openai/gpt-4o-mini"
}

Knowledge Base

The AI ethics knowledge base auto-initializes on first import at ~/.veritas/vector_store/.

API Reference

VeritasClassifier

sklearn-compatible wrapper with built-in bias auditing.

Parameters:

• base_estimator: sklearn estimator (default: LogisticRegression)
• protected_attribute: Name of protected attribute
• privileged_classes: Privileged class values
• run_audit: Run bias audit on fit()
• generate_report: Generate compliance report

Methods:

• fit(X, y, sensitive_features=None) - Train and audit
• predict(X) - Predict labels
• get_bias_report() - Get AIF360 metrics
• get_compliance_report() - Get mitigation recommendations
• has_bias() - Check if bias detected

BiasDetector

Low-level bias detection with AIF360.

Methods:

• audit(model, X_test, y_test, sensitive_features) - Run audit

ComplianceChecker

High-level compliance checking with LLM agent.

Methods:

• check(bias_report) - Analyze bias report
• full_audit(...) - End-to-end training + audit + compliance

Dependencies

• aif360 >= 0.6.1
• langgraph >= 0.3.0
• langchain-* providers
• chromadb >= 0.5.0
• scikit-learn >= 1.3.0

License

MIT