veritas-ml 0.1.0

AI Bias Detection and Ethics Compliance Agent

Veritas - AI Bias Detection and Ethics Compliance

A Python package for detecting bias in ML models and checking compliance with AI ethics laws using RAG-powered recommendations.

Features

• Automatic bias detection using IBM AIF360 toolkit
• AI ethics compliance checking via LangGraph ReAct agent
• RAG-powered law lookup from bundled AI ethics knowledge base
• sklearn-compatible wrappers for drop-in integration
• Auto-initialization - knowledge base ingests automatically on first import

Installation

# From source
pip install -e .

# Set API key for LLM provider
export OPENROUTER_API_KEY="your-key-here"
# OR
export GROQ_API_KEY="your-key-here"

Quick Start

Pattern 1: Drop-in sklearn Replacement

from veritas import VeritasClassifier
from sklearn.linear_model import LogisticRegression

# Wrap any sklearn classifier
clf = VeritasClassifier(
    base_estimator=LogisticRegression(),
    protected_attribute="gender",
    run_audit=True,
    generate_report=True
)

# Fit with sensitive features
clf.fit(X_train, y_train, sensitive_features=sensitive_train)

# Get reports
print(clf.get_bias_report())         # AIF360 metrics
print(clf.get_compliance_report())   # Mitigation recommendations

Pattern 2: Manual Audit After Training

from veritas import BiasDetector, ComplianceChecker

# Train your model normally
model.fit(X_train, y_train)

# Audit separately
detector = BiasDetector(
    protected_attribute="sex",
    privileged_classes=["Male"]
)
bias_report = detector.audit(model, X_test, y_test, sensitive_test)

# Check compliance
checker = ComplianceChecker()
compliance_report = checker.check(bias_report)

Pattern 3: Full End-to-End Audit

from veritas import ComplianceChecker

checker = ComplianceChecker()
result = checker.full_audit(
    X_train=X_train, y_train=y_train,
    X_test=X_test, y_test=y_test,
    sensitive_train=sensitive_train,
    sensitive_test=sensitive_test,
    protected_attribute="sex"
)

print(result["compliance_report"])

Configuration

LLM Providers

Veritas loads settings in this order:

1. Explicit constructor overrides (e.g. ComplianceChecker(llm_provider="groq"))
2. Environment variables (auto-loaded from veritas/.env)
3. defaults.json (veritas/defaults.json or repository-root defaults.json)
4. ~/.veritas/config.json

Environment variables:

# OpenRouter
export LLM_PROVIDER=openrouter
export OPENROUTER_API_KEY=your-key

# Ollama (local)
export LLM_PROVIDER=ollama
export OLLAMA_BASE_URL=http://localhost:11434

# Groq
export LLM_PROVIDER=groq
export GROQ_API_KEY=your-key

Or set defaults.json:

{
  "provider": "ollama",
  "llm": "qwen3.5:4b"
}

You can still use ~/.veritas/config.json:

{
  "provider": "openrouter",
  "llm": "openai/gpt-4o-mini"
}

Knowledge Base

The AI ethics knowledge base auto-initializes on first import at ~/.veritas/vector_store/.

API Reference

VeritasClassifier

sklearn-compatible wrapper with built-in bias auditing.

Parameters:

• base_estimator: sklearn estimator (default: LogisticRegression)
• protected_attribute: Name of protected attribute
• privileged_classes: Privileged class values
• run_audit: Run bias audit on fit()
• generate_report: Generate compliance report

Methods:

• fit(X, y, sensitive_features=None) - Train and audit
• predict(X) - Predict labels
• get_bias_report() - Get AIF360 metrics
• get_compliance_report() - Get mitigation recommendations
• has_bias() - Check if bias detected

BiasDetector

Low-level bias detection with AIF360.

Methods:

• audit(model, X_test, y_test, sensitive_features) - Run audit

ComplianceChecker

High-level compliance checking with LLM agent.

Methods:

• check(bias_report) - Analyze bias report
• full_audit(...) - End-to-end training + audit + compliance

Dependencies

• aif360 >= 0.6.1
• langgraph >= 0.3.0
• langchain-* providers
• chromadb >= 0.5.0
• scikit-learn >= 1.3.0

License

MIT