MiniCrit-7B: Adversarial AI Validation for Autonomous Systems

🛡️ Catch AI reasoning flaws before they become failures

Quick Start • MCP Integration • OpenAI API • LangChain • Training • Benchmarks

🎯 The Problem

Autonomous AI systems fail silently. They produce confident-sounding outputs with hidden flaws—overconfidence, missing risks, logical errors, hallucinations. By the time you notice, it's too late.

Traditional testing catches bugs. MiniCrit catches bad reasoning.

💡 The Solution

MiniCrit is a specialized AI "devil's advocate" that validates reasoning before actions are taken. It integrates with any AI system via MCP (Model Context Protocol) to provide real-time adversarial critique.

Your AI Agent → MiniCrit Validation → Safer Decisions
     ↓                   ↓                    ↓
  "Buy AAPL,         "Overconfidence:      Execute with
   95% confident"     only 2 data points,   reduced size
                      missing earnings       or skip
                      risk"

📊 Results and evaluation status

Read this before relying on any MiniCrit output.

• The checkpoint published at wmaousley/MiniCrit-7B (Qwen2-7B-Instruct base) is a research checkpoint at 9.8% of planned training and has not been evaluated on held-out benchmarks.
• Antagon's only measured MiniCrit result (21/21 seeded flaws detected, 0/21 false positives on a 42-item set, greedy decoding) belongs to a different, unpublished adapter on mistralai/Mistral-7B-v0.1. It does not describe this package's default model.
• Performance figures previously shown here (F1, false-positive rate, latency, Sharpe-ratio and false-signal improvements, live-validation counts, and comparisons with other models) have been withdrawn until they can be republished with the measurement record they came from: model artifact, dataset, decoding settings and method.
• In responses, confidence is null (not yet produced by the model), valid is null, and severity comes from a keyword heuristic over the critique text (severity_basis: "keyword_heuristic"). See Output Format.

🚀 Quick Start

Option 1: pip install (Recommended)

pip install minicrit

from minicrit import MiniCrit

# Initialize (downloads model automatically)
critic = MiniCrit()

# Validate reasoning
result = critic.validate(
    "Stock will rise because it rose yesterday",
    domain="trading"
)

print(result.valid)      # False
print(result.severity)   # "high" 
print(result.critique)   # "This reasoning exhibits recency bias..."
print(result.flags)      # ["overconfidence", "insufficient_evidence"]

Option 2: CLI

# Single validation
minicrit "Buy AAPL, 95% confident" --domain trading

# From file
minicrit --file rationales.txt --output results.json

# JSON output
minicrit "MACD crossover signals buy" --json

Installation Extras

# Core only
pip install minicrit

# With MCP server support
pip install minicrit[mcp]

# With training utilities
pip install minicrit[training]

# Everything
pip install minicrit[all]

Option 3: Docker

cd docker
docker-compose up -d
curl http://localhost:8000/critique \
  -H "Content-Type: application/json" \
  -d '{"rationale": "Buy signal based on MACD crossover", "domain": "trading"}'

Option 4: MCP (Claude Desktop / Claude Code)

// ~/Library/Application Support/Claude/claude_desktop_config.json
{
  "mcpServers": {
    "minicrit": {
      "command": "python3",
      "args": ["/path/to/MiniCrit-7B/src/mcp/server.py"],
      "env": {
        "MINICRIT_ADAPTER": "wmaousley/MiniCrit-7B",
        "MINICRIT_BASE_MODEL": "Qwen/Qwen2-7B-Instruct"
      }
    }
  }
}

Then in Claude: "Use validate_reasoning to check: Buy AAPL, RSI shows oversold"

🔌 MCP Integration

MiniCrit implements the Model Context Protocol (MCP)—the industry standard for AI tool integration, backed by Anthropic, OpenAI, Google, and Microsoft.

Any MCP-compatible AI can call MiniCrit:

┌─────────────────┐         ┌─────────────────┐
│  Claude / GPT   │         │    MiniCrit     │
│  Gemini / etc.  │◄───────►│   MCP Server    │
└─────────────────┘   MCP   └─────────────────┘
         │                           │
         │    Tool: validate_reasoning
         │    Input: {rationale, domain}
         │    Output: {valid, severity, critique, flags}
         │
         ▼
   Safer AI Decisions

MCP Server Options

Claude Desktop Setup

// ~/Library/Application Support/Claude/claude_desktop_config.json
{
  "mcpServers": {
    "minicrit": {
      "command": "python3",
      "args": ["-m", "src.mcp.server"],
      "cwd": "/path/to/MiniCrit-7B",
      "env": {
        "MINICRIT_ADAPTER": "wmaousley/MiniCrit-7B",
        "MINICRIT_BASE_MODEL": "Qwen/Qwen2-7B-Instruct",
        "MINICRIT_PRELOAD": "true"
      }
    }
  }
}

Production HTTP Server

# Start with API key authentication
export MINICRIT_API_KEYS="key1,key2,key3"
export MINICRIT_RATE_LIMIT=60  # requests per minute
python -m src.mcp.server_prod

# Or with gunicorn for production
gunicorn src.mcp.server_prod:app -w 4 -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8000

Production endpoints:

MCP Tools

Programmatic Usage

from src.mcp import (
    ModelManager,
    CritiqueGenerator,
    CritiqueResult,
    InputSanitizer,
    get_model_manager,
    get_critique_generator,
)

# Singleton model manager (thread-safe)
manager = ModelManager.get_instance()

# Generate critiques
generator = CritiqueGenerator(manager)
result = generator.generate(
    rationale="AAPL will rise because momentum is positive",
    domain="trading",
    context="Portfolio allocation decision"
)

print(result.valid)       # False
print(result.severity)    # "high"
print(result.critique)    # "This reasoning exhibits..."
print(result.flags)       # ["overconfidence", "insufficient_evidence"]
print(result.latency_ms)  # 42.3

# Async version
result = await generator.generate_async(
    rationale="Buy signal based on MACD crossover",
    domain="trading"
)

Supported Domains

trading • finance • defense • cybersecurity • medical • risk_assessment • resource_allocation • planning_scheduling • general

Output Format

{
  "valid": null,
  "valid_keyword_heuristic": false,
  "severity": "high",
  "severity_basis": "keyword_heuristic",
  "critique": "This reasoning exhibits recency bias. A single day's price movement has no predictive power...",
  "confidence": null,
  "confidence_source": "not_emitted",
  "flags": ["overconfidence", "insufficient_evidence", "missing_consideration"],
  "domain": "trading",
  "decoding": {"max_new_tokens": 256, "do_sample": false, "repetition_penalty": 1.15},
  "latency_ms": 42.3,
  "request_id": "req_abc123"
}

confidence will be populated from the model once it emits the structured format defined in Antagon's output contract v1. Until then it is null rather than an estimate. severity is matched from keywords in the critique without handling negation ("no critical flaw" matches critical), so treat it as a hint, not a verdict.

Severity Levels: pass → low → medium → high → critical

Common Flags:

• overconfidence - Certainty without sufficient evidence
• missing_consideration - Key factors not addressed
• logical_inconsistency - Contradictions in reasoning
• insufficient_evidence - Claims lack support
• unaddressed_risk - Potential risks not considered

🌐 OpenAI-Compatible API

MiniCrit provides an OpenAI-compatible API server, allowing drop-in replacement for any OpenAI client:

Start the Server

# Start the OpenAI-compatible server
python -m src.openai_compat_server

# Or with uvicorn for production
uvicorn src.openai_compat_server:app --host 0.0.0.0 --port 8080

Use with Any OpenAI Client

from openai import OpenAI

# Point to MiniCrit server
client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="minicrit"  # Any string works
)

response = client.chat.completions.create(
    model="minicrit-7b",
    messages=[
        {"role": "user", "content": "AAPL is bullish because momentum is positive"}
    ],
    temperature=0.7,
    max_tokens=512
)

print(response.choices[0].message.content)

Endpoints

Streaming

# Stream responses
stream = client.chat.completions.create(
    model="minicrit-7b",
    messages=[{"role": "user", "content": "Validate this reasoning..."}],
    stream=True
)

for chunk in stream:
    print(chunk.choices[0].delta.content, end="")

🦜 LangChain Integration

MiniCrit integrates seamlessly with LangChain for building AI pipelines:

Installation

pip install minicrit langchain-core

Basic Usage

from langchain_minicrit import MiniCritLLM, MiniCritChat, MiniCritValidator

# As an LLM
llm = MiniCritLLM(base_url="http://localhost:8080/v1")
critique = llm.invoke("Stock will rise due to momentum")
print(critique)

# As a Chat Model
from langchain_core.messages import HumanMessage, SystemMessage

chat = MiniCritChat(base_url="http://localhost:8080/v1")
response = chat.invoke([
    SystemMessage(content="You are a financial reasoning critic."),
    HumanMessage(content="Buy AAPL because it went up yesterday")
])
print(response.content)

Validation Chain

from langchain_minicrit import MiniCritValidator

validator = MiniCritValidator(
    base_url="http://localhost:8080/v1",
    threshold=70.0  # Minimum score to pass
)

result = validator.validate("MACD crossover signals a buy opportunity")

print(f"Valid: {result.is_valid}")
print(f"Score: {result.score}")
print(f"Critique: {result.critique}")
print(f"Flaws: {[f.antagon_flaw_id for f in result.flaws]}")

In LangChain Pipelines

from langchain_core.runnables import RunnablePassthrough
from langchain_minicrit import MiniCritValidationChain

# Create validation chain
validator = MiniCritValidationChain(threshold=70.0)

# Compose with other runnables
pipeline = RunnablePassthrough() | validator

result = pipeline.invoke("High confidence trade based on sentiment analysis")

🔍 What MiniCrit Detects

Example

Input:

"AAPL long: The stock has risen 3 days in a row, momentum is clearly bullish. 95% confident this continues."

MiniCrit Output:

⚠️ HIGH SEVERITY - Multiple reasoning flaws detected:

1. Overconfidence: 95% confidence is not supported by the evidence provided
2. Recency Bias: 3 days of price movement has minimal predictive value
3. Missing Risk Factors: No consideration of upcoming earnings, macro events, or sector rotation

Flags: overconfidence, insufficient_evidence, unaddressed_risk

🏗️ Architecture

┌──────────────────────────────────────────────────────────────────┐
│                      MiniCrit-7B System                          │
├──────────────────────────────────────────────────────────────────┤
│                                                                  │
│  ┌────────────┐    ┌─────────────┐    ┌───────────────────────┐ │
│  │ AI System  │───▶│   MiniCrit  │───▶│   Validated Output    │ │
│  │ (Any LLM)  │    │   Server    │    │                       │ │
│  └────────────┘    └──────┬──────┘    │ • valid: null (interim)│ │
│                           │           │ • severity: enum      │ │
│                    ┌──────▼──────┐    │ • critique: string    │ │
│                    │  Qwen2-7B   │    │ • flags: list         │ │
│                    │    Base     │    │ • confidence: null    │ │
│                    └──────┬──────┘    └───────────────────────┘ │
│                           │                                      │
│                    ┌──────▼──────┐                              │
│                    │   LoRA      │  40.4M trainable params      │
│                    │  Adapter    │  LoRA on CritiqueBank-11M (9.8% of planned steps)  │
│                    └─────────────┘                              │
│                                                                  │
└──────────────────────────────────────────────────────────────────┘

📈 Training

Model Specifications

Dataset: CritiqueBank-11M

The variation is on the critique side; the rationale side is a closed set of 215 inputs. A model trained on this has seen 215 distinct inputs and should not be assumed to generalise to unseen rationale forms without evaluation on independently generated data. See the dataset card for the full composition.

Correction. Earlier versions of this README described the dataset as five named components (LogicFlaw, FactCheck, BiasDetect, RiskMissing, DomainSpecific) spanning trading, defense and medical domains. No such components or domains exist in the deposited data, which is single-domain. The table above is the measured composition.

Published: DOI 10.5281/zenodo.23091964 (supersedes 10.5281/zenodo.18159342)

Training Progress

Loss
3.2 │██
2.4 │  ████
1.6 │      ██████
0.8 │            ████████████
0.4 │                        ████████████████  ← Current
    └─────────────────────────────────────────
    0%        25%        50%        75%     100%

Run Training

# Lambda Labs (H100)
python train_minicrit_7b.py --config configs/7b_lora.yaml

🧪 Benchmarking

Compare MiniCrit models head-to-head:

python src/benchmark/benchmark_models.py \
  --eval-data data/eval_holdout.jsonl \
  --model-1 wmaousley/MiniCrit-1.5B \
  --model-2 wmaousley/MiniCrit-7B \
  --judge-sample 200  # Optional: LLM-as-judge comparison

Metrics Computed

🔧 Advanced: Improve Your Model

Generate Hard Training Examples

# Uses Claude Sonnet (~$30 for 5K examples)
export ANTHROPIC_API_KEY=your-key
python src/training/generate_hard_examples.py --count 5000

Direct Preference Optimization (DPO)

# Generate preference pairs
python src/training/generate_dpo_data.py \
  --input eval_holdout.jsonl \
  --model wmaousley/MiniCrit-7B \
  --output dpo_pairs.jsonl

# Run DPO training
python src/training/train_dpo.py \
  --model wmaousley/MiniCrit-7B \
  --data dpo_pairs.jsonl \
  --output minicrit-7b-dpo

See docs/MODEL_EXCELLENCE_GUIDE.md for the full improvement roadmap.

📁 Repository Structure

MiniCrit-7B/
├── src/
│   ├── mcp/                    # MCP Server Implementation
│   │   ├── core.py             # Thread-safe ModelManager, CritiqueGenerator
│   │   ├── server.py           # Local stdio (Claude Desktop)
│   │   ├── server_prod.py      # Production HTTP + auth + rate limiting
│   │   └── server_http.py      # Basic HTTP server
│   ├── benchmark/              # Model Evaluation
│   │   └── benchmark_models.py
│   ├── training/               # Training Utilities
│   │   ├── generate_hard_examples.py
│   │   ├── generate_dpo_data.py
│   │   └── train_dpo.py
│   ├── openai_compat_server.py # OpenAI-compatible API
│   ├── config.py
│   ├── data.py
│   ├── model.py
│   ├── training.py
│   ├── evaluation.py
│   └── api.py
├── langchain_minicrit/         # LangChain Integration
│   ├── __init__.py
│   ├── llm.py                  # LLM wrapper
│   ├── chat.py                 # Chat model
│   ├── validator.py            # Validation chain
│   └── callbacks.py            # Callback handlers
├── docker/
│   ├── Dockerfile
│   ├── docker-compose.yml
│   └── requirements.txt
├── configs/
│   ├── claude_desktop_config.json
│   ├── 7b_lora.yaml
│   └── deepspeed_gh200.json
├── scripts/
├── tests/                      # 489 tests
├── docs/
│   ├── DEPLOYMENT_GUIDE.md
│   └── MODEL_EXCELLENCE_GUIDE.md
└── CHANGELOG.md

🐳 Deployment Options

Docker (Recommended)

cd docker
cp .env.example .env
# Edit .env with your settings
docker-compose up -d

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: minicrit
spec:
  replicas: 2
  template:
    spec:
      containers:
      - name: minicrit
        image: antagoninc/minicrit:latest
        resources:
          limits:
            nvidia.com/gpu: 1

Production HTTP Server

# With authentication & rate limiting
export MINICRIT_API_KEYS="key1,key2,key3"
python src/mcp/server_prod.py

See docs/DEPLOYMENT_GUIDE.md for complete instructions.

⚙️ Configuration

Environment Variables

Model Configuration:

Production Server (server_prod.py):

Rate Limit Tiers:

8-bit / 4-bit Quantization

MiniCrit supports quantization via bitsandbytes to reduce memory usage:

# 8-bit quantization (~50% memory reduction)
export MINICRIT_QUANTIZATION=8bit
python -m src.mcp.server

# 4-bit quantization (~75% memory reduction, slight quality loss)
export MINICRIT_QUANTIZATION=4bit
python -m src.mcp.server

Requirements:

• bitsandbytes>=0.41.0 (included in dependencies)
• CUDA-capable GPU
• Falls back to fp16 gracefully if bitsandbytes is unavailable

Input Sanitization (Security)

MiniCrit includes built-in input sanitization to protect against prompt injection attacks:

• Injection Pattern Detection: Blocks patterns like "ignore previous instructions", "system:", etc.
• Control Character Removal: Strips null bytes, zero-width chars, and other control characters
• Length Validation: Enforces max 50,000 character rationales, 100 char domains
• Domain Whitelisting: Only accepts valid domains from the supported list

from minicrit import InputSanitizer, InputSanitizationError

sanitizer = InputSanitizer()
try:
    clean_input = sanitizer.validate_rationale(user_input)
except InputSanitizationError as e:
    print(f"Blocked: {e}")

🎯 Use Cases

📜 Citation

@software{minicrit2026,
  author = {Ousley, William Alexander and Ousley, Jacqueline Villamor},
  title = {MiniCrit: Adversarial AI Validation for Autonomous Systems},
  year = {2026},
  publisher = {Antagon Inc.},
  url = {https://github.com/antagoninc/MiniCrit-7B}
}

🙏 Acknowledgments

• Lambda Labs - GPU compute grant for H100 training
• Anthropic - MCP standard development

No high-performance computing facility and no federal funding of any kind supported this work.

📄 License

Apache 2.0 - See LICENSE

Antagon Inc.
Making AI Systems Safer Through Adversarial Testing

Website • Contact • CAGE: 17E75 • UEI: KBSGT7CZ4AH3

William Alexander Ousley - Co-Founder & CEO
Jacqueline Villamor Ousley - Co-Founder & CTO