AgentDeck 🎮

The game console for AI agents.

An open system for putting AI agents inside explicit game worlds, observing what they do, and producing traceable knowledge about their behavior.

Why Games? · Replay · Quick Start · Research Arc · Research Artifacts · Current Boundary · Docs · Examples · Specs · AI-First

---

🎯 Purpose & Vision

AgentDeck turns behavioral questions into explicit, inspectable situations: define a Game or reuse an existing one, compose its Players, run controlled Matches, preserve the canonical event stream, and determine what the resulting Records support.

It is useful when static prompt-response evaluation is not enough. By putting agents inside structured games, AgentDeck makes state, incentives, and resource tradeoffs explicit so behavior is easier to observe, compare, replay, and explain.

Current 0.4 Scope Boundary

AgentDeck owns execution truth: game state transitions, provider interactions, resolved actions, lifecycle events, runtime configuration, costs, recordings, and replay. It is intentionally obsessive about making a Record describe what happened.

The current 0.4 source candidate ships this execution kernel plus the redesigned Research path: Study preparation and selected execution, exact Record corpora, deterministic Measures, immutable Evidence, and authored Findings with granular citations. Historical Research remains public under research/, and the former implementation remains reproducible at the agentic-edge-research tag. Historical agentdeck-research-* commands are not part of the current API.

The current command surface is one coherent journey:

agentdeck study inspect
agentdeck study validate
agentdeck study run
agentdeck study analyze
agentdeck study report

The architectural boundary remains strict as Research returns: canonical Records state what happened; deterministic Measures and Evidence are derived separately; Findings remain explicit authored interpretations.

New Records preserve the effective Game and Player configuration, a scoped Game implementation fingerprint, exact retained conversation selection, provider-native SDK arguments and response metadata, retries, stop reasons, costs, and state transitions. Built-in Controllers apply only actions explicitly declared in an ACTION: field; mentions inside reasoning or narration fail closed and remain visible as parse failures.

---

🎬 Run, Record, Replay

Run AI-agent matches from Python, record every turn as structured artifacts, then replay the decisions in a browser viewer for inspection and storytelling.

---

🎮 Why Games?

Most LLM benchmarks measure knowledge through static questions. AgentDeck focuses on behavior: maintaining state, adapting over time, and making tradeoffs inside explicit rules.

Game scenarios work well because they make the important variables legible:

• Constrained environments – Isolate specific variables (for example, resource scarcity or turn order)
• Iterative decision making – Agents live with consequences, testing longer-horizon behavior
• Social dynamics – Multiplayer games reveal cooperation, betrayal, and negotiation patterns
• Measurable outcomes – Win/lose provides a clean signal for cost/quality trade-offs

---

🔎 Flagship Evidence

The Agentic Edge study uses AgentDeck to test whether agent design can overcome model-tier gaps in sequential decision games.

In FixedDamage, the same lower-tier model moves from failure to a tier inversion as the agent wrapper changes:

Agent configuration	Opponent	Result
FlashLite S0 action-only	GPT-4o-mini S0 action-only	0/48 wins (0.0%)
FlashLite S1 reasoning controller	GPT-4o-mini S0 action-only	34/48 wins (70.8%)
FlashLite S3 reasoning + HP grounding	GPT-4o-mini S0 action-only	38/48 wins (79.2%)

The VariableDamage transfer result is more cautious: the adapted risk-grounded stack wins its same-model mechanism test, but the cross-tier result is seat-sensitive and not statistically strong. That caveat is the point: AgentDeck is built to expose behavior, not hide messy evidence.

Study artifacts are mirrored on Hugging Face: dataset + recordings · curated replay viewer

The current architecture reproduces the 432 primary/supplemental Matches from the pinned public dataset through Study → corpus → Measures → Evidence → Findings. See the reproduction guide.

---

🚀 Quick Start

Install: pip install agentdeck-ai (import as agentdeck)

AI-first prompt: Ask Claude, Codex, or your coding agent: “Learn AgentDeck from the README, create a tiny tic-tac-toe game, run a few matches, then inspect and replay the Records.”

Version: This documentation describes agentdeck-ai==0.4.0, including the current execution and Research contracts. Upgrading from 0.2.0 requires migration; see the 0.4.0 release notes.

Installation

PyPI stable install:

# Install the version documented here
pip install agentdeck-ai==0.4.0

# With provider SDKs
pip install agentdeck-ai[openai]      # OpenAI SDK
pip install agentdeck-ai[anthropic]   # Anthropic SDK
pip install agentdeck-ai[google]      # Google Gen AI SDK (Vertex mode)
pip install agentdeck-ai[providers]   # All provider SDKs

# Development install
pip install agentdeck-ai[dev]

Source install (for contributors):

git clone https://github.com/agentdeck/agentdeck.git
cd agentdeck
pip install -e ".[dev]"

Your First Run

from agentdeck import (
    ActionOnlyController,
    AgentDeck,
    FixedDamageGame,
    GPTPlayer,
    ReasoningController,
)

# 1. Create a game
game = FixedDamageGame(
    max_health=100,
    attack_damage=20,
    potion_heal=30,
    starting_potions=3,
    information_level="full",  # use "partial" to hide opponent HP/potions
)

# 2. Create AI players: same model, different behavioral interface
players = [
    GPTPlayer(
        name="SameModel-AO",
        model="gpt-4o-mini",
        temperature=0.7,
        controller=ActionOnlyController(),
    ),
    GPTPlayer(
        name="SameModel-RC",
        model="gpt-4o-mini",
        temperature=0.7,
        controller=ReasoningController(),
    ),
]

# Models must be provided explicitly for every provider-backed player.

# 3. Run matches
with AgentDeck(game=game) as deck:
    results = deck.play(
        players=players,
        matches=1,
        seed=42,  # Reproducible!
    )

# 4. Inspect the factual outcome summary
print(f"Win rates: {results.win_rates}")

🔒 Models are explicit
Provider-backed players never fall back to defaults; pass model= for every GPT/Claude/Gemini player.

ℹ️ Provider credentials
Set the provider-specific environment variables before running examples (OPENAI_API_KEY, ANTHROPIC_API_KEY, and VERTEX_PROJECT_ID/VERTEX_LOCATION for Gemini). For Gemini on Vertex, AgentDeck also supports GOOGLE_APPLICATION_CREDENTIALS_B64 for base64-encoded service-account JSON. Start from .env.example for local setup.

📝 .env loading policy
AgentDeck does not auto-load .env at the library level. Source it in your shell or load it in your entry script. In bash/zsh, a simple local setup is: set -a; source .env; set +a

✅ First real provider-backed run Start with matches=1 so you can confirm credentials, recordings, and replay before scaling up.

🎮 FixedDamageGame information level information_level="full" shows both players' HP and potion counts. information_level="partial" hides the opponent's HP and potions while still showing last actions.

Try AgentDeck Without API Keys

• Run python examples/mock_demo.py
• Uses MockPlayer (deterministic) so no LLM providers are needed
• Shows live reporting + progress + stats, and saves recordings under agentdeck_runs/mock_demo/<session>/records/

Recommended Learning Path

1. examples/mock_demo.py — verify the install with a zero-provider run
2. examples/first_game_walkthrough.py — build a tiny game and replay it
3. examples/minimal_experiment.py — run the smallest real provider-backed experiment
4. examples/prepared_assembly.py — seal a complete composition before execution
5. examples/spectator_example.py and examples/replay_minimal.py — add monitoring and replay workflows

For the full ladder, see examples/README.md.

Prepare, Inspect, Then Execute an Exact Composition

For approval or remote-host boundaries, define one entrypoint whose create_assembly() returns a complete Assembly. prepare_assembly() loads and content-addresses that composition without creating Players or calling a provider. After an external authority accepts the returned identity, execute_prepared_assembly() reloads it, rejects any change before Player construction, and executes every declared run through AgentDeck.play.

See examples/prepared_assembly.py and SPEC-ASSEMBLY.

Walkthroughs & Docs

• Build your first game + replay tour: examples/first_game_walkthrough.py
• Examples index: examples/README.md

Artifacts (Recordings + Logs)

After you run a batch, AgentDeck writes artifacts under agentdeck_runs/<session_id>/ (or your configured run_dir):

• records/ contains a batch_<batch_id>.json summary plus one match_*.json per match
• logs/ contains info.log and debug.log by default

Tip: open batch_<batch_id>.json first for the high-level batch summary, then open match_*.json for the full audit trail, replay source, prompts, raw responses, parsed actions, costs, and event timeline.

Parallel Execution (Workload-Dependent Speedups)

from agentdeck import AgentDeck, AgentDeckConfig
from agentdeck import LogLevel

# Configure parallel execution with real-time monitoring
config = AgentDeckConfig(
    seed=42,
    concurrency=10,      # Run 10 matches in parallel
    log_level=LogLevel.INFO
)

# Run 100 matches with automatic progress tracking
with AgentDeck(game=game, session=config) as deck:
    results = deck.play(players=players, matches=100)

# ProgressMonitor is auto-attached when concurrency > 1 (unless monitors=[] is provided)

Performance depends on provider rate limits and workload. For a determinism + concurrency comparison, see examples/test_parallel_execution.py.

---

⚙️ Architecture

The Console Metaphor

AgentDeck follows a gaming console metaphor with clean separation of concerns:

┌─────────────────────────────────────┐
│         AgentDeck (Facade)          │  ← You interact here
├─────────────────────────────────────┤
│         Console (Orchestrator)       │  ← Manages lifecycle
├─────────────┬───────────────────────┤
│    Game     │     EventBus          │  ← Game logic + Events
├─────────────┼───────────────────────┤
│   Players   │     Spectators        │  ← Configured actors + Observers
└─────────────┴───────────────────────┘

Single Turn Flow

Core Components

Games define rules and state

• Required properties: instructions, allowed_actions, default_handshake_template
• Core methods: setup(), get_view(), update(), status()
• State is JSON-serializable dicts (no complex objects)
• Examples: FixedDamageGame and ArchivistChoiceGame

Players are configured actors making decisions

• Three-phase lifecycle: Handshake → Turn → Conclusion
• Built-in: GPTPlayer, ClaudePlayer, GeminiPlayer, HumanPlayer, MockPlayer
• HumanPlayer provides synchronous local play with the same Controller, Record, and replay path
• Composable prompt templates via PromptBuilder

Controllers parse Player responses into actions

• ActionOnlyController - extracts single action token
• ReasoningController - extracts reasoning + action
• Handshake validation is built into the base Controller (default accepts exactly OK)

Renderers format game state for Player consumption

• TextRenderer - human-readable text format
• Custom renderers can provide JSON, images, etc.

Spectators observe matches without affecting execution

• MatchReporter - turn-by-turn reporting
• ProgressDisplay - real-time progress with ETA
• TokenUsageTracker - cost tracking per player/model
• StatsTracker - win rates and performance metrics

Recording & Replay

• Recorder - captures complete match data to JSON
• ReplayEngine - reconstructs matches with event parity guarantee

---

💡 Key Features

1. Event-Driven Observation

Everything is observable through events - no modifications needed to games:

from agentdeck import AgentDeck
from agentdeck.spectators import MatchReporter, TokenUsageTracker

# Add spectators for observation
with AgentDeck(game=game, spectators=[
    MatchReporter(),      # Turn-by-turn reporting
    TokenUsageTracker()   # Cost tracking
]) as deck:
    results = deck.play(players, matches=1)

2. Complete Recording & Replay

Every match is automatically recorded with full metadata:

from pathlib import Path

from agentdeck import AgentDeck, MatchReporter

with AgentDeck(game=game) as deck:
    results = deck.play(players, matches=3, seed=7)

    # Replay from memory (no file I/O)
    deck.replay(match=results[0], spectators=[MatchReporter()], speed=0.0)

    # Or replay from disk (recorded under records/)
    record_dir = Path(deck.session.record_directory)
    match_path = sorted(record_dir.glob("match_*.json"))[0]
    deck.replay(path=match_path, spectators=[MatchReporter()], speed=0.0)

Replay Parity Guarantee: Replay emits identical event stream as live execution, including complete three-phase lifecycle (handshake → gameplay → conclusion).

3. Reproducible Execution

Seeding makes game-level randomness reproducible (player ordering, RNG) and guarantees recording/replay parity. However, LLM outputs are not guaranteed to be deterministic across runs, even with a fixed seed.

from agentdeck import AgentDeck, AgentDeckConfig, MockPlayer

config = AgentDeckConfig(seed=42)
players = [
    MockPlayer(name="Alice", actions=["ATTACK", "POTION"]),
    MockPlayer(name="Bob", actions=["POTION", "ATTACK"]),
]

with AgentDeck(game=game, session=config) as deck:
    results = deck.play(players=players, matches=10)

4. Three-Phase Player Lifecycle

Players go through structured interaction phases:

1. Handshake (Mandatory): Player acknowledges rules and format
2. Turn (Gameplay): Player makes decisions each turn
3. Conclusion (Optional): Player reflects on match outcome

This provides rich canonical data for downstream behavioral analysis.

---

📚 Documentation

• Documentation Index - Main docs entry point
• CONTRIBUTING.md - Workflow, local setup, tests
• Specs - Specification index (source of truth)
• Examples - Runnable examples and tutorials
• Security Policy - Vulnerability reporting process

---

🎯 Design Principles

1. Spec-Driven: Every component has a rigorous specification
2. Observable: Every decision is captured and analyzable
3. Reproducible: Everything we control is reproducible (seeding + recordings + replay parity)
4. Composable: Mix and match components freely
5. Execution Truth: Keep recorded facts, deterministic derivations, and authored interpretation explicitly separate

---

Spec-Driven and AI-First by Design

AgentDeck is human-led and AI-written: a codebase built with AI agents, designed for humans and AI agents, and validated through tests, replayable execution, historical artifacts, and blind QA rounds performed by autonomous agents.

Specs are the source of truth. They define intent, contracts, boundaries, and expected behavior. Code, tests, docs, and examples derive from that specification layer and are validated through execution.

AgentDeck is therefore designed to be legible to both humans and AI agents, treating AI agents as first-class users, contributors, evaluators, and execution operators.

---

📝 License

MIT License (see LICENSE).

---

Built for people and agents who need execution they can inspect.