ConeReplay

Pre-deploy regression testing for multi-agent AI. Record traces from your production agent app, propose a change (new prompt, new tool response, new model), and replay only the events causally affected — everything else is served byte-identically from the recording.

Patent pending (U.S. Provisional Application 64/043,722). See VERSIONING.md for SemVer commitments and docs/trace-format.md for the on-disk contract.

Install

pip install conereplay                   # core + CLI
pip install conereplay[server]           # hosted trace-store server
pip install conereplay[langgraph]        # LangGraph recording adapter
pip install conereplay[providers]        # anthropic / openai oracle backends
pip install conereplay[dev]              # dev tooling (pytest, ruff, mypy, pip-audit)

Requires Python 3.11+. Core has only two runtime deps (networkx, pydantic, pydantic-settings).

CLI quickstart

# Single-trace replay → Markdown report
conereplay replay \
  --trace examples/trace.json \
  --modify examples/modify.json \
  --report report.md

# Corpus-mode regression harness → aggregate report (Markdown or HTML)
conereplay corpus \
  --traces examples/traces \
  --modify examples/corpus-modify.json \
  --report corpus.html \
  --format html

# Emit a Mermaid or Graphviz diagram of a trace's causal DAG
conereplay diagram \
  --trace examples/trace.json \
  --modify examples/modify.json \
  --format mermaid --out cone.mmd

See docs/cli.md for the full reference.

Library quickstart

from conereplay import (
    load_trace, ModificationSpec, selective_replay, compute_divergence,
    render_markdown_report,
)

trace = load_trace("trace.json")
mod = ModificationSpec(target_event_id="e3", substituted_output=b"POLICY: 5 days")
divergent = selective_replay(trace, mod)
report = compute_divergence(trace, divergent)
print(render_markdown_report(trace, divergent, mod, report))

Architecture

Four layers. See ARCHITECTURE.md for the full picture.

Supporting infrastructure:

• config.py — validated runtime config (pydantic-settings, CONEREPLAY_* env)
• flags.py — feature flags (CONEREPLAY_FLAGS_* env)
• logging_setup.py — structured text/JSON logging
• audit.py — append-only JSONL audit trail for compliance
• core/schemas.py — pydantic contract models for trace + modification JSON

Configuration

All configuration is env-driven with the CONEREPLAY_ prefix. Defaults are safe for local use. Sample overrides:

export CONEREPLAY_LOG_LEVEL=debug
export CONEREPLAY_LOG_FORMAT=json
export CONEREPLAY_DATA_DIR=/var/lib/conereplay
export CONEREPLAY_AUDIT_LOG_PATH=/var/log/conereplay/audit.jsonl
export CONEREPLAY_ENABLE_AUDIT=true
export CONEREPLAY_SERVER_DATABASE_URL=postgresql://...

See docs/configuration.md (stub — coming soon) or conereplay/config.py for the full list.

Testing

make install-dev    # install runtime + dev deps (pytest, ruff, mypy)
make test           # run full pytest suite
make test-fast      # fail-fast + failed-first
make test-cov       # with coverage report (htmlcov/)
make lint           # ruff
make typecheck      # mypy

CI (see .github/workflows/) runs tests + lint + typecheck on every push and PR against main. Integration tests that hit real LLM providers are gated behind CONEREPLAY_INTEGRATION=1.

Contributing

See CONTRIBUTING.md.

Security

See SECURITY.md for supply-chain posture and vulnerability disclosure. Patent-pending proprietary code; see LICENSE.

Runbooks

Operational procedures live in runbooks/:

• deployment.md — releases + migrations
• on-call.md — incident triage
• disaster-recovery.md — backups + restore

Documentation

• Handoff
• Backlog
• Architecture
• Local Development
• Testing
• Infrastructure
• Roadmap
• Architecture Decisions

Evidence and sample

• Sanitized end-to-end migration sample
• Reproducible attribution configuration
• Replay benchmark methodology

The sample is synthetic and labels fixture measurements separately from live-provider evidence.