AgentExperience

Give agents memory for what actually worked.
Capture runtime evidence, validate reusable strategy deltas, and apply only experience that earns its token budget.

Quick start · Packages · Experiment · Tutorial · API guide · Security · Contributing

🧠 Evidence-first  ·  🧩 Framework-neutral  ·  🔐 Safe by default  ·  📦 Portable

Most agent memory systems store conversations or long summaries. AgentExperience stores a more conservative object: a small, baseline-relative strategy delta backed by independent evidence. It records how an agent ran, separates success from mere completion, measures the cost and benefit of reuse, and can quarantine experience that regresses.

runtime events → verified outcomes → candidate delta → validation → benefit gate → active reuse
      │                 │                  │                │               │
  observable         auditable         immutable       token-aware     reversible

✨ Why AgentExperience?

🚀 Quick start

Install the framework-independent core:

pip install agent-experience

Or add only the integrations you use:

pip install "agent-experience[langchain,langgraph,mcp]"

Create one runtime, specify the storage path once, and decorate the boundaries you want observed:

from agent_experience import agent_experience

experience = agent_experience("./experience-data")

@experience.tool
def get_weather(city: str) -> dict[str, object]:
    return {"city": city, "temperature_c": 22, "fresh": True}

@experience.run(verify=lambda result: bool(result["fresh"]))
def weather_agent(city: str) -> dict[str, object]:
    return get_weather(city)

print(weather_agent("Berlin"))

That is the entire integration. AgentExperience automatically owns storage, generates stable run and tool identities, propagates causation, sanitizes values, records timing and failures, and queues verified runs for candidate consolidation. There is no Repository, registry, name, contract ID, producer, rule path or CandidateService to configure.

If you only need observation, omit the verifier:

@experience.run
def agent(task: str):
    return do_work(task)

The run is stored, but it cannot create an activatable experience without quality evidence. A standalone @experience.tool call also receives an automatic run context. Call experience.flush() only when a test or short-lived process must wait for background consolidation; normal applications are flushed when the runtime closes.

Harness / Loop protocol (v0.2 preview)

Custom Harnesses can own their Loop while reporting structured evidence through an explicit, concurrency-safe run session:

from agent_experience import HarnessState, Outcome, RunOutcome, agent_experience

experience = agent_experience("./experience-data")
run = experience.start("debug the failing test", harness="custom-loop")

advice = run.select(HarnessState(task="debug the failing test"))
# The Harness decides whether and how to use the advice, then executes its own Loop.

run.complete(RunOutcome(Outcome.SUCCESS, result={"fixed": True}))

For an ACTIVE prompt delta, provide harness_policy={"task_type": ...} and an explicit budget containing max_context_tokens, base_input_tokens, and reserved_output_tokens. The selected bounded rules are returned in SelectionResult.steps; they are never injected automatically, and the Harness records adoption with run.feedback(..., experience_id=..., accepted=True).

AgentExperience does not own planning, tools, retries or stopping conditions. select() may return an explicit ABSTAINED result when no active experience is applicable. Use run.observe() for normalized runtime evidence and run.feedback() for intermediate outcome or adoption signals. The existing decorators remain supported and use the same runtime-owned event store.

See examples/custom_loop_protocol.py for a complete minimal Harness. Integrations can call run_protocol_conformance() in their test suite to verify that each run has one start, one terminal lifecycle event, valid correlation and integrity-checked payloads. Delegated work uses run.start_child(...), preserving explicit parent/child lineage. Closing the Runtime cancels and audits any explicit sessions that were left active.

Adapter capability declarations identify protocol version and whether an integration supports explicit runs, selection, feedback, delegation and async execution. Conformance requirements can therefore distinguish a verified capability from UNSUPPORTED or INCONCLUSIVE. Async Harnesses use the same session contract directly: event persistence is intentionally local and ordered, and AgentExperience does not hide synchronous persistence behind unmanaged worker threads.

examples/codex_like_loop.py demonstrates a bounded Observe → Act → Verify → Retry Loop. The Loop owns retries and stopping; AgentExperience only records evidence, returns advice and receives feedback.

LangGraph can bind normalized graph events to the same explicit run:

run = experience.start("execute graph", harness="langgraph")
bridge = experience.langgraph(run=run)
# Consume public LangGraph stream events through bridge.consume(...)
run.complete(RunOutcome(Outcome.SUCCESS))

The bound EventSink rejects attempts to write another run ID or pass storage-specific options. PROTOCOL_API_VERSION identifies the frozen 0.2 public contract; compatibility tests snapshot the exported models, dataclass fields and lifecycle method parameters.

The reproducible local effectiveness check is python tools/protocol_effects.py; the three-version New York example is python examples/new_york_version_comparison.py. See the v0.2 effect report for the current latency, concurrency, abstention and negative-transfer-safety baseline, and the generated comparison report for the end-to-end Baseline/v0.1/v0.2 travel experiment.

🛠️ Observe an application Skill

A Skill is simply a reusable callable capability. Decorate its public entry point exactly like a Tool; the callable's module, signature and code fingerprint become its automatic identity and version. No Skill name, storage path or experience key is required.

from agent_experience import agent_experience

experience = agent_experience("./experience-data")

@experience.tool
def report_skill(rows: list[dict[str, object]]) -> dict[str, object]:
    report = build_report(rows)  # your existing Skill implementation
    return {"report": report, "passed_checks": validate_report(report)}

@experience.run(verify=lambda result: bool(result["passed_checks"]))
def analyst_agent(rows: list[dict[str, object]]) -> dict[str, object]:
    return report_skill(rows)

AgentExperience observes the Skill boundary, sanitized inputs and outputs, latency, failures and task-level quality evidence. It does not store or execute the Skill's source code, and it does not mistake a successful function return for a correct result.

Framework integrations reuse the same runtime and storage:

# LangChain: pass this once when constructing the agent.
agent = create_agent(model, tools, middleware=[experience.langchain()])

# LangGraph: feed typed stream events into the runtime-owned bridge.
graph_events = experience.langgraph()

# MCP: wrap an existing ClientSession; no second path or repository.
session = experience.mcp(session, trust_domain="company-internal")

📦 Portable experience packages

Share validated experience as a data-only .exp package, then mount it with one line:

report = experience.mount("./team-patterns.exp")
print(report)

Or declare packages once when creating the Runtime; mounting is deferred until decorated Tools and Skills have registered their capabilities:

experience = agent_experience(
    "./experience-data",
    experiences=["./team-patterns.exp"],
)

The Runtime verifies checksums and optional Ed25519 signatures, checks Python/framework/capability compatibility, creates explainable automatic bindings, deduplicates content and returns a complete MountReport. Imported experience always starts in quarantine. A checksum, trusted publisher or successful function return can never activate external experience by itself.

Export only validated or active experience:

from agent_experience import PackageSigner

signer = PackageSigner.load("publisher.private-key")
experience.export(
    "team-patterns.exp",
    name="team-patterns",
    version="1.0.0",
    publisher="my-team",
    signer=signer,
)

For a locally trusted publisher, add its Ed25519 public key once through experience.trust or the CLI. Signature trust proves package origin; caller-controlled local validation still proves whether the experience works in the receiving environment.

👁️ What is observed?

Applications decide what constitutes success through deterministic evaluators or custom adapters. The core provides extension protocols for FeatureExtractor, BaselineResolver, and TokenEstimator; it does not contain domain keyword tables or benchmark-specific thresholds.

⚖️ Experience that must earn its keep

from agent_experience import BreakEvenPolicy

policy = BreakEvenPolicy(
    minimum_measurements=3,
    minimum_holdout_samples=20,
    maximum_input_token_increase=128,
    policy_id="production-break-even",
    policy_version="1",
)

Benefit decisions aggregate measurements for the same immutable revision, weighted by sample count. Rejection reasons are machine-readable: insufficient evidence, quality or success-rate regression, negative net benefit, token-budget overflow, or output truncation.

🧩 Integrations

See the tutorial for setup and lifecycle examples and the API guide for the supported public surface.

🔬 Transparent DeepSeek experiment

The paid end-to-end demo prints every model call, selected rule, token/latency measurement, score, benefit decision and lifecycle transition. It uses travel only as an application-level benchmark; no travel logic exists in the core package.

Copy-Item examples\deepseek_demo_local.example.py examples\deepseek_demo_local.py
# Edit only the ignored deepseek_demo_local.py, then run:
python examples\deepseek_experience_demo.py

The demo performs seven model calls. Generated repositories and reports are ignored by Git.

Real output comparison: a two-day New York itinerary

This is a real controlled A/B run made on 2026-08-14 with deepseek-v4-pro, temperature 0.2 and the same request in both calls. The right-hand call received seven generic output constraints mined deterministically from three successful travel runs; it did not receive a saved New York answer. The complete content of both responses is shown below. The candidate was injected for measurement and remained quarantined rather than being silently activated.

The primary outcome is answer quality, not minimum token usage. Extra tokens are worthwhile when they buy useful completeness, feasibility or clarity; token and latency deltas remain cost signals. This run demonstrates richer structure, but not an automatic quality win. AgentExperience retains the measurements and quarantines experience that does not clear the configured benefit policy.

⌨️ CLI

agent-exp verify ./experience-repo
agent-exp inspect ./experience-repo
agent-exp extract ./experience-repo --minimum-confidence 0.8
agent-exp candidates ./experience-repo
agent-exp benefits ./experience-repo
agent-exp export ./experience-repo shared.exp
agent-exp import ./other-repo shared.exp
agent-exp package inspect ./experience-repo shared.exp
agent-exp package mount ./experience-repo shared.exp
agent-exp package list ./experience-repo
agent-exp package unmount ./experience-repo team-patterns
agent-exp trust add ./experience-repo publisher-public.pem

The original export/import commands remain temporarily available for legacy v1 packages. New code should use the package commands. .exp files are data packages, not trusted executable programs.

🔐 Security model

• inputs and outputs are sanitized before observation;
• raw secrets should never be stored as experience;
• retrieved advice is an untrusted reference and cannot override system or permission policy;
• replay is disabled unless the revision, tool registry, approval policy and verifier all allow it;
• remote MCP resources and prompts are represented by identity/hash where possible;
• imported experience starts in QUARANTINED.

Please report vulnerabilities according to SECURITY.md, not in a public issue.

🧭 Project status

AgentExperience is pre-alpha. Public APIs and persistent schemas may change before 1.0. The built-in backend is local, single-process and single-writer; it is not presented as a distributed event log or multi-tenant service. Review the API guide, security policy and changelog before production adoption.

⭐ Star History

🧰 Development

python -m venv .venv
.venv/Scripts/python -m pip install -e ".[dev]"  # Windows
python -m ruff check src tests examples setup.py
python -m mypy src
python -m pytest -q
python -m build
python -m twine check dist/*

Contributions are welcome. Read CONTRIBUTING.md and the Code of Conduct first.

📄 License

Apache License 2.0. See LICENSE.