ECP Runtime

Reference runtime and CLI for the Evaluation Context Protocol (ECP).

ECP is a vendor-neutral protocol for testing agent outputs, tool calls, and evaluator-visible audit context across frameworks, models, eval platforms, and CI systems.

Install

pip install "ecp-runtime==0.8.0"

Usage

ecp init
ecp validate ecp_eval/manifest.yaml
ecp run --manifest ecp_eval/manifest.yaml --json

Run the flagship example:

ecp run --manifest examples/customer_support_demo/manifest.yaml --report report.html

Useful commands:

ecp validate examples/customer_support_demo/manifest.yaml
ecp conformance --target "python examples/customer_support_demo/agent.py"
ecp doctor

Manifest target values may be either a command for the default stdio transport or an ECP Streamable HTTP endpoint:

target: "http://127.0.0.1:8765/ecp"

Execution Boundaries

A hung agent must never pin a CI job. Two independent limits apply:

ecp run --manifest manifest.yaml --timeout 30 --max-duration 900
  • --timeout (ECP_RPC_TIMEOUT, default 30s) bounds a single RPC.
  • --max-duration (ECP_MAX_DURATION, unset by default) bounds the whole run. This is not redundant: an agent that answers just inside the per-RPC timeout on every step can still run for hours.

When a limit is breached, or the agent crashes or breaks the protocol, the run degrades instead of aborting. The failing step is recorded as failed, the rest of that scenario is marked skipped, and the next scenario starts with a fresh agent. Steps that never produced a result still count as failed checks, so a timeout can never be mistaken for a pass.

Audit Record

Every run produces a structured audit payload, embedded under audit in the JSON report and writable standalone:

ecp run --manifest manifest.yaml --audit-out ecp_audit.json

It records the run id, timestamps, the manifest path and its SHA-256 digest, the resolved target, agent metadata, the configured limits, per-step latency and exit_reason, aggregated token usage, and pass/fail totals. Compare totals.steps_planned against totals.steps_executed to spot a run that degraded rather than one that genuinely passed. The schema is schema/audit.schema.json.

Agents can populate the token figures by returning usage from a step:

Result(public_output="...", usage={"input_tokens": 1204, "output_tokens": 88})

If your manifest includes llm_judge, set an API key and optional judge model:

$env:OPENAI_API_KEY="your_key_here"
$env:ECP_LLM_JUDGE_MODEL="gpt-4o-mini"

Links

Project links

Data verified by PyPI on Aug 20, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.9.0)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for aniketwattamwar from gravatar.com aniketwattamwar

Credits

PyPI data
Data sourced directly from PyPI's database.

Author: ECP Maintainers

GitHub Statistics

Data verified by PyPI on Aug 20, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

Requires

Python >=3.9

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ecp_runtime-0.8.0.tar.gz (32.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ecp_runtime-0.8.0-py3-none-any.whl (26.4 kB view details)

Uploaded Python 3

File details

Details for the file ecp_runtime-0.8.0.tar.gz.

File metadata

  • Download URL: ecp_runtime-0.8.0.tar.gz
  • Upload date:
  • Size: 32.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ecp_runtime-0.8.0.tar.gz
Algorithm Hash digest
SHA256 9d000623bc941803f7669c75063efef0c08359f0810a39737d04c5b4bc91edb9
MD5 a22212382706596c2715a7e1f9f65fbc
BLAKE2b-256 613c461d6bb06b2dc1a3dab40f7c2283c8b5ecb895ba2ac4afa1ac3e7fa39c95

See more details on using hashes here.

Provenance

The following attestation bundles were made for ecp_runtime-0.8.0.tar.gz:

Publisher: release.yml on evaluation-context-protocol/ecp

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ecp_runtime-0.8.0-py3-none-any.whl.

File metadata

  • Download URL: ecp_runtime-0.8.0-py3-none-any.whl
  • Upload date:
  • Size: 26.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ecp_runtime-0.8.0-py3-none-any.whl
Algorithm Hash digest
SHA256 42c6cbc71b80be57261c519734561552d068fbe7ea66aef58f226ac54b92a702
MD5 9b7c647f54740a9d240e882e31453c6b
BLAKE2b-256 f11f7a00fc79eac58e58c918a82898c9db781408c38445ca22dbd3188671fcae

See more details on using hashes here.

Provenance

The following attestation bundles were made for ecp_runtime-0.8.0-py3-none-any.whl:

Publisher: release.yml on evaluation-context-protocol/ecp

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page