ECP Runtime

Reference runtime and CLI for the Evaluation Context Protocol (ECP).

ECP is a vendor-neutral protocol for testing agent outputs, tool calls, and evaluator-visible audit context across frameworks, models, eval platforms, and CI systems.

Install

pip install "ecp-runtime==0.9.0"

Usage

ecp init
ecp validate ecp_eval/manifest.yaml
ecp run --manifest ecp_eval/manifest.yaml --json

Run the flagship example:

ecp run --manifest examples/customer_support_demo/manifest.yaml --report report.html

Useful commands:

ecp validate examples/customer_support_demo/manifest.yaml
ecp conformance --target "python examples/customer_support_demo/agent.py"
ecp doctor

Manifest target values may be either a command for the default stdio transport or an ECP Streamable HTTP endpoint:

target: "http://127.0.0.1:8765/ecp"

Execution Boundaries

A hung agent must never pin a CI job. Two independent limits apply:

ecp run --manifest manifest.yaml --timeout 30 --max-duration 900
  • --timeout (ECP_RPC_TIMEOUT, default 30s) bounds a single RPC.
  • --max-duration (ECP_MAX_DURATION, unset by default) bounds the whole run. This is not redundant: an agent that answers just inside the per-RPC timeout on every step can still run for hours.

When a limit is breached, or the agent crashes or breaks the protocol, the run degrades instead of aborting. The failing step is recorded as failed, the rest of that scenario is marked skipped, and the next scenario starts with a fresh agent. Steps that never produced a result still count as failed checks, so a timeout can never be mistaken for a pass.

Audit Record

Every run produces a structured audit payload, embedded under audit in the JSON report and writable standalone:

ecp run --manifest manifest.yaml --audit-out ecp_audit.json

It records the run id, timestamps, the manifest path and its SHA-256 digest, the resolved target, agent metadata, the configured limits, per-step latency and exit_reason, aggregated token usage, and pass/fail totals. Compare totals.steps_planned against totals.steps_executed to spot a run that degraded rather than one that genuinely passed. The schema is schema/audit.schema.json.

Agents can populate the token figures by returning usage from a step:

Result(public_output="...", usage={"input_tokens": 1204, "output_tokens": 88})

If your manifest includes llm_judge, set an API key and optional judge model:

$env:OPENAI_API_KEY="your_key_here"
$env:ECP_LLM_JUDGE_MODEL="gpt-4o-mini"

Links

Project links

Data verified by PyPI on Aug 20, 2026
Data provided by the project maintainers, verified at the time the release was uploaded to PyPI.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for aniketwattamwar from gravatar.com aniketwattamwar

Credits

PyPI data
Data sourced directly from PyPI's database.

Author: ECP Maintainers

GitHub Statistics

Data verified by PyPI on Aug 20, 2026
The GitHub source repository was provided by the project maintainers and verified by PyPI at the time of upload. Stars, forks, and open issues/PRs are derived from that repository and have not been independently verified.

Requires

Python >=3.9

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ecp_runtime-0.9.0.tar.gz (32.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ecp_runtime-0.9.0-py3-none-any.whl (26.4 kB view details)

Uploaded Python 3

File details

Details for the file ecp_runtime-0.9.0.tar.gz.

File metadata

  • Download URL: ecp_runtime-0.9.0.tar.gz
  • Upload date:
  • Size: 32.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ecp_runtime-0.9.0.tar.gz
Algorithm Hash digest
SHA256 07f99f8eda5a6887cac6a71e1eab1fb12f494910722edcfd4e798c352e17a0f3
MD5 024af2b073dacd6cea6a8826e0d1d1b4
BLAKE2b-256 214d6e3f950b4ff972d5bd24c20fb591c8aa38e6ff504b804b1e3a360f53b7c2

See more details on using hashes here.

Provenance

The following attestation bundles were made for ecp_runtime-0.9.0.tar.gz:

Publisher: release.yml on evaluation-context-protocol/ecp

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file ecp_runtime-0.9.0-py3-none-any.whl.

File metadata

  • Download URL: ecp_runtime-0.9.0-py3-none-any.whl
  • Upload date:
  • Size: 26.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for ecp_runtime-0.9.0-py3-none-any.whl
Algorithm Hash digest
SHA256 57ce9f00f02d54e22ff6431baca13d56eaddb42181c554207618da924fa00a54
MD5 427e54f162ec0a76e0f40e4b2b2df8a4
BLAKE2b-256 6da42f30bb7ca7da54f2ef1bafefe85f6591c9795c8490360a6e9f0b4043a243

See more details on using hashes here.

Provenance

The following attestation bundles were made for ecp_runtime-0.9.0-py3-none-any.whl:

Publisher: release.yml on evaluation-context-protocol/ecp

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page