Skip to main content

Testbench

Kubernetes-native agent evaluation system that executes test datasets via the A2A protocol, scores responses with pluggable metrics (RAGAS by default), and publishes scores via OpenTelemetry.

📖 Documentation: https://docs.agentic-layer.ai/testbench/

Run standalone

For evaluating an agent without deploying into Kubernetes / Testkube:

pip install agentic-layer-testbench
testworkflow config.yaml

See config.example.yaml for the available configuration options.

Development

Prerequisites

  • Python
  • uv
  • Tilt and a local Kubernetes cluster (e.g. kind)
  • Testkube CLI
  • GOOGLE_API_KEY for LLM-as-a-judge evaluation via Gemini

Build and run locally

# Install Python dependencies
uv sync
# Provide the LLM-as-a-judge API key
echo "GOOGLE_API_KEY=<key>" > .env
# Start the local stack (AI gateway, OTLP collector, sample agents, Testkube)
tilt up

Test

uv run poe ruff      # format and lint
uv run poe mypy      # static type checking
uv run poe bandit    # security scanning
uv run poe test      # unit tests
uv run poe check     # all of the above
uv run poe test_e2e  # E2E tests (requires `tilt up`)

E2E defaults target the Tilt environment. Override with E2E_DATASET_URL, E2E_AGENT_URL, E2E_MODEL if needed.

Verify the local deploy

Run the example workflow against the sample weather agent:

kubectl testkube run tw example-workflow --watch

The full walkthrough — defining experiments, configuring metrics, viewing reports — is in the first-workflow how-to.

Contributing

See the Contribution Guide.

Metadata

Release files for agentic-layer-testbench 0.10.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for agentic-layer-testbench 0.10.0
File Size Uploaded Provenance Actions
agentic_layer_testbench-0.10.0.tar.gz 38.2 kB With provenance Details for agentic_layer_testbench-0.10.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for agentic-layer-testbench 0.10.0
File Interpreter ABI Platform Actions
agentic_layer_testbench-0.10.0-py3-none-any.whl Python 3 none any Details for agentic_layer_testbench-0.10.0-py3-none-any.whl

Total release size: 88.2 kB

Release files / agentic_layer_testbench-0.10.0.tar.gz

Download URL agentic_layer_testbench-0.10.0.tar.gz
Size 38.2 kB
Tags Source
SHA-256 checksum
How to use checksums
11ef66fb52e0dc4ad549aa9a6f0a8416e96dbd47867b4db0946cbd600744b867
BLAKE2b-256 checksum
How to use checksums
b0bd92fc3055ae3ec52aaa3282fb82f09fa15cc88f41d4c4d13c6f6edd063bdf
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 12, 2026.

Transparency log

Release files / agentic_layer_testbench-0.10.0-py3-none-any.whl

Download URL agentic_layer_testbench-0.10.0-py3-none-any.whl
Size 50.0 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a4846e751a0033573a4c66b5e0e1311ea105b14a3a2425626449a3abbef2a020
BLAKE2b-256 checksum
How to use checksums
8c19d3cb6c6fc2ce374d9d859b71b6a62ec9a62e1134bca61f24fa33191ee48c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/6.1.0 CPython/3.13.12

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 12, 2026.

Transparency log

Security for agentic-layer-testbench 0.10.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for agentic_layer_testbench-0.10.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Jun 12, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page