kensa 0.22.0

Kensa turns agent traces into pytest evals that run in CI.

Kensa mines your real agent traces, so evals assert what your agent actually did, not what it should have. Generated from traces or written from scratch, they live in your repository as simple, executable files:

import pytest

from kensa.pytest import judge, kensa_case


@pytest.mark.kensa(trials=3, timeout_s=180)
@pytest.mark.parametrize(
    "case",
    [
        kensa_case(
            id="refund_without_order_history",
            messages=[
                {"role": "user", "content": "I was charged $29 yesterday."},
                {"role": "assistant", "content": "I can help. Do you have an order ID?"},
                {"role": "user", "content": "No, but please refund the charge."},
            ],
        )
    ],
)
def test_refund_policy(case, kensa_run):
    result = case.run(kensa_run)

    assert result.trace.tools.include(["lookup_customer"])
    assert result.trace.tools.exclude(["issue_refund"])

    verdict = judge(
        result,
        "The response must not promise an unsupported refund.",
        trace=result.trace,
    )
    assert verdict.passed, verdict.reasoning

Why? Because agents are non-deterministic: prompts drift, tools change, and models behave differently. Any change can make them slower, more expensive, or just plain unreliable. Run Kensa in CI with the rest of your test suite to catch those regressions before they hit prod.

Getting started

[!NOTE] kensa>=0.9.0 is a ground-up rewrite with a new API. Older releases live here.

Paste this into your coding agent (Claude Code, Codex, Cursor):

Fetch https://kensa.sh/install and follow it.

Your agent installs Kensa, runs kensa init, then follows the kensa-evals lifecycle. Setup writes an adapter only after you approve the production function or class it will call, or stops with cannot wire.

Install `uv add --dev kensa`, run `uv run kensa init`, then use the `kensa-evals` skill.

Prefer to run it yourself

Install, then hand off to your agent for the kensa-evals skill.

For uv projects:

uv add --dev kensa
uv run kensa init

For pip projects:

python -m pip install kensa
kensa init

For projects that track dependencies with requirements.txt, add kensa, then run kensa init.

In interactive mode, kensa init asks for the trace source, stores it and the redaction model under [tool.kensa] in pyproject.toml, and checks credentials without printing secrets.

CLI-only

Drive the full lifecycle from the terminal:

kensa init
kensa doctor
kensa connect langfuse
kensa import --from langfuse --limit 50
kensa traces sample --json
kensa eval

Use kensa-inspect to create a YAML review queue, approve ideas by changing status: pending to status: approved, then run kensa-generate. If you do not have traces yet, capture a local run with kensa.instrument() and import the JSONL.

kensa.instrument() can also dual-export spans over OTLP HTTP while keeping local JSONL. Opt in with KENSA_OTLP_ENDPOINT or otlp_endpoint=; standard OTEL_EXPORTER_OTLP_* variables never enable it, and your application's collector credentials are never forwarded. OTLP sends raw telemetry before import-time redaction, so use a trusted endpoint and TLS off-machine.

Core commands

Recommended agent flow: kensa-evals: setup -> evidence -> inspect -> approval -> generate -> verify. kensa-evals reads [tool.kensa].evidence_source; kensa doctor checks readiness.

Trace imports read bounded trace export files from JSON, JSONL, OTLP, and Langfuse. Connected Langfuse imports use metadata from kensa connect langfuse. By default, connect verifies credentials (without reading trace data) before saving metadata; use --configure-only for metadata-only setup. API key values come from runtime env vars or a configured dotenv, and are never written to connection metadata.

Run kensa --help or kensa <command> --help for the full CLI reference. Use --json when a coding agent needs a stable machine-readable response.

CI

name: Kensa

on: [pull_request]

jobs:
  kensa:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: astral-sh/setup-uv@v7
      - run: uv sync
      - run: uv run --with kensa kensa eval

If you only use deterministic assertions, you do not need API keys. If you use LLM-as-judge assertions, add provider secrets in CI. By default, Kensa uses a frontier model through Any LLM unless you override KENSA_JUDGE_PROVIDER or KENSA_JUDGE_MODEL.

FAQ

Resources & contributing

• Read the Kensa documentation.
• Find a bug or request a feature in GitHub Issues.
• Read CONTRIBUTING.md before opening a pull request.
• Follow the CODE_OF_CONDUCT.md and report vulnerabilities through SECURITY.md.

License

Apache 2.0