KUMA

KUMA Python SDK
Knowledge-grounded Universal Measurement for Agents

English  |  简体中文

KUMA is the public Python SDK for testing Agent behavior through a strict Run protocol and bounded Evidence capture. Official services are reached only through public HTTPS; the SDK does not run Agents, execute models, or expose private evaluation logic.

Core capabilities

• Synchronous, framework-neutral Case and Judge workflow.
• Official or custom Providers, including fully local runs.
• Bounded file, log, and optional trace Evidence.
• upload_diff=True with track_files=True sends safe patches; default is hash-only. Trace attaches to configured OTel and uploads only recorded tool content, never invented arguments/results. Setup and source upgrade.
• Default Trace budget: 8 MiB per Run. Agent output: 4 MiB canonical JSON; Runtime Evidence: 5 MiB; complete multipart upload: 8 MiB. Stricter server limits still apply; output is rejected rather than truncated.
• Strategy Groups choose the testing method; an Agent Profile supplies only the Agent, scenario, expected-behavior, and prohibited-boundary context.

Install

Install from PyPI. Requires Python 3.10 or newer; Git is not required:

python -m pip install --upgrade kuma-defuzex

Run this in the Python environment used by your Agent, then restart the Agent. The package name is kuma-defuzex; Python imports and the CLI remain kuma. For optional OTel dependencies and Trace setup, see the Runtime Trace guide.

Versions and update reminders

SDK version: 0.2.5. Publication status is shown in the official Releases. Run kuma updates check (or Python kuma.check_for_updates()) to check stable Release tags: patch-only updates are optional, higher major/minor versions require an upgrade reminder, but never block work or install automatically. Official requests check in the background; import/help/local/custom do not. Set KUMA_DISABLE_UPDATE_CHECK=1 to disable all checks. Offline failures are safe; results are cached for 24 hours per process. Old 0.1.0 users must upgrade manually once to gain reminders. Full policy and limits.

Quick start

Run the deterministic local check without an account, API key, Docker, or network:

kuma quickstart

Full-stack user-flow example

Follow the full-stack user-flow guide to run KUMA with mini-SWE-agent in Docker. This path calls external services and may use service credit and model budget.

Detailed documentation

Save and reuse a Case

Runtime Trace and optional file diffs

English SDK guide · Strategy Groups · Agent tool capabilities · Python API reference · Agent Profile migration · 简体中文 SDK 指南 · 中文 API 参考 · Runtime Evidence contract

Project

Security · Contributing · Apache License 2.0