KUMA

KUMA Python SDK
Knowledge-grounded Universal Measurement for Agents

English  |  Chinese overview

KUMA is the public Python SDK for testing Agent behavior through a strict Run protocol and bounded Evidence capture. Official services are reached only through public HTTPS; the SDK does not run Agents, execute models, or expose private evaluation logic.

Core capabilities

• Run repeatable Case-to-Judgment evaluations through one framework-neutral Python workflow.
• Use KUMA's official services, plug in custom Case and Judge Providers, or keep the workflow fully local.
• Capture bounded, structured Evidence from step results, file changes, explicit logs, and optional OpenTelemetry traces.
• Select the testing method with a versioned Strategy Group, and describe the Agent and its operating context with an Agent Profile.

Install

Install from PyPI. Requires Python 3.10 or newer; Git is not required:

python -m pip install --upgrade kuma-defuzex

Run this in the Python environment used by your Agent, then restart the Agent. The package name is kuma-defuzex; Python imports and the CLI remain kuma. For optional OTel dependencies and Trace setup, see the Runtime Trace guide.

Versions and update reminders

This documentation describes KUMA SDK 0.3.0. Confirm the installed version with python -c "import kuma; print(kuma.__version__)". The observation and correlation APIs below require 0.3.0 or newer. Package availability is determined by PyPI and the matching official Release, not a source checkout alone. Run kuma updates check (or Python kuma.check_for_updates()) to check stable Release tags: patch-only updates are optional, higher major/minor versions require an upgrade reminder, but never block work or install automatically. Official requests check in the background; import/help/local/custom do not. Set KUMA_DISABLE_UPDATE_CHECK=1 to disable all checks. Offline failures are safe; results are cached for 24 hours per process. Old 0.1.0 users must upgrade manually once to gain reminders. Full policy and limits.

Quick start

Run the deterministic local check without an account, API key, Docker, or network:

kuma quickstart

Local observation

Use local observation to inspect an already instrumented Agent without an account, Case, Judge or automatic upload. Try the offline framework examples from this source checkout. Cloud storage requires a separate explicit authenticated call; Run correlation distinguishes execution, capture and evaluation instead of treating them as the same result.

Local observation needs no KUMA service. Cloud storage and official Run correlation require a service implementing the corresponding public contracts, with the required permissions and capabilities enabled. Installing the SDK does not activate server features; unavailable service features fail explicitly.

Full-stack user-flow example

Follow the full-stack user-flow guide to run KUMA with mini-SWE-agent in Docker. This path calls external services and may use service credit and model budget.

Detailed documentation

Save and reuse a Case

Runtime Trace and optional file diffs

SDK guide · Strategy Groups · Agent tool capabilities · Python API reference · Agent Profile migration · Runtime Evidence contract

Project

Security · Contributing · Apache License 2.0