Skip to main content

KUMA geometric wordmark banner

KUMA

KUMA Python SDK
Knowledge-grounded Universal Measurement for Agents

English  |  Simplified Chinese

KUMA is a framework-neutral Python SDK for evaluating AI Agents. It delivers a test Case as step-by-step Inputs, captures bounded Evidence, and produces a Judgment through official or custom Providers. KUMA does not run your Agent or expose private evaluation logic.

What you can do

  • Run repeatable Case-to-Judgment evaluations.
  • Use official services or custom Case and Judge Providers, including fully local workflows.
  • Capture structured Evidence from results, file changes, logs, and optional OpenTelemetry traces.
  • Select a versioned Strategy Group and describe the Agent with an Agent Profile.

Install

Version 0.3.8 adds opt-in high-difficulty generation, explicit case_request_id recovery, privacy classification fixes and optional Judge input budget discovery. Server features depend on service support and account permissions. See release notes.

Requires Python 3.10 or newer:

python -m pip install --upgrade kuma-defuzex

The package is kuma-defuzex; Python imports and the CLI use kuma.

Quick start

Run a deterministic local check without an account, API key, Docker, or network:

kuma quickstart

Successful output starts with:

Local check: PASS
Score: 100/100
Reason: Output exactly matched the published rule.

This checks the bundled local example, not your Agent or the hosted Judge. To evaluate your Agent, follow the SDK guide.

For official Case generation, create_run(..., difficulty="D1") selects challenge count and intensity: D0 injects zero problems; D1 injects one obvious, low-intensity problem; D2 injects two subtler or composed problems requiring stronger recognition, recovery and verification. Necessary inputs and solvability must be preserved. max_steps remains the same upper bound; D2 does not add a step or change Judge severity. No measured failure rate is promised; the service builds the challenge.

File tracking stays within repo_path and skips descendant junctions or other reparse points, reporting incomplete capture. An explicitly configured external storage_path remains supported and does not expand the file tracking boundary.

Next steps

Documentation

SDK guide · Python API reference · Versions and releases · Detailed Judge assessment

Project

Security · Contributing · Apache License 2.0

Metadata

Release files for kuma-defuzex 0.3.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for kuma-defuzex 0.3.8
File Size Uploaded Provenance Actions
kuma_defuzex-0.3.8.tar.gz 307.9 kB With provenance Details for kuma_defuzex-0.3.8.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for kuma-defuzex 0.3.8
File Interpreter ABI Platform Actions
kuma_defuzex-0.3.8-py3-none-any.whl Python 3 none any Details for kuma_defuzex-0.3.8-py3-none-any.whl

Total release size: 669.3 kB

Release files / kuma_defuzex-0.3.8.tar.gz

Download URL kuma_defuzex-0.3.8.tar.gz
Size 307.9 kB
Tags Source
SHA-256 checksum
How to use checksums
89b0e63b536cda1f7cabe4760261b8c57892934bfef1e4dbff983a07f9151c40
BLAKE2b-256 checksum
How to use checksums
0b044104fd00ddfae03dd4a8f49f1dd1c0b123af853c197a044e77ebd58160ab
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 20, 2026.

Transparency log

Release files / kuma_defuzex-0.3.8-py3-none-any.whl

Download URL kuma_defuzex-0.3.8-py3-none-any.whl
Size 361.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
a00426ff4e77b9c2447a19263bf9a95711abedef3ca214465cdd5aa6dc6506b9
BLAKE2b-256 checksum
How to use checksums
e4b165c24725105f390c522ab94eb7f54c79de212f0d9682b890d34dcf4752fa
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 20, 2026.

Transparency log

Security for kuma-defuzex 0.3.8

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for kuma_defuzex-0.3.8.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 20, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page