Skip to main content

KUMA geometric wordmark banner

KUMA

KUMA Python SDK
Knowledge-grounded Universal Measurement for Agents

English  |  Simplified Chinese

KUMA is a framework-neutral Python SDK for evaluating AI Agents. It delivers a test Case as step-by-step Inputs, captures bounded Evidence, and produces a Judgment through official or custom Providers. KUMA does not run your Agent or expose private evaluation logic.

What you can do

  • Run repeatable Case-to-Judgment evaluations.
  • Use official services or custom Case and Judge Providers, including fully local workflows.
  • Capture structured Evidence from results, file changes, logs, and optional OpenTelemetry traces.
  • Select a versioned Strategy Group and describe the Agent with an Agent Profile.

Install

Version 0.3.8 adds opt-in high-difficulty generation, explicit case_request_id recovery, privacy classification fixes and optional Judge input budget discovery. Server features depend on service support and account permissions. See release notes.

Requires Python 3.10 or newer:

python -m pip install --upgrade kuma-defuzex

The package is kuma-defuzex; Python imports and the CLI use kuma.

Quick start

Run a deterministic local check without an account, API key, Docker, or network:

kuma quickstart

Successful output starts with:

Local check: PASS
Score: 100/100
Reason: Output exactly matched the published rule.

This checks the bundled local example, not your Agent or the hosted Judge. To evaluate your Agent, follow the SDK guide.

For official Case generation, create_run(..., difficulty="D1") selects challenge count and intensity: D0 injects zero problems; D1 injects one obvious, low-intensity problem; D2 injects two subtler or composed problems requiring stronger recognition, recovery and verification. Necessary inputs and solvability must be preserved. max_steps remains the same upper bound; D2 does not add a step or change Judge severity. No measured failure rate is promised; the service builds the challenge.

File tracking stays within repo_path and skips descendant junctions or other reparse points, reporting incomplete capture. An explicitly configured external storage_path remains supported and does not expand the file tracking boundary.

Next steps

Documentation

SDK guide · Python API reference · Versions and releases · Detailed Judge assessment

Project

Security · Contributing · Apache License 2.0

Metadata

Release files for kuma-defuzex 0.3.9

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for kuma-defuzex 0.3.9
File Size Uploaded Provenance Actions
kuma_defuzex-0.3.9.tar.gz 309.2 kB With provenance Details for kuma_defuzex-0.3.9.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for kuma-defuzex 0.3.9
File Interpreter ABI Platform Actions
kuma_defuzex-0.3.9-py3-none-any.whl Python 3 none any Details for kuma_defuzex-0.3.9-py3-none-any.whl

Total release size: 671.8 kB

Release files / kuma_defuzex-0.3.9.tar.gz

Download URL kuma_defuzex-0.3.9.tar.gz
Size 309.2 kB
Tags Source
SHA-256 checksum
How to use checksums
74f7111f81346a19b3a1bc5150b43dc8c4f4643de3d234eb672960aa543ad494
BLAKE2b-256 checksum
How to use checksums
110cb76e70eac0c8510cb469a8bc831d959876742de8784106cd59c464012fb2
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 28, 2026.

Transparency log

Release files / kuma_defuzex-0.3.9-py3-none-any.whl

Download URL kuma_defuzex-0.3.9-py3-none-any.whl
Size 362.6 kB
Tags Python 3
SHA-256 checksum
How to use checksums
8fd280f8a76f9ab1fbd301cd07f7e32985d53c8a0f1176f3ba3795b32a3d6947
BLAKE2b-256 checksum
How to use checksums
4ada731aeafd9029ccaba0796b1948f54b751b0a10c1b0445ab687f6dad2d5bf
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 28, 2026.

Transparency log

Security for kuma-defuzex 0.3.9

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for kuma_defuzex-0.3.9.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 28, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page