skilltest-sdk

The Python SDK for the skilltest CLI. A thin, typed wrapper and nothing else: it runs the CLI as a subprocess and parses the stable --format json contract into Pydantic models. Test-framework integrations build on it — use skilltest-pytest if you want pytest collection; use this package directly from any other Python code.

from skilltest_sdk import run_skill, validate_skill, describe_failures, assistant_text

report = run_skill("cases/greet.yaml")
assert report.passed, describe_failures(report)
# Mix in deterministic checks on the transcript:
assert "Dr. Smith" in assistant_text(report.runs[0].transcript)

result = validate_skill("skills/greeter")
assert result.valid

Tool events

Each assistant turn carries the normalized tool events the skill took (shell commands, file edits, tool uses), lifted from oneharness's --events. Assert on what the skill did with tool_calls (the tool_call events across a transcript, in order); each ToolEvent has kind, name, input, output, index:

from skilltest_sdk import run_skill, tool_calls

report = run_skill("cases/edit.skilltest.yaml")
calls = tool_calls(report.runs[0].transcript)
assert any("git commit" in str(c.input) for c in calls)
assert not any("rm -rf" in str(c.input) for c in calls)   # never destructive

Streaming (opt-in)

stream_skill returns a SkillStream you iterate with async for to receive each event live, and break to short-circuit — closing the stream tears the harness down, so a bad turn is cut off instead of paid for in full. .report holds the final report once the stream runs to completion:

from skilltest_sdk import stream_skill

async def guard():
    stream = stream_skill("cases/edit.skilltest.yaml")
    async for ev in stream:  # ev: StreamEvent — .case/.platform/.model/.turn/.event
        if ev.event.name == "bash" and "rm -rf" in str(ev.event.input):
            break
    return stream.report

The skilltest binary is resolved from the bin= argument, the SKILLTEST_BIN env var, or PATH; a provider override comes from provider= or SKILLTEST_PROVIDER. A failing eval is reported (report.passed is false), not raised; bad input raises SkilltestUsageError (CLI exit 2) and provider problems raise SkilltestProviderError (exit 3).

The Pydantic models are generated from schemas/report.schema.json / schemas/validation.schema.json — themselves generated from the CLI's own types — via just gen-contract, and a drift gate in CI fails if anything is stale, so the models cannot diverge from the binary.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

skilltest_sdk-0.5.0.tar.gz (52.0 kB view details)

Uploaded Jul 6, 2026 Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

skilltest_sdk-0.5.0-py3-none-manylinux_2_17_x86_64.whl (1.8 MB view details)

Uploaded Jul 6, 2026 Python 3manylinux: glibc 2.17+ x86-64

skilltest_sdk-0.5.0-py3-none-manylinux_2_17_aarch64.whl (1.7 MB view details)

Uploaded Jul 6, 2026 Python 3manylinux: glibc 2.17+ ARM64

skilltest_sdk-0.5.0-py3-none-macosx_11_0_arm64.whl (1.6 MB view details)

Uploaded Jul 6, 2026 Python 3macOS 11.0+ ARM64

skilltest_sdk-0.5.0-py3-none-macosx_10_12_x86_64.whl (1.7 MB view details)

Uploaded Jul 6, 2026 Python 3macOS 10.12+ x86-64

skilltest_sdk-0.5.0-py3-none-any.whl (20.3 kB view details)

Uploaded Jul 6, 2026 Python 3

File details

Details for the file skilltest_sdk-0.5.0.tar.gz.

File metadata

Download URL: skilltest_sdk-0.5.0.tar.gz
Upload date: Jul 6, 2026
Size: 52.0 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.11.27 {"installer":{"name":"uv","version":"0.11.27","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for skilltest_sdk-0.5.0.tar.gz
Algorithm	Hash digest
SHA256	`0c6dda3781fa747396f7ef1f9d96f11064061091bf76919bc7fa389f46973294`
MD5	`ea5c8a78620f8782a867b25ce045d09d`
BLAKE2b-256	`91314dc1341674075414adf4702ad6cfd56976b16d73463729f9eda786a5ebff`

See more details on using hashes here.

File details

Details for the file skilltest_sdk-0.5.0-py3-none-manylinux_2_17_x86_64.whl.

File metadata

Download URL: skilltest_sdk-0.5.0-py3-none-manylinux_2_17_x86_64.whl
Upload date: Jul 6, 2026
Size: 1.8 MB
Tags: Python 3, manylinux: glibc 2.17+ x86-64
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.11.27 {"installer":{"name":"uv","version":"0.11.27","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for skilltest_sdk-0.5.0-py3-none-manylinux_2_17_x86_64.whl
Algorithm	Hash digest
SHA256	`f4f3904f2dd281fd2859df4ed6a2daef600810eec1517c2f4c0afc56216bbffc`
MD5	`1b001049a447576c41f6163631b7a87e`
BLAKE2b-256	`df2c961d9b20b360df92a1ed9784d50c7bbf7e30c684598dd47cb9794fb4ff5c`

See more details on using hashes here.

File details

Details for the file skilltest_sdk-0.5.0-py3-none-manylinux_2_17_aarch64.whl.

File metadata

Download URL: skilltest_sdk-0.5.0-py3-none-manylinux_2_17_aarch64.whl
Upload date: Jul 6, 2026
Size: 1.7 MB
Tags: Python 3, manylinux: glibc 2.17+ ARM64
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.11.27 {"installer":{"name":"uv","version":"0.11.27","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for skilltest_sdk-0.5.0-py3-none-manylinux_2_17_aarch64.whl
Algorithm	Hash digest
SHA256	`1975c0697a703e780840e629f47f29736d740ee5c2ac95a3a06a852645fd0d18`
MD5	`84a5a6cb94f8342ab9d4c594732677b8`
BLAKE2b-256	`251f2194f5a58c8a142561b94fdf3cd82cbb96db4d10b15337e44feb9d8dde1b`

See more details on using hashes here.

File details

Details for the file skilltest_sdk-0.5.0-py3-none-macosx_11_0_arm64.whl.

File metadata

Download URL: skilltest_sdk-0.5.0-py3-none-macosx_11_0_arm64.whl
Upload date: Jul 6, 2026
Size: 1.6 MB
Tags: Python 3, macOS 11.0+ ARM64
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.11.27 {"installer":{"name":"uv","version":"0.11.27","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for skilltest_sdk-0.5.0-py3-none-macosx_11_0_arm64.whl
Algorithm	Hash digest
SHA256	`df17db647f95a9c8037c604da522c713811dfcf9e57e3c5ae1f816afae87442a`
MD5	`229c052b765e2aca4469e20da08123f5`
BLAKE2b-256	`751936f97dad153cd42d10e112bce625c75ad6fe53520c3f2d0ece66bd5ba945`

See more details on using hashes here.

File details

Details for the file skilltest_sdk-0.5.0-py3-none-macosx_10_12_x86_64.whl.

File metadata

Download URL: skilltest_sdk-0.5.0-py3-none-macosx_10_12_x86_64.whl
Upload date: Jul 6, 2026
Size: 1.7 MB
Tags: Python 3, macOS 10.12+ x86-64
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.11.27 {"installer":{"name":"uv","version":"0.11.27","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for skilltest_sdk-0.5.0-py3-none-macosx_10_12_x86_64.whl
Algorithm	Hash digest
SHA256	`967aae513cd2ca24bbffcffea140909fe13aa99f3a31a313f1729393b5402db4`
MD5	`e42c1fc23a8ccdf075713f587b878739`
BLAKE2b-256	`560ce7f04d3a6252e54d0730f0718ea23596a4c9e7973d4ef0d7033b8c3d0417`

See more details on using hashes here.

File details

Details for the file skilltest_sdk-0.5.0-py3-none-any.whl.

File metadata

Download URL: skilltest_sdk-0.5.0-py3-none-any.whl
Upload date: Jul 6, 2026
Size: 20.3 kB
Tags: Python 3
Uploaded using Trusted Publishing? No
Uploaded via: uv/0.11.27 {"installer":{"name":"uv","version":"0.11.27","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

File hashes

Hashes for skilltest_sdk-0.5.0-py3-none-any.whl
Algorithm	Hash digest
SHA256	`a9fd18b1d2dd6296f5da89f43dcebce11f6bb793255e374e309356e36a7f02a1`
MD5	`1d5684c496184d130b7215f34f46e5db`
BLAKE2b-256	`8e31c942903ad3224e0d4f3fbf7c34dd76ae18740c0d3f4f61b581726cc4a388`

See more details on using hashes here.

Release history Release notifications | RSS feed

0.10.0

Jul 7, 2026

0.9.0

Jul 7, 2026

0.8.0

Jul 7, 2026

0.7.0

Jul 7, 2026

0.6.0

Jul 7, 2026

This release

0.5.0

Jul 6, 2026

0.4.0

Jul 6, 2026

0.3.0

Jun 12, 2026

0.2.2

Jun 12, 2026

0.2.1

Jun 12, 2026

0.1.1

Jun 12, 2026

0.1.0

Jun 12, 2026

skilltest-sdk 0.5.0

skilltest-sdk

Tool events

Streaming (opt-in)

Key dates

1 maintainer

Credits

License expression

Requires

Download files

Source Distribution

Built Distributions

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

File details

File metadata

File hashes

Release history Release notifications | RSS feed