vla-robustness-kit
vla-robustness-kit generates deterministic, simulator-free perturbation
diagnostics over episode instructions, metadata, and task-phase lists.
At a Glance
| Job | Exercise a robustness-report pipeline across language, vision metadata, embodiment metadata, and phase-order variants. |
| Built for | VLA evaluation developers and robotics teams prototyping review workflows. |
| Differentiator | Runs without a simulator, GPU, robot, image transformation, or model service. |
| Produces | Markdown pass rates, failure clusters, and follow-up data collection recommendations. |
Install
python -m pip install "vla-robustness-kit==0.1.2"
Verified Quickstart
Run from a source checkout:
vla-robustness-kit run examples/mock_episode_set \
--policy mock \
--out /tmp/vla-robustness-report.md
The bundled fixture emits 22 deterministic perturbation trials and a Markdown diagnostic report.
What Is Actually Perturbed
- language instruction strings;
- lighting and camera-pose metadata;
- control-rate and coordinate-frame metadata;
- task-phase ordering.
The standalone CLI currently accepts only --policy mock. The bundled mock
policy assigns deterministic pass/fail outcomes from variant names and fixture
tags. It does not invoke a VLA model or inspect image pixels.
LeRobot and RLDS helper modules normalize selected metadata fields for callers,
but the CLI reads an episodes.jsonl file or a directory containing that file.
Runtime, Data, and Network Boundary
- The CLI reads local JSONL and writes Markdown locally or to stdout.
- It does not make network requests, run a simulator, use a GPU, access robot hardware, or call a model.
- No redaction is applied. Instructions, metadata values, and episode IDs can appear in downstream trial data, so sanitize inputs before sharing.
- Pass rates from the mock policy validate the diagnostic plumbing only. They are not evidence of policy robustness or robot safety.
Limitations
- The standalone CLI currently supports only the bundled mock policy and metadata-level perturbations. It does not transform images or execute a VLA.
- Reports diagnose the supplied fixture or episode set only. They are not benchmark scores, simulator results, or safety guarantees.
Compatibility
robostudio-engine exposes a direct robostudio probe integration. Its wrapper
can run the bundled mock policy or an explicitly supplied local executable
adapter; that BYO executable path is an engine feature, not a capability of the
standalone vla-robustness-kit CLI.
Publication Status
Verified on 2026-07-13:
- PyPI:
vla-robustness-kit==0.1.2 - GitHub release:
v0.1.2 - Bundled reports are deterministic tutorial diagnostics, not benchmark or deployment evidence.
Next Action
Use the mock policy to validate report ingestion and review, then connect a
separately reviewed executable policy adapter through robostudio-engine for
real model outputs.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file vla_robustness_kit-0.1.2.tar.gz.
File metadata
- Download URL: vla_robustness_kit-0.1.2.tar.gz
- Upload date:
- Size: 9.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3e62c29727b585a36fd51f6adef73dbd3ef33ba1a92bcffee707d7f6876e31ef
|
|
| MD5 |
b66708a758fdbbea7a3edf59ad080bd8
|
|
| BLAKE2b-256 |
a8393f072705f2f77f6c47a3f0fc65f482cc16e5b3910f4659c6307aacd6f61b
|
Provenance
The following attestation bundles were made for vla_robustness_kit-0.1.2.tar.gz:
Publisher:
release-python.yml on auraoneai/vla-robustness-kit
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
vla_robustness_kit-0.1.2.tar.gz -
Subject digest:
3e62c29727b585a36fd51f6adef73dbd3ef33ba1a92bcffee707d7f6876e31ef - Sigstore transparency entry: 2165396710
- Sigstore integration time:
-
Permalink:
auraoneai/vla-robustness-kit@da63f88222708bf5b38ca36ce4e5923ffac1bea2 -
Branch / Tag:
refs/tags/v0.1.2 - Owner: https://github.com/auraoneai
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release-python.yml@da63f88222708bf5b38ca36ce4e5923ffac1bea2 -
Trigger Event:
release
-
Statement type:
File details
Details for the file vla_robustness_kit-0.1.2-py3-none-any.whl.
File metadata
- Download URL: vla_robustness_kit-0.1.2-py3-none-any.whl
- Upload date:
- Size: 10.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
eee113fcd3c89833cd6455407e11f61c4771492fa5c390ca86f6fdeb0c77b324
|
|
| MD5 |
abee6f5c1944a0e19237879e1c1efecc
|
|
| BLAKE2b-256 |
92e1f812b2daf8f71742ec1fc58d88c0ed1ff530d423426c6f1a541a3d9864eb
|
Provenance
The following attestation bundles were made for vla_robustness_kit-0.1.2-py3-none-any.whl:
Publisher:
release-python.yml on auraoneai/vla-robustness-kit
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
vla_robustness_kit-0.1.2-py3-none-any.whl -
Subject digest:
eee113fcd3c89833cd6455407e11f61c4771492fa5c390ca86f6fdeb0c77b324 - Sigstore transparency entry: 2165396724
- Sigstore integration time:
-
Permalink:
auraoneai/vla-robustness-kit@da63f88222708bf5b38ca36ce4e5923ffac1bea2 -
Branch / Tag:
refs/tags/v0.1.2 - Owner: https://github.com/auraoneai
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
release-python.yml@da63f88222708bf5b38ca36ce4e5923ffac1bea2 -
Trigger Event:
release
-
Statement type:
