This release is a pre-release and may not be stable for production use.
agentic-dataset-conformance
Fifteen normative assertions about governed datasets, as language-neutral executable vectors, checkable against any implementation without access to its internals.
pip install --pre agentic-dataset-conformance # 0.1.0rc1 is a pre-release
agentic-dataset-conformance run # against the built-in subject
Not yet on PyPI during the release candidate; install from a clone until it is.
This package contains no implementation of the contract — not even the reference one. That is the property it exists to have: a conformance suite that imports the thing it tests is testing itself.
What the assertions rule out
AD-001 descriptor_valid |
a dataset in admission without a well-formed contract |
AD-002 capability_registered |
an executable action with no capability metadata behind it |
AD-003 grant_required_for_execution |
execution reachable without an authorization artifact |
AD-004 refusal_has_no_grant |
a refusal that still mints authority |
AD-005 indeterminate_has_no_grant |
unknown authority becoming permission |
AD-006 unknown_capability_denied |
default-allow on an unregistered tool |
AD-007 authorization_scope_preserved |
scope widening between admission and execution |
AD-008 cache_is_policy_scoped |
a cached answer crossing an authorization boundary |
AD-009 provenance_complete |
a result that cannot be traced to what produced it |
AD-010 refusal_recorded |
a refusal that leaves no evidence |
AD-011 dataset_revision_recorded |
evidence that cannot identify which data was used |
AD-012 policy_version_recorded |
evidence that cannot identify which rules applied |
AD-013 remote_execution_preserves_scope |
MCP or A2A delegation as an escalation path |
AD-014 agent_handoff_preserves_scope |
sub-agent handoff as an escalation path |
AD-015 prohibited_execution_rate_zero |
any prohibited action executing at all, ever |
Every one is checked structurally. Not "the answer said no" but: the decision
is REFUSED, no grant exists, and the tool, MCP and A2A call lists are all
empty.
Testing your implementation
Implement four methods — load_world, capabilities, step, reset — and
return an Observation from each step. interface.py is the whole contract
and verbs.md is the control-verb vocabulary.
agentic-dataset-conformance run --subject mypackage.conformance:make_subject
--subject takes module:attribute, where the attribute is a subject, or a
callable returning one or several. agentic_dataset_conformance.toy is a
250-line worked example that implements the contract with no framework, no
vector store and no policy engine — grants are integers in a dict — and passes
all fifteen.
Checking that the suite would notice
agentic-dataset-conformance run --matrix
Seventeen deliberately broken variants, each removing exactly one guarantee. Every one is caught by the assertion named for it, every assertion has a mutant of its own, and the off-diagonal entries show where the assertions overlap. A suite that cannot fail is decoration.
The vectors are CC0
agentic-dataset-conformance vectors --export ./vectors
The worlds and vectors under data/ are dedicated to the public domain:
no attribution required, no conditions. Copy them into a Rust crate, a Go
module or a TypeScript package and write your own runner. Python is one runner,
not the specification.
The software around them is Apache-2.0, so the distribution as a whole is
Apache-2.0 AND CC0-1.0.
What conformance does and does not establish
capabilities() is the subject's own report of itself. A subject that
under-reports passes AD-002 while hiding a tool. Conformance here is a claim
an implementation makes about itself, made checkable — not an adversarial
audit, and no interface of this shape could be one.
Links
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file agentic_dataset_conformance-0.1.0rc1.tar.gz.
File metadata
- Download URL: agentic_dataset_conformance-0.1.0rc1.tar.gz
- Upload date:
- Size: 33.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.12.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2d03ac48c1c4894a8e41f9556514410cb089aa12c6a6ce9b6566818584298312
|
|
| MD5 |
e39892c8188c10a1ea635aea25976a84
|
|
| BLAKE2b-256 |
696026bc10567edd19a63fb5c27aaaf5ca3379a349889116b501d7a7cc1d9d09
|
File details
Details for the file agentic_dataset_conformance-0.1.0rc1-py3-none-any.whl.
File metadata
- Download URL: agentic_dataset_conformance-0.1.0rc1-py3-none-any.whl
- Upload date:
- Size: 42.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.12.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3830b36124106a9be6243a4a13e70355e0d6eef5bda7e0c0d1a7dcfe3b2d05af
|
|
| MD5 |
510b5e0cc3b19e993be6396c6332a580
|
|
| BLAKE2b-256 |
5a8c4ed3abd3164ee7df3ff1bdba1f2e80fc37e55dffa52560d50b56aa73aeb0
|
