agentic-dataset-conformance
Fifteen normative assertions about governed datasets, as language-neutral executable vectors, checkable against any implementation without access to its internals.
pip install agentic-dataset-conformance
agentic-dataset-conformance run # against the built-in subject
This package contains no implementation of the contract — not even the reference one. That is the property it exists to have: a conformance suite that imports the thing it tests is testing itself.
What the assertions rule out
AD-001 descriptor_valid |
a dataset in admission without a well-formed contract |
AD-002 capability_registered |
an executable action with no capability metadata behind it |
AD-003 grant_required_for_execution |
execution reachable without an authorization artifact |
AD-004 refusal_has_no_grant |
a refusal that still mints authority |
AD-005 indeterminate_has_no_grant |
unknown authority becoming permission |
AD-006 unknown_capability_denied |
default-allow on an unregistered tool |
AD-007 authorization_scope_preserved |
scope widening between admission and execution |
AD-008 cache_is_policy_scoped |
a cached answer crossing an authorization boundary |
AD-009 provenance_complete |
a result that cannot be traced to what produced it |
AD-010 refusal_recorded |
a refusal that leaves no evidence |
AD-011 dataset_revision_recorded |
evidence that cannot identify which data was used |
AD-012 policy_version_recorded |
evidence that cannot identify which rules applied |
AD-013 remote_execution_preserves_scope |
MCP delegation as an escalation path |
AD-014 agent_handoff_preserves_scope |
sub-agent handoff, over A2A, as an escalation path |
AD-015 prohibited_execution_rate_zero |
any prohibited action executing at all, ever |
Every one is checked structurally. Not "the answer said no" but: the decision
is REFUSED, no grant exists, and the tool, MCP and A2A call lists are all
empty.
Testing your implementation
Implement four methods — load_world, capabilities, step, reset — and
return an Observation from each step. interface.py is the whole contract
and verbs.md is the control-verb vocabulary.
agentic-dataset-conformance run --subject mypackage.conformance:make_subject
--subject takes module:attribute, where the attribute is a subject, or a
callable returning one or several. agentic_dataset_conformance.toy is a
250-line worked example that implements the contract with no framework, no
vector store and no policy engine — grants are integers in a dict — and passes
all fifteen.
Checking that the suite would notice
agentic-dataset-conformance run --matrix
Nineteen deliberately broken variants, each removing exactly one guarantee. Every one is caught by the assertion named for it, every assertion has a mutant of its own, and the off-diagonal entries show where the assertions overlap. A suite that cannot fail is decoration.
The vectors are CC0
agentic-dataset-conformance vectors --export ./vectors
The worlds and vectors under data/ are dedicated to the public domain:
no attribution required, no conditions. Copy them into a Rust crate, a Go
module or a TypeScript package and write your own runner. Python is one runner,
not the specification.
The software around them is Apache-2.0, so the distribution as a whole is
Apache-2.0 AND CC0-1.0.
What conformance does and does not establish
capabilities() is the subject's own report of itself. A subject that
under-reports passes AD-002 while hiding a tool. Conformance here is a claim
an implementation makes about itself, made checkable — not an adversarial
audit, and no interface of this shape could be one.
Links
1 maintainer
Metadata
1 maintainer
Release files for agentic-dataset-conformance 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| agentic_dataset_conformance-0.2.0.tar.gz | 33.8 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| agentic_dataset_conformance-0.2.0-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 78.2 kB
Release files / agentic_dataset_conformance-0.2.0.tar.gz
| Download URL | agentic_dataset_conformance-0.2.0.tar.gz |
|---|---|
| Size | 33.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
036498bbaca9bbea9585637145d14f913cc4647c73b1123b38a5dc3df9d612a0
|
|
BLAKE2b-256 checksum How to use checksums |
726393e92253f99ad73cd520b0914343b8ab56d9217462db33bb81c912699ab7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.14
|
Release files / agentic_dataset_conformance-0.2.0-py3-none-any.whl
| Download URL | agentic_dataset_conformance-0.2.0-py3-none-any.whl |
|---|---|
| Size | 44.3 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
466aebbd6b1303434db83d10b1715b1751165e3f0f8d19ca83b896488293d600
|
|
BLAKE2b-256 checksum How to use checksums |
99b7affbafcc41671d48cd04f2f826a7a275a5bf0376572636bc97b6bace3042
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.14
|
Security for agentic-dataset-conformance 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
