YY Corpus Evidence

Create deterministic, local corpus manifests and make missing, incomplete, or conflicting provenance declarations visible before a dataset release or training run.

YY Corpus Evidence records evidence. It does not decide whether data is legal, compliant, safe, or authorized for training.

Requirements

The initial release supports CPython 3.11 on Windows. Other Python versions and platforms are unsupported until they pass the release matrix.

Installation

python -m pip install yy-corpus-evidence

Commands

yy-corpus-evidence scan DATASET --declarations declarations.json --output manifest.json
yy-corpus-evidence verify DATASET manifest.json --declarations declarations.json
yy-corpus-evidence check manifest.json --require-declarations

scan inventories a local file tree. verify compares the current tree and declarations with a prior manifest. check --require-declarations fails when required evidence is unknown, incomplete, or conflicting.

Manifest disclosure

A manifest is not redacted. Review it before sharing it.

• Relative file paths appear verbatim.
• Declaration text appears verbatim, including declared_by, statement.value, source_url, note, and the source name.
• File contents do not appear. Byte sizes and SHA-256 digests do.
• A digest can confirm whether a file already possessed by another party is present in the corpus.

There is no hashed-path or redaction mode. Treat a manifest as being as sensitive as the names and statements it contains.

Privacy and safety defaults

• no network access, telemetry, or update check;
• no file content in output;
• no absolute root, username, hostname, or timestamp in output;
• no traversal of symlinks, junctions, or other reparse points; and
• output refuses to sit inside the scanned source unless --allow-output-inside-source is explicitly passed.

The scanner reads file content locally to calculate SHA-256. It does not retain or emit the raw content.

Evidence states

Declarations are human assertions. The tool records them and reports gaps; it does not determine whether an assertion is correct.

Each asset receives one state:

• unknown - no applicable declaration;
• declared - one internally consistent set of applicable declarations;
• incomplete - a governing declaration references an absent evidence file; or
• conflict - equally specific declarations of the same kind disagree.

An exact declaration outranks a prefix declaration. Among prefix declarations, the longest matching prefix wins.

The tool deliberately performs no cross-kind legal reasoning. For example, it does not infer whether a free-text permission contradicts a free-text restriction. Human review remains necessary.

Integrity limitations

manifest_id is a deterministic, unkeyed identity calculated from the manifest's canonical bytes. It detects corruption and accidental divergence. It is not a signature, authentication mechanism, or tamper-evidence system. Anyone who changes a manifest can recompute a valid identifier.

A scan also reads a live filesystem and is not an atomic snapshot. It detects common concurrent changes using size, modification time, and inode checks, but the intended threat model is a cooperative local scan rather than a hostile filesystem.

Exit codes

Errors do not emit tracebacks or exception text because those may disclose local paths, file names, or declaration text.

JSON Schemas

The distribution includes exact Draft 2020-12 structural schemas for declaration input and manifest output under yy_corpus_evidence/schemas/.

The schemas constrain emitted structure. Runtime checks still enforce computed identities, declaration references, semantic duplicates, summary counts, and evidence-state behavior.

Support and security

Read SUPPORT.md before filing an issue and SECURITY.md before reporting a security-sensitive defect. Do not submit proprietary corpora, unredacted manifests, secrets, personal information, or private paths.

Project overview and documentation: yycore.ai/projects/corpus-evidence.

Development provenance

YY Corpus Evidence was developed under Cameron Emery's direction with material code-generation and review assistance from multiple generative-AI systems. Cameron defined the product scope, requirements, and risk boundaries and is responsible for maintenance and distribution.

AI agents generated, reviewed, selected, rejected, and revised implementation material. The tool and provider identities are preserved in a private internal provenance record. No AI system is identified as a legal author or owner, and this disclosure does not imply affiliation or endorsement.

The copyright notice applies only to protectable human-authored contributions, selection, coordination, arrangement, modifications, and separately owned adapted material. No claim is made that machine-generated expression becomes human-authored or independently copyrightable. Read PROVENANCE.md for the public development summary, clean-export policy, and limits of Git attribution evidence.

License

Copyright 2026 Cameron Emery for protectable human-authored contributions, selection, coordination, arrangement, modifications, and separately owned adapted material.

Licensed under the Apache License, Version 2.0, only to the extent of rights held by the applicable contributor. See LICENSE, NOTICE, and PROVENANCE.md.