caf

CAF (content addressable files) generates deterministic random files for storage-system testing and verifies that the stored files have not changed. It can generate many small files or split large files across worker threads.

Create up to 10 MB of files. The default file size is 4 KiB:

caf gen --max-disk-usage 10MB

Then verify the store:

caf verify

Generation writes CAF v3 by default. Use --format v2 when a new chain must use the older whole-file BLAKE2b-160 identity:

caf gen --format v2 --max-disk-usage 10MB

With CAF v3, use a seed to share a command that reproduces the same dataset:

caf gen --seed blahblah --max-files 100 --file-size 4096 --format v3

In a fresh directory, the same seed and generation arguments produce the same file sizes, contents, relative paths, parent links, and CAF metadata across releases. The directory and worker count may differ. Seed text is exact UTF-8, including whitespace and case; an empty seed is a usage error. Omitting --seed chooses fresh randomness. Filesystem timestamps, ownership, inode numbers, and physical allocation are outside this guarantee. Generation appends to existing stores, so reproducing an existing store also requires identical starting contents and invocation history.

--seed requires CAF v3, the default format; combining it with --format v2 is a usage error. Seeded dataset generation specifies the CAF v3 behavior, including exact statistical size sequences. dev corrupt-file --seed continues to take an integer.

caf verify detects v2 and v3 from each file's 60-byte header. A store may contain chains of both versions, but every file in one chain must use the same version. Both versions use the same 40-character IDs, sharded data paths, chain-tip markers, and .metadata/all aggregate.

Both commands use parallel workers by default: half the available logical CPUs, with at least two workers on multicore systems and a cap of eight. Override the worker budget with --jobs. CAF v3 generates or reads physical file blocks independently and reduces their BLAKE3 leaf hashes to a deterministic Merkle file ID. CAF v2 preserves its ordered BLAKE2b-160 whole-file identity. The worker count affects runtime, not generated bytes or verification results:

caf gen --max-files 1 --file-size 4GB --jobs 8
caf verify --jobs 8

In an interactive terminal, both commands show live byte/file progress, throughput, and an estimated time remaining. Progress is rendered on standard error and is automatically disabled when output is redirected or captured.

Run caf gen --help for size distributions and more examples. See the file-format specification for the v2 and v3 header, content, identity, path, chain, and metadata rules.