Skip to main content

darsay

Keep a model forever. Run it tomorrow.

pipx install darsay

darsay archive Qwen/Qwen3-0.6B
darsay run     qwen--qwen3-0.6b "Say hello"

A pinned snapshot of the Hub — hashed, licensed, and still loadable as-is.
The Hub is a living website. A darsay bundle is a museum piece that still runs.
Python 3.10+ · macOS and Linux · Apache 2.0

PyPI CI Apache 2.0 Python 3.10+ Manifest schema 1.6.0

Start here · Concepts · Examples · All docs · Contributing


The idea in four lines

You have a vault. It holds bundles. A catalog is a list of sources you intend to archive — shareable before any payload exists.

Each bundle is one pinned revision of one source: an immutable payload (model/ or data/), a manifest.json of recorded facts, and one file you write by hand (curation.md).

archive puts the snapshot in the vault. run speaks to it offline. export packs it into a single .mvb.tar that any tar can open in 2040. list is the vault as a catalog view; list summer overlays a want-list.

The payload never changes. The Hub can.

  Hugging Face                         your disk
  ────────────                         ─────────
  Qwen/Qwen3-0.6B                      vault/
       │                                 │
       │  darsay archive                 │
       └──────────────►──────────────────┤
                                         │
                          qwen--qwen3-0.6b/<rev>/
                          ├── model/           frozen snapshot
                          ├── manifest.json    recorded facts
                          └── curation.md      your notes
                                         │
                          darsay run ────┘──► tokens, offline
On the Hub In a bundle
A living website A pinned git revision
Can be gated, rewritten, deleted Payload is immutable after archive
“latest” is a moving target The manifest records what was true
You hope it still loads darsay run is offline proof

New here? Take the guided first bundle (~five minutes with a tiny model). Want the mental model in full? Concepts. Want copy-paste recipes? Examples.

Install

Requires Python 3.10+. Isolated CLI tools are the intended way to run a release; see distribution.

pipx install darsay
# or, no install at all:
uvx darsay --help
# or:
uv tool install darsay
Homebrew tap — org tap, not homebrew/core

Unqualified brew install darsay will not find it.

brew install darsay-io/darsay/darsay
Development checkout
python3 -m venv .venv
.venv/bin/pip install -e .                      # core: huggingface_hub only
.venv/bin/pip install -e ".[fast-hash,smoke]"   # + blake3, tokenizers
.venv/bin/pip install -e ".[inference]"         # + transformers/torch for in-process smoke
.venv/bin/pip install -e ".[datasets]"          # + pyarrow for measured dataset row counts
.venv/bin/pip install -e ".[dev]"               # pytest
.venv/bin/pytest                                # unit + integration; see docs/TESTING.md

The extras only serve in-process smoke tests. darsay run needs none of them — hydration builds its own isolated env per engine.

The vault root defaults to ~/darsay (override with --vault or $DARSAY_HOME; --vault works before or after the subcommand). Bundles are gitignored in this checkout — they live on disk or in your backup tier, not in the git repo.

The three verbs

1. Estimate — price the source from Hub metadata. Nothing downloaded.

darsay estimate Qwen/Qwen3.8-27B
Qwen/Qwen3.8-27B @ main -> 1d4bf0f2ff60
  image-text-to-text | license apache-2.0
  parameters:   27.78B BF16  [upstream safetensors metadata]
  payload:      32 files, 51.8 GiB
  engines:      transformers
  completeness: complete
  download:     ░░░░░░░░░░░░░░░░░░░░░░░░    0.0%   0 B / 51.8 GiB
                nothing banked yet — full 51.8 GiB in 32 files to fetch
  bundle:       ~/darsay/qwen--qwen3.8-27b/1d4bf0f2ff60  (new)
  disk:         needs ~55.5 GiB, free 1022.6 GiB — OK

To archive: darsay archive Qwen/Qwen3.8-27B

Exits non-zero when free space is insufficient, so it doubles as a guard in scripts. --variants lists the quantized ecosystem. --include '*Q4_K_M*' prices one file inside a huge GGUF pack. When a partial archive already exists, the download block prices the resume instead — verified, partial, and unverified bytes are banked, and only the remainder counts against disk:

  download:     ██████▋░░░░░░░░░░░░░░░░░   27.4%   14.2 GiB / 51.8 GiB
                banked 14.2 GiB = 12.1 GiB verified in 8 files + 2.1 GiB partial in 1 file
                still to fetch 37.6 GiB in 24 files

2. Archive — pin a revision, fetch bytes, hash them, write the manifest.

darsay archive Qwen/Qwen3-0.6B

Ctrl-C is fine. Budgets are fine. The same command resumes:

darsay archive Qwen/Qwen3.8-27B --max-gb 10     # tonight
darsay archive Qwen/Qwen3.8-27B --max-gb 10     # tomorrow
darsay archive Qwen/Qwen3.8-27B                 # finish, verify, register

3. Run — isolated env, fully offline, payload untouched.

darsay run qwen--qwen3-0.6b "Say hello"

Or skip the tool and point any Hugging Face-compatible loader at the payload:

from pathlib import Path
from transformers import AutoModelForCausalLM, AutoTokenizer

path = Path.home() / "darsay/qwen--qwen3-0.6b/<rev>/model"
tok = AutoTokenizer.from_pretrained(path)
model = AutoModelForCausalLM.from_pretrained(path)

No unpacking. No conversion. The archived files are the model.

A bundle

vault/qwen--qwen3-0.6b/<revision12>/
├── model/              # immutable payload: pristine snapshot of the upstream repo
├── manifest.json       # machine-readable record — the source of truth
├── README.md           # human-readable summary, generated from the manifest
├── VERIFICATION.md     # latest verification report
├── verification.json   # verification history (last 50 runs)
├── curation.md         # curator's notes — the only hand-edited file
├── exports.json        # log of single-file exports (after first export)
├── hydration.json      # runnable-env record + run history (after first hydrate)
├── transfer.json       # disposable resumable-transfer ledger
├── transfer.lock       # transient writer lock (only during archive/assemble)
└── LICENSE             # upstream license text, surfaced at the root

The payload under model/ (or data/ for datasets) is immutable after archiving; the bundle hash covers it alone.

What the manifest records
Section Contents
identity name, family, publisher, version, release date, bundle id
source provider, address, pinned commit, transfer accounting, mirrors, signatures, popularity + tags at archive time
licensing SPDX id, license files, commercial / redistribution / modification / attribution flags, patent grant, trademark terms
inventory per-file size + SHA-256 (+BLAKE3), upstream checksum match, deterministic bundle hash
model_metadata parameter count by dtype (from safetensors headers — no torch), architecture, context, tokenizer, languages
runtime engines from shipped formats, estimated min RAM/VRAM, measured hardware from darsay run
validation checksum verification, completeness, tokenizer + inference smoke tests
relationships parents, finetunes, known quantizations + GGUF repos (snapshot at archive time)
archive date, host, storage tier, backups, last integrity check
security integrity status, unexpected-change flags, trust level
curation historical significance, capabilities, limitations, notes (via curation.md)

schema_version is recorded in every manifest. Field-by-field reference: docs/MANIFEST.md.

Commands

You want to… Type
Price a source, download nothing darsay estimate Qwen/Qwen3.8-27B
Keep it darsay archive Qwen/Qwen3-0.6B
Talk to it darsay run qwen--qwen3-0.6b "Say hello"
See what you have darsay list (STATUS / SOURCE / HAVE; path in --json / info)
Share a want-list darsay catalog new summer then list summer / archive --next summer
Disk use darsay du
Stop before the disk fills on by default (2 GiB floor, exit 10); darsay config shows it, --min-free 10G raises it
Leave bandwidth for everyone else darsay archive Qwen/Qwen3.8-27B --max-rate 5M (or max_rate = "5M" in config.toml)
Survive a dropped network nothing to type — the panel reads offline, keeps what arrived, resumes when it is back (--max-offline 4h waits longer)
Delete a bundle darsay rm qwen--qwen3-0.6b --yes
Re-hash and compare darsay verify qwen--qwen3-0.6b
Diagnose the whole vault darsay doctor (offline; --fix is journaled and undoable)
Pack one file for a USB drive darsay export qwen--qwen3-0.6b -o /backups
Bring that file back darsay import /backups/<file>.mvb.tar

Source refs are provider-qualified — huggingface:Qwen/Qwen3-0.6B, huggingface:datasets/owner/name. Unprefixed owner/name / datasets/owner/name and huggingface.co URLs are Hugging Face shorthand. An unprefixed owner/name that exists only as a dataset is resolved at pin time to huggingface:datasets/owner/name.

The rest of the CLI
darsay estimate Qwen/Qwen3.8-27B --variants
darsay estimate unsloth/Qwen3.8-27B-GGUF --include '*Q4_K_M*'
darsay archive  unsloth/Qwen3.8-27B-GGUF --include '*Q4_K_M*'
darsay classify OBLITERATUS/Qwen3.8-27B-OBLITERATED   # master/print verdicts per weight set
darsay archive  OBLITERATUS/Qwen3.8-27B-OBLITERATED --full   # whole repo, skipping the masters-first default
darsay estimate datasets/saidutta69/fable-5-premium
darsay archive  datasets/saidutta69/fable-5-premium
darsay archive  Qwen/Qwen3.8-27B --revision v1.0      # pin a tag or commit instead of main
darsay archive  Qwen/Qwen3.8-27B --max-gb 10          # pause; rerun to resume
darsay archive  Qwen/Qwen3.8-27B --min-free 10G       # pause below 10 GiB free
darsay archive  Qwen/Qwen3.8-27B --yes                # bank what fits even if the disk cannot hold it all
darsay archive  Qwen/Qwen3.8-27B --max-rate 5M        # cap at 5 MiB/s
darsay archive  Qwen/Qwen3.8-27B --max-offline 4h     # wait out a lost network longer
darsay config                                         # effective settings + config files
darsay doctor                                         # offline diagnostics; writes evidence only
darsay doctor --fix --json                            # low-risk repairs + stable automation output
darsay doctor undo latest                             # restore the prior bytes/state
darsay archive  Qwen/Qwen3.8-27B --dry-run            # verified / partial / missing
darsay archive  Qwen/Qwen3.8-27B --shard 1/3 --max-gb 20
darsay --vault ./combined assemble /usb/alice/<bundle> /usb/bob/<bundle>
darsay list --json
darsay du
darsay rm       <bundle> --yes
darsay complete zsh                                   # eval "$(darsay complete zsh)"
darsay smoke    <bundle> [--inference]
darsay info     <bundle>                              # path, id, or unique prefix
darsay regen    <bundle>                              # rebuild README after editing curation.md
darsay hydrate  <bundle> [--dry-run]
darsay envs [--prune]
darsay dehydrate <bundle>
darsay rm       <bundle> -n                           # -n / --dry-run: same checks, nothing written

Every command that writes — archive, assemble, import, export, rm, regen, hydrate, run, dehydrate, envs --prune, the catalog verbs, estimate CATALOG — takes -n / --dry-run: the same checks, the same report in the conditional, nothing written, and the real command to paste.

Adding another host is a source provider, not a new CLI: docs/SOURCES.md.

When you need more

Pause, resume, share the work. Ctrl-C, --max-gb, a dropped network, a USB stick, --shard 1/3 with a collaborator. Partial bundles are relocatable. Incremental transfer · recipe

Datasets are the same shape. Addressed datasets/owner/name, payload under data/. Everything else is identical. Datasets · recipe

Export is a plain tar. Deterministic .mvb.tar, marker first, no compression, recoverable with stock tar. MVB format · recipe

Quants are history or cache, never both. Archive the master. Archive published quants that matter. Derive the rest at run time. Quantization

verify re-hashes the payload against the manifest and exits non-zero on drift — suitable for cron. import fully re-hashes before a bundle enters the vault; failures register nothing.

run hydrates first: picks an engine from what the payload ships (safetensors → transformers, GGUF → llama-cpp), builds a dedicated venv outside the bundle under <vault>/.runtime/, and infers with HF_HUB_OFFLINE=1. Envs are disposable. Design: docs/HYDRATION.md.

Documentation

Start here Then When you need the spec
Getting started Concepts · Examples Documentation home
Manifest MVB format the two documents a 2040 reader still needs
Hydration · Incremental · Datasets Sources · Quantization how the verbs actually work
Design Distribution · Testing why Python, how a release is consumed, what CI keeps

The tool is replaceable. The formats are not — plain JSON and plain tar, each documented for recovery without darsay. Full rationale: docs/DESIGN.md.

License

Apache License 2.0. See LICENSE. Bundles record upstream model and dataset licenses separately in manifest.json; those do not change the license of this tool.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

darsay-0.14.5.tar.gz (2.4 MB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

darsay-0.14.5-py3-none-any.whl (218.7 kB view details)

Uploaded Python 3

File details

Details for the file darsay-0.14.5.tar.gz.

File metadata

  • Download URL: darsay-0.14.5.tar.gz
  • Upload date:
  • Size: 2.4 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for darsay-0.14.5.tar.gz
Algorithm Hash digest
SHA256 3dfe75291abfe0ae9f2cc586c13882a3a8e02fa5947a668e732056a0fb0e8161
MD5 74a007167658377cbbf43141918a59fc
BLAKE2b-256 c07a27a87e418081534962db0bbd9374be6516c9159ddfded3429b3c6f0751f8

See more details on using hashes here.

Provenance

The following attestation bundles were made for darsay-0.14.5.tar.gz:

Publisher: release.yml on darsay-io/darsay

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file darsay-0.14.5-py3-none-any.whl.

File metadata

  • Download URL: darsay-0.14.5-py3-none-any.whl
  • Upload date:
  • Size: 218.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/7.0.0 CPython/3.13.14

File hashes

Hashes for darsay-0.14.5-py3-none-any.whl
Algorithm Hash digest
SHA256 db302beadda460ff6645ae8a68e1ee655367a93c63952ba380932d493c8b89c1
MD5 1060ec9dc068826c8ad6afdffc75000e
BLAKE2b-256 1e5cae0fca7dcbb1a4b37631fb8a181dd7e0b1374716c2a2e044e38aebeb77a6

See more details on using hashes here.

Provenance

The following attestation bundles were made for darsay-0.14.5-py3-none-any.whl:

Publisher: release.yml on darsay-io/darsay

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

0.14.15

2 files

0.14.14

2 files

0.14.13

2 files

0.14.12

2 files

0.14.11

2 files

0.14.10

2 files

0.14.9

2 files

0.14.8

2 files

0.14.7

2 files

0.14.6

2 files

This release

0.14.5 This release

2 files

0.14.4

2 files

0.14.3

2 files

0.14.2

2 files

0.14.1

2 files

0.14.0

2 files

0.13.0

2 files

0.12.0

2 files

0.11.0

2 files

0.10.0

2 files

0.9.0

2 files

0.8.0

2 files

0.7.0

2 files

0.6.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page