Pythonaibrain-Warden

warden is an industrial-grade, npm-inspired dependency & environment manager for Python. It owns its own manifest (warden.toml) and a deterministic, hash-pinned lock file (warden.lock), resolves against the real PyPI index, and installs into isolated, named virtual environments.

Console output

Terminal output goes through rich: real progress bars for downloads, live spinners while resolving dependencies or bootstrapping pip, aligned tables for warden show deps/env/lock/targets, and a bordered result panel for warden verify's final verdict. It degrades gracefully when piped or redirected (CI logs, > output.txt) — no raw ANSI codes or garbled spinner frames.

Every dynamic value (package name, version spec, file path) is rendered through rich.text.Text rather than interpolated into a markup string. This is a correctness requirement, not a style choice: a dependency specifier can legitimately contain literal square brackets (pkg[extra1,extra2]>=1.0), which rich markup would otherwise parse as style tags and silently swallow.

Design principle: a thin CLI over a real Core API

Every subcommand is a thin argparse wrapper around warden.core, the Warden Core API. No command implements its own logic in cli.py — it parses arguments and calls into core:

CLI
 |
 +-- install --+
 +-- update ---+
 +-- build ----+--> warden.core --> config / lockfile / registry / resolver /
 +-- publish --+                    installer / venv_manager / fixer / verify / ...
 +-- run ------+

This keeps behavior identical whether it's invoked from the terminal, from a test, or embedded programmatically (import warden.core as core).

Correct platform/interpreter wheel selection

Artifact selection is ranked against packaging.tags.sys_tags() -- the same mechanism pip's own resolver uses -- instead of taking "the first wheel PyPI happens to list." PyPI's file listing isn't ordered by platform relevance, so packages with no pure-Python wheel (C extensions like markupsafe, cryptography, etc., which ship a separate wheel per OS/interpreter/ABI combination) previously risked getting an arbitrary wheel -- e.g. a macOS wheel selected while resolving on Windows, which then fails to install with a "not a supported wheel on this platform" error from pip. warden install now always picks the best match for the exact interpreter/platform doing the resolving, and only falls back to building from an sdist if literally no wheel matches.

A single incompatible or broken package also no longer aborts the whole install: warden install/warden update install everything they can, report exactly which package(s) failed and why, and still complete downstream steps (like editable-installing the project itself) that don't depend on the failure -- instead of leaving the project in a half-installed state with a confusing, unrelated error on the next warden run.

Environments are Warden-native, not venv.EnvBuilder

warden wenv doesn't call Python's venv module. It builds the standard CPython venv layout itself — directory structure, interpreter symlink/copy, a real pyvenv.cfg, and a pip bootstrap via ensurepip — end to end, so Warden owns the whole process instead of delegating to stdlib behavior it can't fully control.

This also fixes a real cross-platform correctness issue: process launch always resolves the target executable to an absolute path inside the environment (warden.venv_manager.resolve_executable) rather than relying on PATH search. That matters specifically on Windows, where CreateProcess resolves a bare name like "python" against the calling process's PATH, not the custom env= dict handed to subprocess — so entry = "python -m mypackage" would previously launch the system interpreter instead of the venv's, even with the venv's Scripts dir prepended to PATH. warden run / warden exec now never depend on that resolution happening correctly.

The active environment

Every command that touches an environment (install, run, exec, fix, verify, update, uninstall) targets the project's active environment when --env isn't given — not always a hardcoded "default". warden wenv <name> (its default action, create) and warden use <name> (a shortcut for warden wenv <name> use) both switch it, the same way nvm use or git checkout -b make something the thing subsequent commands act on:

warden wenv cli        # creates 'cli' AND makes it active
warden install click   # installs into 'cli' -- no --env needed
warden run              # runs in 'cli' -- no --env needed
warden use default      # switch back to 'default' (created if it doesn't exist yet)
warden use               # bare form: just reports which env is active

--env <name> always overrides the active one for that single command. warden wenv list marks which environment is currently active. The active environment is tracked in .warden/active-env — local machine state, not project policy, so it's never written into warden.toml and never shared between contributors or CI.

Why not just pip / poetry / pdm?

Warden isn't trying to replace pip as the low-level installer — it uses pip install <exact-artifact> --no-deps under the hood, because reinventing wheel unpacking would be pointless and risky. What Warden adds:

• A lock file that is the single source of truth. Once something is locked, installing it downloads that exact URL, verifies its sha256 against the lock, and installs with --no-deps — pip's own resolver is never allowed to silently substitute a different version.
• Multiple named environments per project (warden wenv test, warden wenv ci, ...), not just one .venv.
• A hard split between "fix the environment" and "change the requirements." warden fix repairs installed-vs-locked drift — missing packages, wrong versions — by reinstalling exactly what's locked. It never edits warden.toml or warden.lock. Changing what's required is install/update's job, deliberately, always.
• warden verify — a strict, CI-grade pass/fail gate: manifest validity, lock/manifest consistency, real sha256 re-verification of every locked artifact, and exact installed-vs-locked environment comparison. Non-zero exit on any failure, safe to put in a pipeline.
• warden build produces a self-contained .warden.wheel — a Warden-native archive with the manifest, lock, source, and (by default) every dependency artifact vendored in, so it can be reconstructed and installed fully offline.
• warden publish ships that .warden.wheel to a registry URL you configure ([registry].publish-url in warden.toml); pass --pypi if you'd rather do a conventional build+twine release to PyPI instead.

Install

pip install pythonaibrain-warden

(or, from this source tree: pip install -e .)

Quick start

warden init myapp
cd myapp
warden install                 # sync env from warden.toml (creates it if empty)
warden install requests        # add a dependency, latest version
warden install "flask@3.0.0"   # add a dependency, exact version
warden run                     # runs [project].entry inside the 'default' env
warden verify                  # strict reproducibility check (CI-friendly)

Command reference

Command	Description
warden init <name>	Scaffold a new project (src layout, pyproject.toml, warden.toml, git-ignore, tests dir).
warden install [pkg[@ver] ...] [-f|--force]	Add package(s) (latest version if @ver omitted), resolve, write warden.lock, install into the target env, editable-install the project itself. No args = sync env from the existing manifest/lock. A package already installed at exactly the locked version is skipped, not reinstalled — --force bypasses that and reinstalls everything.
warden lock [pkg[@ver]]	Pin a package into warden.toml and/or just regenerate warden.lock from the current manifest.
warden update [pkg ...] [-f|--force]	Bump dependencies to the latest version allowed by their constraint, relock, reinstall. Omit args to update everything. Same skip-if-already-current behavior as install for anything whose version didn't actually change.
warden wenv <name> [create|remove|list|use]	Manage named virtual environments. create (the default action) and use both make that environment the active one — the one every other command targets when --env isn't given. warden wenv list (bare) lists all of them, marking the active one.
warden use [env name]	Shortcut for warden wenv <name> use. Bare warden use (no name) reports the current active environment instead of switching.
warden run [--env NAME] [-- extra args]	Run [project].entry from warden.toml inside an environment.
warden run-file <file> [-- args]	Run one specific file. If it has a [targets] override, it runs in that file's own dedicated environment (auto-provisioned on first use); otherwise it falls back to the default env.
warden target set|remove|list <file> [pkg@ver]	Manage per-file dependency overrides — pin a specific file to a specific dependency version, distinct from the rest of the project. See below.
warden exec [--env NAME] -- <command...>	Run an arbitrary command inside an environment — distinct from run, which only runs the configured entry point.
warden fix [package]	Repair environment drift against warden.lock — reinstalls whatever is missing or at the wrong version. Never edits warden.toml/warden.lock; that's what install/update are for.
warden verify [--env NAME] [--offline]	Strict, CI-grade reproducibility check: manifest validity, lock consistency, real artifact hash re-verification, exact env-vs-lock comparison. Non-zero exit on any failure.
warden build [--offline] [--no-vendor]	Build a .warden.wheel — manifest + lock + source + (by default) every dependency artifact vendored in for offline installs.
warden install-wheel <file> [--target DIR] [--env NAME] [--no-install]	Reconstruct and set up a project from a .warden.wheel built by warden build — unpacks source/manifest/lock, creates the environment, installs every locked package (from the wheel's vendored copies when present, fully offline).
warden backup [--output PATH] [--no-vendor]	Snapshot the project — manifest, lock, source, a record of what's actually installed in each env, and (by default) every dependency artifact vendored in — to backups/<name>-<version>-<timestamp>.warden.backup. Take one before anything risky (update, clean, hand-editing warden.toml).
warden restore <file> [--target DIR] [--env NAME] [--no-install]	Rebuild a project from a warden backup snapshot — same offline-first install behavior as install-wheel.
warden publish [--pypi] [--repo NAME] [--dry-run]	Publish the built .warden.wheel to [registry].publish-url. --pypi instead builds a real sdist+wheel and uploads via twine.
warden cache show|clean|verify	Inspect, clear, or hash-verify the global ~/.warden/cache (registry metadata + downloaded artifacts).
warden show [deps|env|lock|cache|package <name>[@version]]	Inspect the project. Bare warden show prints a full overview.
warden clean [--cache]	Remove .warden/venvs/, dist/, build/, __pycache__/, *.egg-info/. Never touches warden.toml/warden.lock. --cache also clears the global cache.
warden version	Warden version, Python version, platform.
warden help [command]	Top-level help, or help for one subcommand.
warden uninstall <pkg>	Remove a dependency from the manifest, relock, uninstall from the env.
warden list / warden freeze / warden tree / warden search <name>	Bonus power tools: full lock listing, pip freeze-style output, dependency tree, PyPI metadata lookup by name.

warden.toml

[project]
name = "myapp"
version = "0.1.0"
description = "demo app"
python = ">=3.9"
entry = "python -m myapp.main"

[dependencies]
requests = "==2.34.2"

[dev-dependencies]
pytest = "^8.0.0"

[environments]
default = ".warden/venvs/default"
test = ".warden/venvs/test"

[targets]
"scripts/legacy_report.py" = { requests = "==2.28.0" }

[registry]
index-url = "https://pypi.org/simple"
json-url = "https://pypi.org/pypi"
publish-url = ""   # set this to enable `warden publish`

Version constraint syntax

Syntax	Meaning
*	any version
1.2.3	exact pin (==1.2.3) — the default is explicit, not fuzzy
^1.2.3	compatible up to next breaking change (>=1.2.3,<2.0.0; npm-style, 0.x treated as unstable)
~1.2.3	patch-level only (>=1.2.3,<1.3.0)
>=2.0,<3.0	any raw PEP 440 specifier, passed through as-is

warden.lock

Every entry is a fully pinned, hash-verified artifact:

[[package]]
name = "requests"
version = "2.34.2"
sha256 = "2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0"
url = "https://files.pythonhosted.org/.../requests-2.34.2-py3-none-any.whl"
filename = "requests-2.34.2-py3-none-any.whl"
kind = "wheel"
requires = ["charset_normalizer<4,>=2", "idna<4,>=2.5", "..."]
dependents = ["<project>"]

Commit warden.lock to version control — it's what makes installs reproducible across machines and CI.

.warden.wheel

warden build produces dist/<name>-<version>.warden.wheel, a zip archive:

myapp-0.1.0.warden.wheel
├── WARDEN_BUILD.json      build/execution metadata (project info, package list + hashes)
├── warden.toml
├── warden.lock
├── src_root/...           a clean copy of the project source
└── vendor/*.whl|*.tar.gz  every locked dependency artifact (unless --no-vendor)

This is a Warden-native artifact, not a PEP 427 wheel — it's meant to be fully reconstructable and (with vendor/ present) installable offline by Warden itself. warden install-wheel some.warden.wheel is the exact counterpart to warden build: it unpacks the source and lock, creates the environment, and installs every package — from the wheel's vendored copies when present, so a fully-vendored .warden.wheel sets up completely offline; anything not vendored falls back to a normal registry download.

vendor/ also includes the project's build-system tools (typically setuptools/wheel, from pyproject.toml's [build-system].requires) alongside the runtime dependencies. This matters because editable- installing the project itself (pip install -e .) defaults to PEP 517 isolated builds, which fetch setuptools/wheel from PyPI on every call unless they're already importable in the target environment — and a freshly created Warden environment only ever has pip, since modern CPython's ensurepip no longer bundles them. Without vendoring these too, a "fully offline" .warden.wheel would still silently need network access for this one step. warden install-wheel/warden restore both install these from the archive first (via pip --no-index --find-links) and then run the editable install with --no-build-isolation, so the whole thing genuinely never touches the network when everything's vendored — verified by physically blocking DNS resolution to PyPI's hosts during a real install-wheel run, not just checking for a --no-vendor flag.

Backups and disaster recovery

warden backup is warden build's sibling with a different purpose: not a release artifact, but a restore point you take before anything risky (warden update, warden clean, hand-editing warden.toml). It writes a .warden.backup to backups/<name>-<version>-<timestamp>.warden.backup containing everything .warden.wheel does, plus a snapshot of exactly what was installed (name + version) in each environment at backup time, for drift comparison later.

warden restore some.warden.backup reverses it — unpacks source/manifest/ lock into a target directory and rebuilds the environment the same offline-first way install-wheel does. Both restore and install-wheel refuse to write into a directory that already exists and isn't empty, so you can't accidentally clobber something by pointing at the wrong target.

Per-file dependency targets — different files, different versions, same project

Sometimes one project genuinely needs two versions of the same package at once — a legacy script that only works against numpy==1.24, a new one that needs numpy>=1.26, both living in the same repo:

myproject/
 ├── scripts/legacy_report.py   -> numpy==1.24.0
 └── scripts/new_report.py      -> numpy==1.26.4

warden target set <file> <pkg@version> records an override for that file in warden.toml's [targets] table:

[targets]
"scripts/legacy_report.py" = { numpy = "==1.24.0" }
"scripts/new_report.py"    = { numpy = "==1.26.4" }

Each target is not a fiction — it's a real, separate Warden environment (.warden/venvs/target-<slug>/), with its own real, independently resolved lock file under locks/<slug>/warden.lock (kept alongside warden.toml, not under .warden/, because — like warden.lock — it's reproducibility policy you should commit, not disposable generated state). A target's effective dependency set is the project's base [dependencies] with that file's overrides layered on top, so an override can either re-pin an existing dependency or introduce a file-only one.

warden run-file scripts/legacy_report.py runs that file inside its own target environment automatically — resolving, locking, and provisioning it on first use if it isn't set up yet. A file with no [targets] entry just runs in the default environment, same as always. warden install (the bare, no-argument form) provisions every declared target automatically alongside the default environment, so a fresh clone gets the whole multi-version setup from warden.toml in one command.

Target environments show up in the same places any other named environment does — warden wenv list, warden show env — since under the hood a target is just a normal Warden environment with an auto-generated name (warden target list / warden show targets gives the friendlier per-file view). warden target remove <file> [package] drops one override, or the whole target if no package is given.

fix vs install/update — read this once

Warden draws a hard line between two operations that are easy to conflate:

FIX ENVIRONMENT                    CHANGE DEPENDENCY REQUIREMENTS
(warden fix)                       (warden install / warden update)

inspect environment                 edit warden.toml
       |                                   |
compare against warden.lock         re-resolve
       |                                   |
detect mismatch                     write a new warden.lock
       |                                   |
reinstall exactly what's locked     install the new lock

fix will never rewrite what your project requires — it only restores what's installed to match what's already locked. If fix can't resolve a mismatch (e.g. the lock itself references an artifact that no longer exists), that's a policy problem for install/update/lock, not something fix will paper over.

Resolution model (read this before relying on it in deep dependency trees)

Warden performs greedy, deterministic resolution: for every package it picks the highest version satisfying every constraint discovered so far, walking the graph breadth-first. This is fast and has zero surprises for the overwhelming majority of real dependency trees. It is not a full SAT/PubGrub backtracking resolver like pip's — a genuine diamond conflict (two packages requiring mutually-exclusive version ranges of a third) is reported clearly rather than silently "resolved" incorrectly.

Architecture

See ARCHITECTURE.md for the full internal design writeup — module responsibilities, the dependency graph, the warden install request lifecycle traced end to end, and the reasoning behind the bigger design decisions (the Core API pattern, the hand-built venv, the active environment, per-file targets, the two archive formats). What follows here is just the module list.

warden/
  cli.py          argparse dispatch -- thin, calls into core.py exclusively
  core.py         the Warden Core API every command actually runs through
  config.py       warden.toml manifest (load/save/validate/mutate, per-file targets)
  lockfile.py     warden.lock (load/save, keyed by PEP 503 canonical name)
  registry.py     PyPI JSON API client, with on-disk caching
  resolver.py     version-spec parsing (^, ~, PEP 440) + greedy resolution
  installer.py    download (progress bar) -> sha256 verify -> `pip install --no-deps`
  venv_manager.py Warden-native environment lifecycle (not stdlib `venv.EnvBuilder`)
  utils.py        `Log`/`rich` console output, hashing, subprocess helpers
  runner.py       runs [project].entry, or an arbitrary command, in an env
  fixer.py        environment-drift diagnosis + repair (never touches policy)
  verify.py       strict CI-grade reproducibility gate (rich Panel verdict)
  build.py        `.warden.wheel` artifact builder
  backup.py       `.warden.backup` snapshot builder
  restore.py      rebuilds a project from a `.warden.backup`
  wheel_installer.py  rebuilds a project from a `.warden.wheel`
  archive_common.py   shared pack/unpack primitives for both archive formats
  publisher.py    `.warden.wheel` registry upload, or PyPI build+twine
  cache.py        global cache show/clean/verify
  clean.py        generated-state cleanup (env/build/dist), policy-safe
  show.py         project/deps/env/lock/targets/package inspection (rich tables)
  init.py         project scaffolding
  toml_writer.py  tiny dependency-free TOML serializer for our own schema

Caching

• Registry metadata: ~/.warden/cache/registry/*.json (30 min TTL)
• Downloaded artifacts: ~/.warden/cache/dist/* (content-addressed by filename, verified by sha256 on every use — a corrupted cache entry is detected and re-downloaded automatically)
• Manage either with warden cache show|clean|verify

Exit codes

Every failure mode has a stable exit code (see warden/exceptions.py) — ManifestError=2, LockfileError=3, RegistryError=4, ResolutionError=5, EnvironmentError_=6, InstallError=7, ProjectNotFoundError=8, PackageNotFoundError=9, IntegrityError=10, PublishError=11, BuildError=12, BackupError=13, RestoreError=14, WheelInstallError=15 — so CI pipelines can branch on why a step failed.