scitex-agent-container 0.21.7

Declarative YAML-based framework for defining, managing, and orchestrating AI coding agent instances

SciTeX Agent Container (scitex-agent-container)

Agent in Apptainer

Full Documentation · uv pip install scitex-agent-container[all]

---

Problem and Solution

#	Problem	Solution
1	Scripting an agentic workflow is hard.	scitex-agent-container (sac) declares the agent as a single YAML file (spec.yaml).
2	Subagents don't scale across hosts, projects, and contexts.	sac lets agents spawn full agents on local AND remote hosts.
3	Controlling agent permissions is difficult.	sac runs every agent inside Apptainer — full mount/env/security options exposed in spec.yaml.
4	Supporting the A2A protocol by hand is time-consuming.	sac needs just one YAML field (spec.a2a.port).
5	Version-controlling Apptainer recipes is laborious.	sac enables layered Apptainer images with a sandbox/update/freeze workflow via scitex-container.

Installation

uv pip install "scitex-agent-container[all]"

Quickstart

Step 1 — Build the base image (one-time, ~5 min)

sac image build base

Step 2 — Create agent directories

# Each agent lives in its own directory; the directory name is the agent name.
mkdir -p ~/.scitex/agent-container/agents/hello-agent-{1,2}

Step 3 — Write spec.yaml (copy into each agent directory, adjust startup_prompts)

# ~/.scitex/agent-container/agents/hello-agent-1/spec.yaml
apiVersion: scitex-agent-container/v3
kind: Agent

spec:
  runtime: apptainer

  apptainer:
    image: ~/.scitex/agent-container/containers/sac-base.sif

  claude:
    model: haiku
    flags:
      - --dangerously-skip-permissions

  startup_prompts:
    - "Reply with the string 'Hello! I am hello-agent-1' and nothing else."

  health:
    enabled: true
    interval: 60
    method: sdk-alive

  restart:
    policy: never

Or copy the bundled example: cp -r examples/agents/hello-agent ~/.scitex/agent-container/agents/hello-agent-1

Step 4 — Run

# Start in foreground (waits for completion)
sac agents start hello-agent-1 hello-agent-2 --foreground

# Check status
sac agents list

# Start in background, read output, stop, delete
sac agents start hello-agent-1 hello-agent-2
sac agents tail  hello-agent-1 hello-agent-2 --json
sac agents stop  hello-agent-1 hello-agent-2
sac agents delete hello-agent-1 hello-agent-2 -y

Tutorial

examples/ walks through the runtime in 15 lessons (image build, sandbox/update/freeze, versioning, run/send/tail, logs/exec, stop/remove, binds, env+user, writing your first spec.yaml, to_home/, A2A endpoint, health+restart, multi-host, debugging). Run them read-only with bash examples/00_run_all.sh, or --apply to execute the mutating ones.

Models

Pick the model per agent under spec.claude.model:

Alias	Model (current)	Use for
opus	Claude Opus 4.7	Hardest reasoning; slowest
sonnet	Claude Sonnet 4.6 (default)	Balanced capability and speed
haiku	Claude Haiku 4.5	Fast, cheap, light tasks

Aliases auto-track the latest version of each family; append [1m] for the 1M-token context window (opus[1m], sonnet[1m]). Pin an exact build with a full ID like claude-opus-4-7 or claude-haiku-4-5-20251001. To target a non-Anthropic, Anthropic-compatible backend (e.g. DeepSeek), use spec.claude.provider instead. Full model reference →

How it works

scitex-agent-container (sac) materializes a spec.yaml into a long-lived, externally addressable Claude agent:

  spec.yaml   ─┐
  to_home/    ─┴─→ sac agents start ──→ apptainer instance
                                          │
                                          ▼
                              long-lived Claude SDK session
                              │
                              ├── <workdir>  (= spec.workdir, mounted rw)
                              ├── spec.mounts[]  ← host-path allowlist (ro/rw)
                              ├── state-dir  (~/.scitex/agent-container/runtime/<name>/)
                              └─→ POST /v1/turn  (per-agent A2A inbound)

SAC-from-SAC (in-SIF spawn). An agent running INSIDE an apptainer SIF can spawn a child agent on the bare host by calling sac agents start <child> as normal — the CLI auto-detects the in-SIF condition (APPTAINER_CONTAINER) and POSTs the spawn RPC to the host's sac listen instead of trying nested apptainer (which the supported HPC shape forbids). The host re-runs ACL gating, records the parent → child lineage, and shells the real start against the bare host's apptainer. Wiring is automatic: SAC_LISTEN_BASE_URL + SAC_LISTEN_BEARER are injected at container launch.

Full architecture → — launch flow, to_home merge rules, A2A inbound, control plane, restart/health.

YAML Spec Reference (v3) → — annotated full example + field table (apiVersion, spec.apptainer., spec.claude., a2a, health, restart).

Talking to a Running Agent → — three transports (A2A POST /v1/turn, sac agents send, host-level sac listen), when to use which, copy-pasteable curl examples.

Container Isolation → — 10 Apptainer-default leak paths + sac's hardened-by-default countermeasures (--containall auto-prepended, opt-out via spec.apptainer.relaxed: true). The reference for reproducibility claims.

Configuration and Runtime Directories

Full directory reference → — complete tree, configuration cascade (CLI flag → env var → project config → user config).

~/.scitex/agent-container/
├── agents/<name>/spec.yaml    ← agent definition (SSoT)
├── containers/sac-base.sif    ← built images (gitignored)
└── runtime/<name>/            ← live state: pid, heartbeat, session.jsonl

Apptainer images → — base vs scitex layers, sandbox/freeze workflow, version pinning.

Host listen as a persistent service

sac listen is the host's HTTP/JSON control plane (push hub, spawn broker, lead inbox). For long-running deployments install the bundled systemd-user unit so it auto-starts on boot and auto-restarts on crash:

install -m 0644 scripts/systemd/sac-listen.service ~/.config/systemd/user/
systemctl --user daemon-reload
systemctl --user enable --now sac-listen.service
journalctl --user -u sac-listen.service -n 50          # logs
curl -s http://127.0.0.1:7878/v1/health                # healthcheck

See scripts/systemd/README.md for the full recipe + the federated-jobs vs hand-maintained-services split.

1 Interfaces

CLI ⭐⭐⭐ (primary)

# Agent lifecycle
sac agents start  <name> [--foreground]   # daemon by default; --foreground streams stdio
                                           # inside a SIF: auto-brokers to host listen
                                           # (no apptainer-in-apptainer needed)
sac agents stop   <name>                  # graceful SIGTERM, escalate to SIGKILL after 5 s
                                           # --force tolerates an unreachable bound host
sac agents restart <name>
sac agents delete <name>                  # stop + remove spec dir + runtime dir + registry
sac agents forget <name> [--force]        # local-only state.db cleanup for the
                                           # "agent is gone, only stale rows persist" case
                                           # (no ssh, no signal)
sac agents send   <name> "<prompt>"       # send a follow-up turn to a running session
sac agents send   <name> --key ESC        # interrupt current turn
sac agents list [<name>] [--snapshot] [--priority]
sac agents health <name>
sac agents tail   <name>                  # render session.jsonl (structured transcript)
sac agents recall <name>                  # human-readable session summary
sac agents check  <name>                  # preflight (validates yaml + probes runtime deps)
sac agents find   <capability>

# Control plane (HTTP/JSON, loopback-only)
sac listen [--bind 127.0.0.1:7878]       # boot per-host REST API (bearer-auth)
                                           # single-instance flock guard fails loud
                                           # on a duplicate launch (PID + lockfile shown)
sac peer post-turn <to> "<msg>"          # local agent-to-agent message via sac listen

# Image lifecycle (delegates to scitex-container)
sac image build [base|scitex] [--sandbox] [--runtime apptainer|docker]
sac image sandbox SOURCE                  # SIF → writable sandbox
sac image update  SANDBOX [-p PKG]        # pip install --upgrade
sac image freeze  SANDBOX OUT.sif         # sandbox → SIF
sac image list                            # installed versions
sac image switch  VERSION                 # atomic flip
sac image rollback                        # restore previous
sac image status                          # unified dashboard
sac image snapshot [-o env.json]          # reproducibility capsule

# Account / quota
sac accounts list / save / delete / switch        # stored-credential rotation
sac accounts status                       # one-shot quota snapshot (5h%, 7d%, tier)
sac accounts sync-live / watch-live       # auto-snapshot live cred on `claude /login`
sac accounts watch-quota                  # auto-rotate when quota threshold hit

# Network / peers
sac host list / add / remove / set / probe / exec / validate
sac host ssh-opts                         # print sac's ssh ControlMaster flags (shell-quoted)
sac host add-peer / list-peers / remove-peer      # cross-host listen-bearer registry
sac host probe-hub                        # WSL → fleet-hub layered connectivity probe
sac peer post-turn AGENT TEXT             # A2A outbound (loopback or cross-host)
sac a2a serve <yamls...>                  # A2A inbound for non-SDK runtimes

# Fleet (peer-aware multi-agent orchestration)
sac fleet launch  PEER <name>...          # rsync specs to PEER, start each remotely
sac fleet notify  done|blocker|status --summary "..."   # agent→lead push (ADR-0013)

# Diagnostics / introspection
sac doctor [--fleet]                      # diagnose agent-spec source drift
sac subagent get-state                    # Claude Code Agent-tool subagent state

# Federated scheduled jobs (delegates to scitex-dev ecosystem)
sac dev systemd list / install / uninstall    # generate ~/.config/systemd/user/sac.*
sac dev cron    list / install / uninstall    # crontab entries
sac dev daemon  list / install / uninstall    # interactive daemons

# Misc
sac installation boot                     # first-time host bootstrap (venv, PATH, cron)
sac event ingest                          # Claude Code hook event ingestor
sac db   query / show / clean / export / import / migrate   # state.db inspection
sac registry sync [--from PEER | --to PEER | --all]   # cross-host comms_nodes anti-entropy
sac registry reconcile                    # singleton placement reconcile across fleet
sac --help-recursive                      # full subcommand tree

Part of SciTeX

scitex-agent-container is part of SciTeX. Install via the umbrella with pip install scitex[agent-container] to use as scitex.agent_container (Python) or scitex agent-container ... (CLI).

scitex-orochi adds cross-host message routing, a Slack-like chatops UI, and a peer registry on top of sac. The dependency is one-way — orochi reads sac's on-disk state; sac never imports orochi. For details, see docs/sac-and-orochi.md — architecture diagram, responsibility split, how to wire server:orochi-push.

Four Freedoms for Research

0. The freedom to run your research anywhere — your machine, your terms.
1. The freedom to study how every step works — from raw data to final manuscript.
2. The freedom to redistribute your workflows, not just your papers.
3. The freedom to modify any module and share improvements with the community.

AGPL-3.0 — because we believe research infrastructure deserves the same freedoms as the software it runs on.

---