Kartikeya

Skanda · Murugan — the six-faced commander of the divine armies, born to lead the devas. The engine that marshals and runs a fleet's tasks. Colloquially: Kart.

A standalone, host-agnostic task queue + sandboxed worker. Submit shell (or, optionally, LLM-workflow) tasks to a queue; a worker claims them and runs each in a bubblewrap sandbox with an explicit mount/credential/network policy.

Kartikeya is the execution engine extracted from the Willow fleet (willow-2.0/core/kart_* — historical path) and made to stand on its own — no fleet, no specific host, no required database server.

Status

Extracted and published. The sandbox/worker/execute core is fully landed (sandbox.py, worker.py, execute.py, queue.py), tested, and released on PyPI as kartikeya — pip install kartikeya gets whatever the latest tag published (see .release-please-manifest.json for the version this tree currently reports), or pip install willow-mcp, which depends on it and floors it at whatever range willow-mcp's own pyproject.toml currently pins — naming the number here would only go stale the next time either project tags a release. The cap is a real compatibility range rather than decoration: bump-minor-pre-major is false here, so a breaking change cuts 1.0.0 instead of hiding in a minor. The staged lift in docs/DESIGN.md is done through stage 4; stage 5 (legacy fleet monolith deleting its core/kart_* copy) is tracked in willow-mcp#111.

Design goals

• Host-agnostic. The only coupling — "where do tasks live" — is a small TaskQueue interface a host implements. Kartikeya owns the sandbox, worker loop, lanes, and command scan; the host owns storage and file roots.
• Zero-infra by default. Ships a reference SqliteTaskQueue, so pip install kartikeya can execute tasks with no Postgres and no fleet.
• Backend-swappable. SQLite (bundled), Postgres, or a custom backend behind the same interface.
• Sandboxed and network-gated. Tasks run network-isolated unless the stored task text carries a # allow_net directive; inference credentials reach only network-enabled tasks. GitHub credentials never enter the sandbox on any mode: a task that needs to push asks the host, which holds the key and performs the push under its own authorization. (Who is allowed to write that directive is the host's call — see the security note in docs/DESIGN.md.)

Install

pip install kartikeya            # base: shell tasks, SQLite backend
pip install "kartikeya[postgres]"  # + Postgres backend helpers
pip install "kartikeya[llm]"       # + LLM-workflow task type

Quickstart

Resource caps (memory + PID limits) prefer a delegated cgroup parent. On a fresh install, run once:

kartikeya setup-cgroup    # installs ~/.config/systemd/user/kart.slice
kartikeya cgroup-status   # should print ready: /sys/fs/cgroup/...

systemd worker (operative): a shell profile export does not reach a user-service kart worker. After setup-cgroup succeeds, wire the parent into the worker environment and restart:

systemctl --user set-environment KART_CGROUP_PARENT=/sys/fs/cgroup/.../kart.slice
systemctl --user restart <your-kart-worker-unit>

(Or set Environment=KART_CGROUP_PARENT=... in a drop-in for the worker unit.)

The export line printed by setup-cgroup is for CLI / interactive runs only. Without a delegated parent, Kart falls back to task-scoped prlimit/ulimit inside the sandbox (PID cap by default; virtual-memory cap only with KART_RLIMIT_USE_AS=1). WILLOW_KART_NO_RLIMIT=1 disables caps entirely (escape hatch only).

With a delegated parent, each task gets its own cgroup leaf. If that leaf cannot be created, configured or joined, the task is refused (cgroup_setup_failed / cgroup_join_failed) rather than run without its memory cap.

Concurrency per worker process: the fast lane runs up to KART_FAST_WORKERS tasks at once (default 3) and the batch lane up to KART_BATCH_WORKERS (default 1, since batch work is long and heavy). kartikeya worker --slots N overrides either. Like KART_CGROUP_PARENT, set these in the worker unit's Environment=, then restart it.

Coming with stage 2 — once the worker core lands, this section documents kartikeya worker end to end (submit → worker runs → poll).

License

MIT © Sean Campbell