Metaproc

Generic process framework for structured multi-step agent workflows.

Metaproc executes dependency-aware process specs through local code handlers or agent CLI adapters. It provides planning, fan-out, resume, validation, status, trace, resource, credential-pool, and optional GCP Batch primitives while leaving domain schemas, prompts, handlers, and policies to downstream packages.

Features

• Markdown process specs with YAML frontmatter, typed inputs, dependencies, steps, outputs, fan-out, and composite processes
• Local Python and shell handlers plus Claude Code, Codex, Gemini, and pi CLI adapters
• Resumable DAG execution with fingerprints, completion validation, overrides, and structured .state and .logs artifacts
• RunPool concurrency, retry, resource-pressure, credential-pool, and ledger-backed resource observability controls
• Optional GCP Batch dispatch and Secret Manager integration
• A packaged Metabrowser plugin for process specs, plans, traces, logs, and resource reports
• Self-documenting: bundled manuals via metaproc help and a generated portable Agent Skill installed by metaproc skill

Installation

Metaproc requires Python 3.12 or newer and uses uv. Run an exact release without a persistent installation:

uvx metaproc@0.2.1 --help

For a persistent tool installation:

uv tool install metaproc
metaproc --help

Install the optional local browser integration with uv tool install 'metaproc[browser]'. Cloud dependencies are similarly isolated in the gcp and gcp-batch extras.

Metaproc currently supports Linux and macOS. Its process-control and resource-monitoring features require a POSIX operating system.

See installation for source-checkout, upgrade, and Agent Skill instructions.

Quickstart

Run the deterministic source-checkout example without an agent CLI, network call, or cloud credential:

make install
uv --config-file uv.toml run --frozen metaproc run-process \
  examples/offline-smoke/offline-smoke.process.md \
  --var RUNS_DIR="$(pwd)/.runs" \
  --var RUN_ID=quickstart

The process writes three outputs and its structured run state beneath .runs/quickstart/. Re-running it demonstrates completion caching; pass --force to execute every step again.

Client process specs may require additional variables before execution. The most common one is RUNS_DIR: an absolute path used to template output locations via {{run.parent_dir}}. Workflow launchers resolve RUNS_DIR from settings they own and pass the absolute value into run-process / run-step; Metaproc stays workflow-agnostic and does not synthesize it. The offline example passes the value explicitly. Template variables are case-sensitive; see conventions for the casing rules and the small set of framework built-ins.

Documentation

Metaproc documents itself: the manuals below ship inside the package, and everything else lives in docs/.

Start Here

Agents get the same routing automatically: metaproc skill metaproc --install writes a portable Agent Skill into .agents/skills/metaproc/ and .claude/skills/metaproc/ that delegates to these manuals.

Reference

Runbooks

Operational procedures live in docs/runbooks/:

Architecture

Architecture docs live in docs/arch/; the maintained index with status and ownership is development.md § Architecture Docs. arch-metaproc-core is the primary implementation reference; metaproc-design-rev3-proposals holds design proposals not yet implemented. The Metabrowser integration is split between the external MetaBrowser architecture and the Metaproc-owned plugin.

Contributing and Policies

Commands

The CLI is organized into a few families; metaproc --help lists every command, metaproc <command> --help documents each one, and metaproc help operator maps monitoring questions to commands.

Process Specs

Process specs define multi-step DAGs that run-process walks automatically. The repository ships provider-agnostic and per-adapter self-test processes:

See testing architecture for when to use each tier and how to set up per-adapter credentials. Downstream packages own their domain process specs, schemas, handlers, fixtures, and runbooks.

Working with a Coding Agent

This repository includes AGENTS.md, with the build, test, dependency, layout, and release conventions a coding agent needs for routine work. CLAUDE.md imports the same instructions for Claude Code.

For an ordinary change, tell your agent: “Read AGENTS.md, implement this change, and run the required checks.” For toolchain or template maintenance, ask it to use the simple-modern-uv skill, which distinguishes selective feature adoption from a full Copier update.

Development

The repository follows the simple-modern-uv structure and uses uv, Ruff, BasedPyright, pytest, Biome, TypeScript, Flowmark, and Lefthook:

make install
make format
make verify

make verify checks both locks, formatting, Python and browser lint, types, tests, dependency audits, public hygiene, source and wheel contents, and an isolated installed wheel. See development, agent instructions, and supply-chain security.

Compatibility

During the 0.x series, the command-line interface, process-spec format, documented plugin entry points, and Pydantic models explicitly linked from the architecture docs are the supported integration surfaces. Other Python imports are implementation details and may change between minor releases.

Cloud images do not pick up local source edits automatically; publish or upload a wheel and set both METAPROC_WHEEL_GCS and METAPROC_WHEEL_SHA256, or rebuild the downstream image. See cloud-dispatch.

License

Metaproc is AGPL-3.0-or-later; see LICENSE. If you modify Metaproc and let users interact with that modified version over a network, AGPL section 13 requires offering those users the corresponding source code for the running version.

The vendored ELK browser component is a separately licensed work; its license and distribution notice are listed in NOTICE.md.