VibeSOP

Engineering tools and empirical research for reliable AI-assisted development.

中文 · Documentation · Project status · Research

VibeSOP is a multi-agent AI engineering workflow system. It routes requests to the right skill or agent, verifies delivery against explicit criteria, and records observable execution evidence. Around that core it provides skill selection, governance over what may gate a release, and experience/knowledge accumulation across AI coding agents. The repository also contains experiments on when skills, specifications, orchestration, review, and memory improve the work—and when they add overhead.

SkillOS is the skill-management subsystem, not the whole project. VibeSOP also covers routing, verification, observability, governance, and experience/knowledge accumulation. Reliability is the objective; the presence of these tools does not establish an automatic or proven end-to-end software factory. See the project positioning.

Version and availability

The capabilities below describe this release. Local experiment data is not part of the installed package. Details and release evidence: project status.

What you can do

The hook path hands skill context to the host coding agent. Runtime, loop and validation tools have their own execution paths. Platform configuration support, hook support and end-to-end verification should be checked separately in the integration guide.

Quick start

Python 3.12+ is required. Install the public package with uv:

uv tool install vibesop
vibe --version
vibe quickstart

The routing demo uses a local lightweight path; LLM-enhanced routing requires a configured provider. Review the selected skill and plan before relying on it.

To work from source:

git clone https://github.com/nehcuh/vibesop-py.git
cd vibesop-py
uv sync --extra dev
uv run vibe --version
uv run vibe quickstart

Inside a source checkout, use uv run vibe in place of vibe to avoid accidentally invoking an older globally installed package.

Integrations

Generate configuration for the agent you use, then restart that agent:

These are configuration-generation targets, not a claim of identical runtime behavior across agents. Use vibe doctor and the platform-specific documentation to check your environment.

LLM configuration

For a CLI subprocess, configure a supported provider, for example:

export ANTHROPIC_API_KEY="your-key"
vibe route "help me debug this code"

An in-process integration can supply its host LLM through AgentRouter.set_llm(). Provider options and platform-specific setup are in the configuration guide and agent integration guide.

Workflow examples

vibe route "help me debug this code"
vibe plan list
vibe recall "configuration merge lost user hooks"
vibe loop list
vibe doctor

recall needs recorded traces and its embedding dependencies. Cross-project retrieval is explicit (--cross-project) and requires a populated pool. A blocked plan needs its reported problem resolved; it must not be treated as a completed or ready-to-run task. See the verification contract.

Routing evidence (source candidate 8.5.0)

vibe observe routing reports no-match, near-miss, and decision-source evidence from local route spans. It is report-only: it never edits the eval dataset, thresholds, or routing policy. Generate a hermetic eval payload, then observe spans against it:

# 1. Produce a fresh hermetic eval payload (near-miss over-injection evidence).
uv run python scripts/eval_routing.py --hermetic --json --json-out /tmp/eval-routing.json

# 2. Observe local route spans against it (reads .vibe/observability/spans.jsonl
#    by default; exit 4 means not enough scorable spans yet).
uv run vibe observe routing --eval-json /tmp/eval-routing.json --json

Spans come from real vibe route runs. See the operator runbook for metric definitions, thresholds, exit codes, and cron/CI wrappers.

For commands and realistic scenarios, see the CLI reference, command handbook, and use cases.

Research and engineering principles

Our experiments ask how specifications, skills, models and execution environments interact; whether more reviewers or fixed expert roles justify their cost; and whether stored experience produces useful future behavior.

• Research overview: findings, source material and limitations.
• Experiment registry: settled cohorts and unfinished v2 research.
• Research survey: the broader evidence record.
• Engineering methodology: proposed practices, with tested components distinguished from untested end-to-end hypotheses.
• Article collection: explanations for a broader audience.

We select skills when useful, define acceptance criteria, retain failures and interruptions, and separate model review from execution evidence. We do not infer universal gains from more skills, more agents, or more stored traces. Dataset, model, budget and measurement conditions belong beside each reported result.

Some raw runs live in a checksummed local archive and are not included in a Git clone or the wheel. Experiment evidence manifests describe their locations and restoration requirements. Research protocols and package releases have separate version histories.

Development

uv sync --extra dev
uv run ruff check src/ tests/
uv run ruff format --check src/ tests/
uv run basedpyright --level error
uv run pytest

A documented test command is not a claim that the current checkout passed it. Verification scope and dated evidence belong in the relevant change or report.

Start with the architecture guide, contribution guide, and current roadmap. The next release should reconcile source changes, migration notes and release checks; a positioning update alone does not justify a new version.

Documentation and project history

All documentation · Project status · Design principles · Changelog · Historical reviews · Workspace recovery

License and acknowledgments

MIT. VibeSOP integrates with community skill ecosystems including superpowers, oh-my-codex, and other installable packs. Skills and host agents retain their own authorship, licenses and runtime requirements.

Report issues and discuss the project on GitHub.