skillsaw-runbooks

A skillsaw plugin that lints operational runbooks as agent content. Runbooks are step-by-step procedures executed under pressure — increasingly by AI agents as well as on-call engineers — so they deserve the same lint coverage as any other instruction file.

This plugin also serves as a complete, published example of every skillsaw plugin extension point: rules, a custom repository type with content paths, a lint tree contributor, and a plugin CLI.

What it does

When a repository has markdown files under runbooks/, the plugin:

• detects the runbook repository type, which activates the rules below and pulls runbooks/**/*.md into skillsaw's content linting (so every builtin content-* rule covers your runbooks too);
• attaches runbooks/catalog.json (if present) to the lint tree so the catalog rule can check it.

Rules

A passing runbook:

---
title: Database failover to the replica
owner: storage-team
---

# Database failover

## Steps

1. Freeze writes ...

## Rollback

Repeat with the old primary as the candidate ...

A failing one — no owner, no Rollback section:

---
title: Flush the Redis cache
---

# Cache flush

## Steps

1. redis-cli --scan --pattern 'rates:*' | xargs redis-cli del

$ skillsaw lint
⚠ WARNING (runbook-required-sections) [runbooks/cache-flush.md]: Runbook is missing a 'Rollback' section
⚠ WARNING (runbook-frontmatter) [runbooks/cache-flush.md]: Runbook frontmatter is missing 'owner'

Catalog

Optionally keep an index of your runbooks in runbooks/catalog.json:

{
    "runbooks": [
        "db-failover.md",
        "cache-flush.md"
    ]
}

Paths are relative to runbooks/. The runbook-catalog rule warns when a runbook on disk is missing from the catalog or a catalog entry has no file. Without a catalog file the rule stays quiet — it is opt-in by presence.

Install

$ pip install skillsaw-runbooks
$ skillsaw lint

That's it — skillsaw discovers the plugin automatically. Requires a skillsaw new enough to support plugins (>= 0.15).

CLI

The plugin ships a small companion CLI, dispatched through skillsaw:

$ skillsaw runbooks list
runbooks/db-failover.md: Database failover to the replica — storage-team
runbooks/cache-flush.md: Flush the Redis cache — (no owner)

$ skillsaw runbooks rules
runbook-required-sections — Runbooks must contain the required sections (default: Steps, Rollback)
...

Configuration

Configure rules by ID in .skillsaw.yaml, like any builtin rule:

rules:
  runbook-required-sections:
    severity: error
    sections: ["Steps", "Rollback", "Verification"]
  runbook-frontmatter:
    fields: ["title", "owner", "last-reviewed"]

Disable a single rule, the whole plugin, or all plugins:

rules:
  runbook-catalog:
    enabled: false

plugins:
  disable: [runbooks]

$ skillsaw lint --no-plugins

Development

$ python -m venv .venv
$ .venv/bin/pip install -e '.[dev]'
$ .venv/bin/pytest

To develop against an unreleased skillsaw, install it first: .venv/bin/pip install -e ../skillsaw.

License

Apache-2.0