doodle-lint 1.2.0

Measures whether your agent skill actually fires. Plus 16 static rules for the obvious stuff.

---

Overview

SKILL.md is the Agent Skills open standard, the format Claude Code, VS Code, Codex, Cursor, Gemini CLI and 30-odd other tools read to extend an agent with custom capabilities.

Most tooling for it checks whether a skill file is well-formed. doodle does that too, but the more interesting question is whether the skill will ever be selected. Agents pick skills by matching the user's request against skill descriptions, so a description can be perfectly formed and still lose every match:

description: Performs heuristic evaluation of committed changesets.
user types:  "check my code"
overlap:     0%

Nothing there is wrong. It just never runs, and a conventional linter reports the file as clean.

doodle measures that gap two ways:

• doodle surface: offline. Compares your description against realistic user phrasings and names the words you're missing. No API key.
• doodle eval: empirical. Runs prompts through the model and reports how often the skill actually fires.

Across 77 published skills, descriptions shared a mean of 24% of their vocabulary with realistic requests, and description length explained under 8% of the variance. See the Trigger Surface Report.

Plus 16 static rules, each citing the authoring guide or a documented community issue, for the conventional checks.

Install

pip install doodle-lint

Requires Python 3.10 or newer. Runtime dependencies: PyYAML, pyspellchecker, and tomli on Python 3.10. PyPI publication is pending; once complete, pip install doodle-lint will work.

Verify:

doodle --version
doodle --list-rules

Usage

# Scaffold a new skill that passes the linter out of the box
doodle init my-skill --eval          # creates ./my-skill/{SKILL.md,eval.yaml}

# Measure trigger surface: does the description use the words users type?
doodle surface path/to/SKILL.md

# Same analysis with embedding similarity (catches paraphrases lexical overlap misses)
# Requires: pip install "doodle-lint[semantic]"
doodle surface path/to/SKILL.md --semantic

# Lint a single skill
doodle path/to/SKILL.md

# Lint a directory recursively
doodle ./skills

# JSON output for CI
doodle ./skills --format=json

# SARIF output for GitHub code scanning
doodle ./skills --format=sarif > doodle.sarif

# Apply auto-fixes for safe rules
doodle ./skills --fix

# Promote info to warning, warning to error
doodle --strict ./skills

# Disable a specific rule
doodle --ignore=body/emoji ./skills

# Explain a rule
doodle --explain desc/vague-trigger

Exit codes: 0 clean, 1 warnings, 2 errors, 3 tool error.

Phase 2: trigger-accuracy eval

# Requires: pip install ".[eval]", npm install -g promptfoo, ANTHROPIC_API_KEY
doodle eval --generate path/to/SKILL.md    # Draft a starter eval.yaml via Claude
doodle eval path/to/SKILL.md               # Run the eval, report the score
doodle eval path/to/SKILL.md --dry-run     # Preview the Promptfoo config

Full workflow: docs/EVAL.md.

VS Code / Cursor extension

The extension is published on the Open VSX Registry and works in Cursor, VSCodium, Windsurf, and VS Code with Open VSX enabled.

Features: real-time diagnostics, status bar with finding counts, hover messages linking to the rule catalog, quick-fix code actions for fixable rules, and a doodle eval bridge command.

Install from your editor's Extensions panel (search doodle), or via CLI:

# Cursor, VSCodium, Windsurf (Open VSX is the default gallery)
code --install-extension krishyaid-coder.doodle-lint

# Vanilla VS Code: install the VSIX from the latest GitHub Release
curl -L -o /tmp/doodle.vsix \
  https://github.com/krishyaid-coder/doodle/releases/latest/download/doodle-lint-0.2.0.vsix
code --install-extension /tmp/doodle.vsix

The extension shells out to the CLI installed above. Upgrading the CLI upgrades the extension's rules automatically.

Full extension docs: vscode/README.md.

Pre-commit hook

If your project uses pre-commit, add doodle in four lines:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/krishyaid-coder/doodle
    rev: v1.0.0
    hooks:
      - id: doodle

Then:

pre-commit install
git commit -m "test"    # doodle runs automatically on any changed SKILL.md files

Two hooks are provided:

• doodle: runs by default on every commit, blocks commits that produce warnings or errors
• doodle-fix: runs manually via pre-commit run doodle-fix --all-files, applies safe auto-fixes and stages the results

CI (GitHub Action)

- uses: krishyaid-coder/doodle@v0
  with:
    path: ./skills
    strict: true
    fail-on: warning   # warning | error | never

Rules

Each rule cites either Anthropic's authoring documentation or a documented community issue. Full spec with examples, in-sample frequency, and citations: RULES.md.

Starter templates

Category-specific templates for common skill types are shipped in eval-suites/ and wired into doodle init:

doodle init --list-templates
doodle init my-reviewer --template=code-reviewer --eval

The scaffold pairs a tuned SKILL.md description (passes the linter out of the box) with a matching eval.yaml containing category-appropriate should_fire and should_not_fire prompts. Available templates:

Template	For skills that...
code-reviewer	Review diffs, PRs, staged changes for correctness and security
refactorer	Restructure code without changing observable behavior
sql-generator	Write SQL from natural-language analytics questions
docs-writer	Produce READMEs, API references, and docstrings
test-writer	Author unit, integration, and end-to-end tests
security-auditor	Audit code for OWASP issues, injection risks, and CVEs
debugger	Investigate errors and find root causes
data-engineer	Design ETL pipelines, dbt models, Airflow DAGs
api-designer	Design REST endpoints, GraphQL schemas, service contracts
skill-creator	Author or improve Claude skills

Full contributor guide for new templates: eval-suites/README.md.

Quality badge

Skill authors can advertise the doodle grade of their SKILL.md in the skill's own README:

doodle badge path/to/SKILL.md

Prints a shields.io-backed markdown snippet, for example:

[![doodle A](https://img.shields.io/badge/doodle-A-green?style=flat-square)](https://github.com/krishyaid-coder/doodle)

Grade rubric:

Grade	Meaning
A+	Zero findings
A	Info-level findings only
B	1 or 2 warnings, no errors
C	3 or 4 warnings, no errors
D	5 or more warnings, no errors
F	Any error (skill will misload or fail to trigger)

The badge reflects the same rules doodle lint would apply today, respecting your .doodle.toml overrides. Other formats:

doodle badge SKILL.md --format=url    # just the shields.io URL
doodle badge SKILL.md --format=text   # just the grade letter
doodle badge SKILL.md --format=json   # structured, includes finding counts
doodle badge SKILL.md --link=https://github.com/you/your-skills   # override the badge target

Rule	Severity	Fixable	Description
desc/too-long	warning	no	Description longer than 250 characters
desc/too-short	warning	no	Description shorter than 60 characters or missing
desc/no-trigger-phrase	warning	no	No explicit "Use when" or "Trigger with" phrasing
desc/vague-trigger	warning	no	Trigger overlaps Claude's default behavior
desc/weak-trigger-surface	info	no	Description doesn't use the vocabulary users type
desc/typo	info (off by default)	no	Description contains a likely misspelling
body/too-long	warning	no	Body longer than 500 lines
body/way-too-long	error	no	Body longer than 1500 lines
body/absolute-user-path	warning	no	Contains /Users/, /home/, or ~/ outside fences
body/emoji	info (off by default)	yes	Emoji in body. Enable via --strict or config
fm/name-mismatch-dir	warning	no	Frontmatter name doesn't match parent directory
fm/missing-allowed-tools	warning	no	Extended dialect uses tools but omits scoping
fm/unknown-field	info	no	Anthropic dialect has non-standard field
hygiene/desc-blank-lines	info	yes	Description contains embedded blank lines
hygiene/trailing-whitespace	info	yes	Line has trailing whitespace
hygiene/final-newline	info	yes	File does not end with a newline

Architecture

One small Python package. Data flow: files → parser → rule registry → severity gate → formatter.

flowchart LR
    A[SKILL.md files] -->|read| B[Parser]
    B -->|ParsedSkill| C[Rule registry]
    Cfg[.doodle.toml] -->|custom rules<br/>+ overrides| C
    C -->|Finding stream| D[Severity gate]
    D -->|filtered| E{Formatter}
    E -->|text| F[stdout]
    E -->|json / sarif| G[CI / dashboard]

Components:

• parser: splits frontmatter from body and auto-detects the dialect.
• rule registry: runs built-in and custom rules, applies severity overrides, filters by dialect and per-path globs.
• custom rules: pattern and frontmatter-required rules loaded from .doodle.toml.
• formatter: renders findings as colored text, JSON, or SARIF.

Full component reference, sequence diagrams, and extension points: docs/ARCHITECTURE.md.

Dialects

Two SKILL.md dialects exist in the wild. doodle auto-detects.

• anthropic: minimal frontmatter (name, description, optional license). Used by anthropics/skills.
• extended: community schema with version, author, tags, allowed-tools. Used by alirezarezvani/claude-skills, jeremylongshore/claude-code-plugins-plus-skills, and DietrichGebert/ponytail.

Some rules apply to both. Others are dialect-scoped.

Custom rules (teams and enterprise)

Drop a .doodle.toml in your project root. No Python required for regex-based rules or frontmatter requirements.

[options]
dialect = "extended"
fail-on = "warning"
# Opt-in: desc/weak-trigger-surface uses embeddings (needs doodle-lint[semantic]).
# Default is lexical: fast, offline, right for pre-commit/CI.
# semantic = true

[severity]
"body/emoji" = "off"
"body/too-long" = "info"

[[paths]]
glob = "**/experiments/**/SKILL.md"
disabled = ["desc/vague-trigger", "body/too-long"]

[[rules]]
id = "acme/no-customer-pii"
kind = "pattern"
pattern = "(?i)\\bcustomer_[a-z]+\\b"
applies-to = "body"
severity = "error"
message = "Customer PII tokens are not allowed in skills."
suggestion = "Use 'user_<role>' instead."

[[rules]]
id = "acme/require-team-tag"
kind = "frontmatter-required"
fields = ["team", "data-classification"]
severity = "error"
message = "Internal skills must declare team and data-classification."

Config is discovered by walking up the directory tree. Force a path with --config. For rules requiring Python logic, see docs/EXTENDING.md.

Roadmap

Version	Feature	Status
v0.1	Static linter, 12 rules, CLI, GitHub Action	shipped
v0.2	.doodle.toml config, custom rules, per-path overrides, severity overrides	shipped
v0.3	--fix, SARIF output, parse-error suggestions	shipped
v0.4	doodle eval trigger-accuracy harness, --generate starter eval suites	shipped
vscode 0.2	VS Code extension on Open VSX Registry	shipped
v0.5	doodle init skill scaffold that passes the linter out of the box	shipped
v0.6	desc/typo spelling check backed by pyspellchecker, with a curated allowlist for AI vocabulary	shipped
v0.7	doodle badge quality badges (shields.io-backed, A+/A/B/C/D/F rubric)	shipped
v0.8	Two new hygiene rules with auto-fix (trailing-whitespace, final-newline)	shipped
Docs (July)	Quality Report refreshed to 200 skills across 6 repos	shipped
v0.9	Community eval-suite library: 10 category templates (code-reviewer, refactorer, sql-generator, docs-writer, test-writer, security-auditor, debugger, data-engineer, api-designer, skill-creator)	shipped
v1.0	Stable release. PyPI published. Pre-commit hook integration.	shipped
v1.1	doodle surface: offline trigger-surface analysis, plus the Trigger Surface Report on 77 published skills	shipped
Phase 4	Managed scanner and quality-badge dashboard for marketplace operators (paid service)	exploring, waits for demand
Phase 5	LSP extraction for Neovim, Zed, JetBrains via the same rule engine	if demand

Validation

I ran doodle against 200 published SKILL.md files across six repositories: DietrichGebert/ponytail, anthropics/skills, obra/superpowers, vercel-labs/skills, alirezarezvani/claude-skills, and jeremylongshore/claude-code-plugins-plus-skills. 81 percent of the sample had at least one quality finding. The rate is statistically identical to the June edition's 62-skill sample (82 percent), confirming the signal is not sampling artifact.

Standout results: obra/superpowers is 86 percent clean; anthropic first-party is 6 percent clean (1 of 18); ponytail is 0 for 6.

Full methodology, per-repository breakdown, and raw data: docs/QUALITY_REPORT.md.

Documentation

• Trigger Surface Report: do published descriptions use the words users type? Methodology, raw data, and honest limitations
• Quality Report: static-rule findings across 200 published skills, with methodology and raw data
• Architecture: diagrams, components, extension points, trade-offs
• Rule spec: every rule with citation, example, and in-sample frequency
• Extending: add a rule in Python or via .doodle.toml
• Eval guide: Phase 2 trigger-accuracy workflow
• VS Code extension: editor integration reference
• Starter templates: community eval-suite library, one entry per skill category
• Why doodle: impact argument, honest risk assessment
• Contributing: ground rules and PR checklist

License

MIT. The CLI and rule-set are MIT-licensed permanently. Any future hosted services will be a separate repository under a separate license.