harness-eval

A linter for AI code agent setups, not for code. It auto-detects which AI tools a project uses (Claude Code, Cursor, Windsurf, Cline, Copilot, Gemini CLI, OpenCode), builds a component graph across all of them, and runs 92 deterministic rules to catch issues that per-file linters miss: credential exfiltration chains, confused deputy attacks, skill/hook conflicts, and token budget blowouts.

Most tools test whether a skill produces correct output. This one checks the setup itself: CLAUDE.md, GEMINI.md, AGENTS.md, skills, commands, hooks, MCP configs, agents, .cursor/rules/*.mdc, .cursorrules, .github/prompts/, .opencode/.

Quick start

pip install harness-eval
harness-eval lint .          # 92 deterministic rules, fully offline
harness-eval security .      # security scan (18 rules)

See docs/INSTALL.md for all installation options and configuration.

How to use it

Available as a CLI tool, a GitHub Action, a Tekton Task (OpenShift Pipelines), a Claude Code plugin, and Cursor commands. Each is documented in docs/INSTALL.md.

Cross-component analysis

This is the core differentiator. Most linters check files in isolation. harness-eval builds a component graph that traces data flows across skills, agents, hooks, and MCP servers, then runs cross-component rules against it. This catches classes of issues that per-file analysis cannot:

• A hook reads credentials from env, passes them to a skill, which forwards them to an MCP server with broad network access
• A command's allowed_tools list doesn't cover the tools its instructions actually use
• Settings.json permissions.deny blocks a tool that CLAUDE.md instructs the agent to use
• Two assistants' instruction files (CLAUDE.md and GEMINI.md) have drifted apart
• A skill is defined but never referenced from any instruction file (orphan)

Multi-tool projects are fully supported. When a project uses both Claude Code and Cursor, all components are evaluated together.

Supported AI tools

Inspection rules

92 deterministic rules across 11 categories: structural, frontmatter, content, quality, security, cross-component, commands, CLAUDE.md, MCP, hooks, and agents. Four presets: recommended (default), strict, security, pre-workflow.

For the complete rule list with examples, detection techniques, and framework mappings (OWASP, MITRE ATLAS), see docs/rules-reference.md.

Privacy

lint and security (without --review) are fully offline. LLM review is opt-in: only review, security --review, and skill --rubric send snippets to a remote provider (Gemini or Anthropic via CLI, or in-session as a plugin/command).

Before any remote LLM call, likely secrets (tokens, PEM keys, API_KEY= assignments, known prefix patterns) are replaced with [REDACTED] (HE-2). Scans also skip .env, credentials paths, and *.pem / *.key / id_rsa globs by default (HE-3); add more with --exclude.

See docs/how-can-you-know-its-safe-to-use-this-tool.md for details.

Contributing

See CONTRIBUTING.md for adding rules and submitting PRs.

Changelog

See CHANGELOG.md for release history.

Roadmap

See open issues for planned improvements and feature requests.