Skip to main content

Audit AI skill safety — CLI + MCP server for static, semantic, and adversarial skill audits across Claude Code, Cursor, Codex, and OpenCode.

Project description

TAR Engine — audit AI skill safety from Claude Code, Cursor, Codex CLI

Audit AI skill safety without leaving your editor or your CI pipeline. TAR Engine exposes its skill audit pipeline two ways: a tar-engine CLI for CI, and an MCP server so any MCP-compatible agent — Claude Code, Cursor, Codex CLI — can call the audit pipeline directly from a conversation.

Five skill formats supported out of the box: SKILL.md, .claude/commands/*.md, Codex skill.yaml, manifest.json, OpenCode opencode.json.

What you get

Two surfaces ship in the same tar-engine package:

  • tar-engine — a one-shot CLI that walks a directory, audits every skill it finds, and exits with a CI-friendly status code. Drop it into GitHub Actions or a pre-commit hook to fail builds with risky skills.
  • tar-engine-mcp — the MCP server below (bundled), for interactive use from Claude Code / Cursor / Claude Desktop.

Both talk to the same backend (default https://tarai.dev, override with TAR_ENGINE_URL).

CLI quick start

# scan a directory of skills, fail the build if any score is below 70
tar-engine scan ./skills --min-score 70

# list discovered skills without running the audit
tar-engine list ./skills

# JSON output for downstream processing
tar-engine scan ./skills --json

Discovery covers five formats out of the box:

File pattern Format
**/SKILL.md OpenClaw, Claude Code, generic md
**/.claude/commands/*.md Claude Code custom commands
**/skill.yaml / .yml Codex
**/manifest.json Codex / Claude Code (key-detected)
**/opencode.json OpenCode

Each skill's audit payload includes its primary file plus sibling .sh / .py / .js / .ts / .yaml / .json helper files in the same directory tree (capped at 200 KB total). This catches the "SKILL.md looks clean but install.sh does the dirty work" pattern.

CI integration

GitHub Actions — fail the build if any skill scores below 70:

- name: Audit AI skills
  run: |
    pipx install tar-engine
    tar-engine scan ./skills --min-score 70 --format sarif -o results.sarif

- name: Upload SARIF to GitHub Security
  uses: github/codeql-action/upload-sarif@v2
  with:
    sarif_file: results.sarif

Pre-commit hook (.git/hooks/pre-commit):

#!/usr/bin/env bash
tar-engine scan ./skills --min-score 80 --strict || exit 1

The exit code is 0 on success, 1 when any skill is below the --min-score threshold or --strict finds any warning+ severity finding, and 2 for usage errors / missing path.

Optional defense in depth — if shellcheck / semgrep / trufflehog / gitleaks are installed locally, tar-engine scan orchestrates them alongside the backend audit pipeline and surfaces their findings under the same schema. Run tar-engine check-tools to see which are available and how to install the rest.


MCP tools (the LLM invokes these on its own when the user describes audit intent):

  • audit_skill_text(skill_text, lang?, domain?) — audit a SKILL.md you have in hand. Returns structured findings with rule_id, severity, line-level evidence, category, and remediation hints.
  • audit_skill_url(url, lang?, domain?) — fetch a SKILL.md from a URL and audit it.
  • list_audit_rules(category?, lang?) — list the rule registry the pipeline applies (PI-, SS-, FA-, DE-, CE-, MP-, SEM-, AR- prefixes).
  • get_audit_baseline(skill_name, limit?) — historical audit records and trend for a named skill.

Prompts (slash commands users can trigger explicitly):

  • audit-skill — audit a SKILL.md (paste text or URL)
  • audit-best-practices — the three defensive constraints every SKILL.md should include before publishing
  • audit-trend — show how a skill's audit score has changed over time

Configure your client

Claude Code

Add to ~/.config/claude-code/mcp.json:

{
  "mcpServers": {
    "tar-engine": {
      "command": "docker",
      "args": [
        "exec", "-i",
        "tar-engine-oss",
        "python", "/app/mcp-server/tar_engine_mcp.py"
      ]
    }
  }
}

Restart Claude Code. Available commands:

  • Type / to see audit-skill, audit-best-practices, audit-trend
  • Or just say "audit my SKILL.md" and Claude will call the right tool

Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json (macOS) or %APPDATA%\Claude\claude_desktop_config.json (Windows):

{
  "mcpServers": {
    "tar-engine": {
      "command": "docker",
      "args": [
        "exec", "-i",
        "tar-engine-oss",
        "python", "/app/mcp-server/tar_engine_mcp.py"
      ]
    }
  }
}

Cursor

Settings → MCP → Add Server → use the same docker exec command.

Run directly (without docker)

If you run the engine locally without docker:

TAR_ENGINE_URL=http://localhost:8765 \
  python -m pip install mcp httpx
TAR_ENGINE_URL=http://localhost:8765 \
  python mcp-server/tar_engine_mcp.py

Then in your client config, point command at this script directly.

Environment

The MCP server reads:

  • TAR_ENGINE_URL (default http://localhost:8765) — base URL of the running engine
  • TAR_ENGINE_TIMEOUT (default 180) — per-request timeout in seconds. Full-stack audit (L1 + L3 + L4) routinely takes 30-60s; default leaves headroom.
  • OPENAI_API_KEY / OPENAI_BASE_URL / OPENAI_MODEL — forwarded to the engine for the semantic (L3) and adversarial-fuzz (L4) layers. Without these the audit still runs L1 (regex) but skips L3/L4.

Example user flows

Audit a skill you're writing. Open Claude Code in a project that has a SKILL.md. Ask "audit this skill for security issues." Claude reads the file, calls audit_skill_text, and walks you through findings with line numbers and fix suggestions.

Check a community skill before installing. "Audit github.com/someone/cool-skill SKILL.md." Claude calls audit_skill_url with the raw URL, returns findings, you decide whether to install.

See your audit trend. After running audits on the same SKILL.md multiple times, "show me the trend for my-skill." Claude calls get_audit_baseline and reports mean/stddev/trend plus rule_ids that keep recurring.

Learn the defensive pattern. Type /audit-best-practices. Get the three-line SKILL.md prefix that closes the W23 edition's worst attack class (encoded-instruction injection that succeeded against 5 of 5 Anthropic-official skills).

Historical baseline persistence

Every audit triggered through the MCP server lands in the same cockpit_audit_history table the web endpoint writes to. The baseline you see via get_audit_baseline accumulates across all paths — MCP, CLI, web, direct HTTP. Same skill, same identity, one trend.

Set no_history: true in the audit body when you want a one-off audit that doesn't pollute the baseline (e.g. auditing a third-party skill you don't own).

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tar_engine-0.3.1.tar.gz (34.9 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

tar_engine-0.3.1-py3-none-any.whl (31.3 kB view details)

Uploaded Python 3

File details

Details for the file tar_engine-0.3.1.tar.gz.

File metadata

  • Download URL: tar_engine-0.3.1.tar.gz
  • Upload date:
  • Size: 34.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.6

File hashes

Hashes for tar_engine-0.3.1.tar.gz
Algorithm Hash digest
SHA256 b8b6fa69dcbc2ef6aa630525f5c3fbc205cea0ab668207a37c74a3e291df9a6c
MD5 020dd58d6b7bf42bd023b61344273726
BLAKE2b-256 2b76dbf911df6fd0710c4158a3400a3bc4c2ebdb16cc1bed3f0831831c77a4d6

See more details on using hashes here.

File details

Details for the file tar_engine-0.3.1-py3-none-any.whl.

File metadata

  • Download URL: tar_engine-0.3.1-py3-none-any.whl
  • Upload date:
  • Size: 31.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.6

File hashes

Hashes for tar_engine-0.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 e871cb7045726aede53e6a624d2540acea01fc2e7383d4d726375e1b243bb9e9
MD5 7b63df46a82880cc288b409b7498bcea
BLAKE2b-256 c810f0aec6ab1dcaead97c437a77e55b1d268d6312d669d5a7ded68d32b0c73b

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page