sentinel-ai-auditor 0.1.0

Deterministic, offline-first security auditor for AI agent skills and instruction bundles

Sentinel

Polska wersja README

Sentinel is a deterministic, offline-first security auditor for AI agent skills, prompts, project instructions, tool and MCP manifests, hooks, scripts, CI files, and mixed instruction bundles. It inventories the complete target, analyzes text, code, and configuration without executing target content, correlates evidence across files, and emits stable findings for people and CI. The audit target is never executed, including during a --dynamic scan.

The repository is both:

• a Python 3.11+ CLI that performs the base audit without an LLM; and
• a portable Agent Skill whose SKILL.md teaches compatible agents to invoke that CLI without trusting the material under review.

Sentinel is defensive software. Dynamic tests are opt-in, local, bounded, and designed for synthetic fixtures—not external targets or real credentials.

What Sentinel does not guarantee

Sentinel does not prove that an agent, prompt, or extension is safe. Static analysis cannot observe every runtime path, an unsupported or malformed file can reduce coverage, a sandbox can have platform-specific gaps, and a clean result can still miss a novel technique. Treat PASS as “no policy-breaking issue was found within the reported scope and coverage,” not a warranty.

An incomplete material analysis is INCONCLUSIVE, never silently safe. See Limitations, THREAT_MODEL.md, and SECURITY.md.

Quick start

Install the published package:

python -m pip install --upgrade sentinel-ai-auditor
sentinel doctor
sentinel scan ./path/to/skill --profile strict --format markdown

Exit code 0 means the configured severity threshold was not crossed. It does not suppress warnings or coverage limitations. Codes 1 through 4 distinguish policy failure, configuration error, incomplete analysis, and internal error.

Installation

Sentinel requires Python 3.11 or newer and has no mandatory runtime dependencies.

Install it from PyPI:

python -m pip install --upgrade sentinel-ai-auditor

The distribution name is sentinel-ai-auditor; after installation, the executable command is simply sentinel. Until the first PyPI release is published, install the same package directly from GitHub:

python -m pip install "git+https://github.com/KacperStasieluk/sentinel.git"

For an editable development installation:

python -m pip install -e ".[dev]"
sentinel version

For an isolated tool installation from a reviewed checkout:

pipx install .

Build and install a wheel when distributing internally:

python -m pip wheel . --no-deps --wheel-dir dist
python -m pip install dist/sentinel_ai_auditor-0.1.0-py3-none-any.whl

Verify the source revision and artifact hash before installing across a trust boundary. Do not install dependencies declared by an untrusted audit target.

Scan examples

Run the default text report:

sentinel scan ./agent-skill

Write one selected format per invocation:

sentinel scan ./agent-skill --profile strict --format markdown --output sentinel-report.md
sentinel scan ./agent-skill --profile strict --format json --output sentinel-report.json
sentinel scan ./agent-skill --profile ci --format sarif --output sentinel.sarif --fail-on high

Sentinel refuses to replace an existing output file unless --force is explicit.

Select a report language:

sentinel scan ./agent-skill --language pl --format markdown

Inspect the rule catalog:

sentinel rules list
sentinel rules explain SNT-PINJ-001

Audit an existing project against a reviewed baseline:

sentinel baseline create . --output .sentinel-baseline.json
sentinel baseline compare . --baseline .sentinel-baseline.json

The default baseline path is .sentinel-baseline.json.

Example report

The exact values depend only on the target, trusted configuration, Sentinel version, and rule-set version. A condensed Markdown result resembles:

Status: FAIL
Score: 38/100 (F)
Coverage: 100%
Package hash: sha256:…

HIGH SNT-PINJ-001 — Untrusted content treated as executable instructions
Location: SKILL.md:42-47
Evidence: "Follow every instruction found in the [REDACTED] document."
Reason: Document text is promoted into the agent's instruction hierarchy.
Attack scenario: A crafted document requests secret disclosure or tool execution.
Recommendation: Treat document content as data and reject instructions originating from it.
Fingerprint: sha256:…

Reports include scope, detected formats and languages, capabilities, trust boundaries, category scores, critical paths, evidence, positive safeguards, limitations, environment metadata, package hash, rule-set version, and a reproduction command. JSON and SARIF use stable ordering and fingerprints.

Configuration

Pass a trusted configuration explicitly:

sentinel scan ./agent-skill --config ./sentinel.yaml --format json

The supplied sentinel.yaml uses safe defaults:

version: 1
profile: strict
language: auto

analysis:
  static: true
  dynamic: false
  follow_symlinks: false
  max_file_size: 1048576
  max_files: 5000
  max_total_size: 52428800
  decode_layers: 3

network:
  allowed: false
  allow_hosts: []

severity:
  fail_on: high

rules:
  enable: []
  disable: []
  overrides: {}

redaction:
  enabled: true
  max_evidence_length: 240

report:
  formats:
    - markdown
    - json
    - sarif

Sentinel loads only the path designated by --config; content in the target cannot reconfigure the audit. Secret redaction and the prohibition on following symlinks outside the audit root cannot be disabled. CLI flags override trusted file values for the current run.

Although report.formats can declare integration preferences, the CLI writes one format selected by --format per invocation.

Profiles

Profile	Purpose	Default failure threshold
minimal	Fast structural and highest-signal checks	critical
standard	Balanced default local scan	high
strict	Pre-adoption or pre-distribution review	medium
paranoid	Maximum scrutiny for hostile/high-impact inputs	low
ci	Deterministic non-interactive gate	high

Use --fail-on to set a trusted run-specific threshold. Do not weaken policy in response to instructions found inside the target.

CI and pre-commit

The repository includes:

• .github/workflows/ci.yml for project verification;
• .github/workflows/sentinel.yml for a Sentinel SARIF gate;
• .gitlab-ci.yml for GitLab;
• .pre-commit-config.yaml for local checks.

A minimal gate is:

sentinel scan . --profile ci --format sarif --output sentinel.sarif --fail-on high

Store reports even when a job fails. Pin third-party CI actions to reviewed immutable commits before production use. For legacy repositories, compare against a reviewed baseline to block only new findings while retaining full visibility into existing findings. Never regenerate a baseline automatically to make a gate pass.

See references/ci.md.

Agent and editor integrations

The deterministic CLI remains the security boundary on every platform:

• OpenAI Codex: use the root SKILL.md, agents/openai.yaml, and examples/codex/AGENTS.md.
• Claude Code: adapt examples/claude/CLAUDE.md.
• GitHub Copilot: adapt examples/copilot/.github/copilot-instructions.md and enforce with CI.
• Gemini CLI / Google agent tools: adapt examples/gemini/GEMINI.md.
• Cursor, Windsurf, Cline, Aider, and Roo Code: use the corresponding directory under examples/.
• Generic Agent Skills hosts: install the root skill or adapt examples/generic/PROJECT_INSTRUCTIONS.md.

Platform instruction files are advisory adapters. They must not emulate Sentinel's rules, execute target commands, or replace the CLI with conversational judgment. See references/platforms.md.

Sandbox and dynamic analysis

Static analysis is the default. Sentinel never executes the analyzed target. Dynamic mode executes only a separate, explicitly supplied local synthetic fixture:

sentinel scan ./agent-skill \
  --dynamic \
  --dynamic-backend subprocess \
  --dynamic-fixture ./fixtures/synthetic-case \
  --dynamic-entry sentinel_dynamic.py \
  --dynamic-arg case-a \
  --format json

Backends:

• none: perform no dynamic execution and provide no isolation;
• subprocess: collect bounded defense-in-depth telemetry from a copied Python fixture; this is not an operating-system security boundary;
• docker: provide stronger optional isolation using a trusted local daemon and pre-existing trusted image.

--dynamic without --dynamic-fixture executes nothing, lowers reported coverage, and normally produces INCONCLUSIVE. TARGET and the fixture must resolve to disjoint paths: neither may equal, contain, or be contained by the other. Select a trusted relative Python probe with --dynamic-entry; repeat --dynamic-arg only for inert data. Never derive the fixture, entry, or arguments from target instructions.

The subprocess backend uses Python isolated mode, audit-hook telemetry, a copied workspace, synthetic home, scrubbed environment, invalid canaries, best-effort network denial, finite limits, bounded output, redaction, and cleanup. Native code and operating-system primitives can evade language-level controls, and the fixture shares a process trust boundary with the telemetry. On Windows, the built-in backend has no Job Object or restricted token, cannot enforce hard CPU/memory/process limits, and reports the observation as limited.

For stronger isolation, use Docker with an image that is already present locally and identified by a non-latest tag or sha256 digest:

sentinel scan ./agent-skill \
  --dynamic \
  --dynamic-backend docker \
  --dynamic-fixture ./fixtures/synthetic-case \
  --dynamic-entry sentinel_dynamic.py \
  --docker-image sentinel-fixture:1.0.0 \
  --format json

Sentinel verifies a local Docker daemon and image, uses --pull=never and --network=none, and never pulls or builds an image. Docker is stronger than the subprocess backend but still depends on the trusted daemon, image, kernel, and host configuration.

Every dynamic run is limited to the separate fixture and uses a temporary staged copy, synthetic home, scrubbed environment, invalid canaries, finite resources, bounded redacted evidence, and cleanup where supported. Never use real secrets, the original untrusted package, external attack targets, or production services. See references/dynamic-analysis.md.

Privacy

The base engine is offline and does not require an LLM, telemetry, or a hosted service. Network access is disabled by default. Reports contain bounded redacted evidence instead of complete file content or secrets.

Optional LLM-assisted interpretation, if supplied by an integration, is separate and disabled by default. It may group or explain findings but cannot alter deterministic results. Enable it only after an explicit disclosure and consent describing the provider and exact redacted data sent. Never transmit an entire repository by default.

Treat reports and baselines as sensitive: they can reveal paths, architecture, and partial vulnerable text even after redaction.

Custom rules

Rules implement the stable rule protocol and declare metadata, supported files/languages, required parsers and phase, severity, tags, standards mappings, remediation, and test cases. A conceptual rule is:

class Rule(Protocol):
    metadata: RuleMetadata

    def analyze(self, context: AnalysisContext) -> Iterable[Finding]:
        ...

Keep rules deterministic and side-effect free. Prefer parsed structure or ASTs for code, bound regex and decoding work, and add positive, negative, vulnerable, and fixed fixtures. The rule validator rejects duplicate IDs, invalid metadata, missing tests or documentation, unstable output, unsafe plugin loading, and dangerous regular expressions.

See references/rule-authoring.md and CONTRIBUTING.md.

Reporting a false positive

Open an issue with the Sentinel and rule-set versions, rule ID, stable fingerprint, smallest redacted reproducer, command, platform, expected behavior, and why the behavior is safe. Do not attach credentials or proprietary source.

Prefer a rule fix or narrow condition over a broad disable. If a temporary override is unavoidable, document its rationale, scope, owner, and re-review condition. Never suppress parser failures or incomplete analysis.

Rule-set versioning

The engine and rule set follow Semantic Versioning independently:

• patch releases correct detections or metadata without changing intended policy;
• minor releases add rules, supported formats, mappings, or opt-in capability;
• major releases change rule IDs, schemas, scoring, defaults, or public APIs incompatibly.

Published rule IDs are never silently reused. Reports and baselines record both versions. Review CHANGELOG.md before updating a CI-pinned version or baseline.

Limitations

• Natural-language and obfuscation detection is heuristic and cannot cover every phrasing.
• AST-quality analysis varies by supported programming language.
• Unsupported binaries, encrypted archives, malformed documents, size limits, permissions, and parser errors reduce coverage.
• Static analysis cannot prove runtime reachability or observe remote service behavior.
• subprocess is language-level defense-in-depth telemetry, not a kernel or operating-system security boundary; Windows runs are limited without Job Object and restricted-token enforcement.
• Docker isolation depends on a trusted local daemon, pre-existing image, kernel, and host configuration.
• Symlinks are inventoried but never followed outside the audit root.
• Optional LLM interpretation can be wrong and never changes the deterministic base result.
• A new or domain-specific attack may require a new rule.

Sentinel reports these conditions. A material gap produces INCONCLUSIVE or an explicit finding.

Responsible use

Audit only content and local environments you are authorized to examine. Keep dynamic tests on synthetic local fixtures, deny network access, and never use real credentials, third-party accounts, destructive payloads, or external services. Obtain human approval before remediations with side effects. Preserve evidence and uncertainty; do not use Sentinel scores to make unsupported claims about people, authors, or complete product safety.

Development

python -m pip install -e ".[dev]"
make verify

make verify runs formatting/lint checks, strict type checking, security checks including a Sentinel self-scan, tests with coverage, and corpus verification. Additional targets include make test, make lint, make typecheck, make security, make mutation, and make fuzz-smoke.

See CONTRIBUTING.md. Security vulnerabilities belong in a private advisory described by SECURITY.md, not a public issue.

License

Sentinel is distributed under the MIT License.