LintLang

Product page: lintlang.ai

LintLang statically analyzes the natural-language instructions that control AI agents, catching ambiguous tools, missing limits, and conflicting directives before runtime.

It flags patterns such as:

• empty, vague, or overlapping tool descriptions;
• tool pairs with no term that distinguishes one from the other (H1.6);
• missing stop conditions and unbounded retries;
• inconsistencies between tool schemas and their descriptions;
• unscoped context and vague instructions;
• conflicting output formats and malformed message roles;
• embedded prompts and uncalibrated thresholds in Python pipelines.

LintLang's default static checks are deterministic and local. They make no LLM, API, telemetry, or network calls.

LintLang was developed as the engineering offshoot of A Taxonomy of Epistemic Failure Modes in Large Language Models, but its bounded detectors do not claim to implement or validate every failure mode in the paper.

Technical note

Tool Differentia: Relational Static Analysis for AI Agent Tool Descriptions documents LintLang H1.6, the bounded pairwise check for tool descriptions that do not supply an analyzed distinction from a neighboring tool. It is a technical note, not a semantic-equivalence proof or a runtime-selection evaluation. Use its version-independent concept DOI, 10.5281/zenodo.21817243, for citation; the current archived release is Version 1.0.1.

Quick start

Run once without installing, using uv:

uvx lintlang scan AGENTS.md

For a persistent command in an isolated environment, use pipx:

pipx install lintlang
lintlang scan AGENTS.md

If pipx's app directory is not on PATH, run pipx ensurepath, open a new shell, and retry the scan.

Or install from PyPI into the current Python environment:

python -m pip install lintlang

Requires Python 3.10+.

From your project root, point LintLang at an actual instruction file:

lintlang scan AGENTS.md

If your project uses another filename, replace AGENTS.md with its prompt, tool-definition, agent-configuration, or supported directory path.

Character.AI's public Larch repository pins lintlang==0.3.1 in recurring CI. LintLang also has independent Gentoo packaging in the unofficial Haven overlay, not the official tree or GURU.

When you are ready to make HIGH or CRITICAL findings block CI:

lintlang scan AGENTS.md --fail-on fail

Each finding identifies the affected location, the detected pattern, its severity, and a suggested review action.

Try the bundled example

The source repository includes a deliberately broken example:

git clone --depth 1 https://github.com/hermes-labs-ai/lintlang.git
cd lintlang

lintlang scan samples/bad_tool_descriptions.yaml --fail-on fail

Excerpt from lintlang 0.5.3:

LINTLANG v0.5.3

FAIL — 1 CRITICAL, 2 HIGH, 7 MEDIUM, 3 LOW

H1: Tool Description Ambiguity

  [CRITICAL] H1.1 tool:process_ticket
  Tool 'process_ticket' has no description.

  [HIGH] H1.2 tool:get_user_info
  Tool 'get_user_info' has a very short description (13 chars):
  "Get user info"

…

H2: Missing Constraint Scaffolding

  [HIGH] system_prompt
  System prompt defines tools but contains no termination conditions,
  retry budgets, or progress checks.

The command exits with status 1 because it includes --fail-on fail.

Verdicts and CI behavior

PASS applies only to recognized content extracted from the requested inputs and the checks and severity filters selected for that run. It does not mean that every structure in an arbitrary JSON or YAML file was extracted. A clean LintLang scan is not evidence that an agent is safe or runtime-correct.

By default, findings are reported without failing the process.

• --fail-on fail blocks on FAIL.
• --fail-on review blocks on REVIEW or FAIL.
• Missing, malformed, unreadable, or otherwise unscannable requested inputs remain nonzero regardless of the chosen finding threshold.
• An invocation that finds no eligible files exits nonzero.

Filters such as --min-severity are applied before the verdict. For initial adoption, keep the full output visible and use --fail-on fail to block only the highest-severity findings.

Add it to CI

From a Git repository containing AGENTS.md, CLAUDE.md, a Copilot instructions file, or an agent YAML/JSON config, create the pinned GitHub Code Scanning workflow in one command:

lintlang init --github

Use --path path/to/instructions when auto-detection should not choose the input. The initializer will not replace a different existing workflow unless you pass --force; inspect that diff before committing it. Generated workflows pin the latest reviewed, already-released LintLang action to its immutable commit, with the release tag retained as a human-readable comment. The pin can intentionally trail the package being prepared because that package's release commit does not exist yet when its artifacts are built.

After choosing one real instruction path in your repository:

jobs:
  lint-agent-instructions:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false

      - name: Inspect agent instructions
        uses: hermes-labs-ai/lintlang@v0.5.3
        with:
          path: AGENTS.md

The release tag pins both the action and the LintLang source it installs. Upgrade that pin deliberately and inspect newly introduced findings before making them blocking.

Hermes Agent verification hook

When LintLang and Hermes Agent are installed in the same Python environment, Hermes discovers LintLang through its native hermes_agent.plugins entry-point contract. LintLang registers one bounded pre_verify hook: after a coding turn changes a recognized agent instruction, prompt, skill, tool, or agent-config surface, it runs the same local deterministic scan before the turn finishes.

PASS and REVIEW do not interrupt the turn. FAIL or an input ERROR keeps the turn open once with the exact lintlang scan command to run. The hook self-throttles on Hermes' attempt field and ignores ordinary source and documentation files, so it cannot create an unbounded retry loop or turn a general code edit into a prompt-lint gate.

Verify discovery with:

hermes plugins list

Disable the lintlang plugin through Hermes' normal plugin controls if the workspace should use only LintLang's CI or pre-commit surfaces.

Add it to pre-commit

Add the hook to .pre-commit-config.yaml with the explicit instruction paths to scan:

repos:
  - repo: https://github.com/hermes-labs-ai/lintlang
    rev: v0.5.3
    hooks:
      - id: lintlang
        args: [AGENTS.md]

Activate it and test the configured paths:

pre-commit install
pre-commit run lintlang

Replace or extend args with the prompt, tool-definition, agent-configuration, or supported directory paths your repository owns. The hook scans only those configured paths and reports findings without blocking on a verdict by default.

After reviewing the repository's baseline, opt into blocking FAIL findings:

hooks:
  - id: lintlang
    args: [AGENTS.md, --fail-on, fail]

Missing, unreadable, or malformed configured inputs still return nonzero.

Use it with Gemini CLI

The repository root is also a Gemini CLI extension. Its non-blocking AfterTool hook returns LintLang repair guidance after Gemini changes supported files with write_file or replace. See the Gemini CLI extension guide for the pinned, isolated dependency contract and installation steps.

Machine-readable output and GitHub Code Scanning

For machine-readable output:

lintlang scan AGENTS.md --format json --fail-on fail

For deterministic SARIF 2.1.0 output on stdout:

lintlang scan AGENTS.md --format sarif --fail-on fail > lintlang.sarif

Relative inputs are resolved from the current directory. Artifact URIs are URI-encoded paths relative to the nearest Git worktree root (or the current directory when there is no Git worktree). A resolved source outside that root is a fatal output error rather than an absolute-path leak. Python AST findings carry supported line spans; YAML, JSON, and text findings intentionally remain file-level.

The composite Action can write the same report with its optional sarif-file input. In that mode SARIF stdout is redirected to the requested file, while verdict messages remain on stderr and fail-on keeps its normal exit status. Directory creation or file-write errors are fatal.

To ask GitHub to ingest the report without exposing Code Scanning write permission to LintLang or its scan-time dependencies, use separate scan and upload jobs. The upload job checks out source without persisting credentials so GitHub can calculate missing fingerprints. The artifact handoff runs even after a blocking LintLang verdict; the scan job still keeps that failure as its conclusion:

name: LintLang Code Scanning

on:
  push:
  pull_request:

jobs:
  scan:
    runs-on: ubuntu-latest
    permissions:
      contents: read
    steps:
      - name: Check out repository
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false

      - name: Run LintLang
        uses: hermes-labs-ai/lintlang@cad2dca3054b8bfb5d0a6b93ecf19f9d74ab64fe # v0.5.0
        with:
          path: AGENTS.md
          fail-on: fail
          sarif-file: lintlang.sarif

      - name: Preserve LintLang SARIF
        if: always()
        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        with:
          name: lintlang-sarif
          path: lintlang.sarif
          if-no-files-found: error

  upload-sarif:
    needs: scan
    if: always() && (github.event_name == 'push' || (github.actor != 'dependabot[bot]' && github.event.pull_request.head.repo.full_name == github.repository))
    runs-on: ubuntu-latest
    permissions:
      actions: read
      contents: read
      security-events: write
    steps:
      - name: Check out repository for SARIF fingerprinting
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false

      - name: Download LintLang SARIF
        uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
        with:
          name: lintlang-sarif

      - name: Upload LintLang SARIF
        uses: github/codeql-action/upload-sarif@5595ccaf912efad79be6eef63a5619ff05969be3 # v4
        with:
          sarif_file: lintlang.sarif

The complete copy-paste workflow is examples/github-code-scanning.yml. LintLang emits code-quality/static-language results without security tags, security severity, source snippets, or custom fingerprints. GitHub's upload Action may calculate fingerprints during ingestion.

What it inspects

LintLang currently accepts:

• JSON and YAML objects using recognized top-level agent fields such as system_prompt, instructions, tools, functions, messages, and selected response-schema fields;
• .txt, .md, and .prompt instruction files;
• Python files, using AST extraction for prompt-like strings and threshold assignments.

Nested vendor-specific layouts and raw top-level YAML arrays are not automatically normalized. A syntactically valid input must still match a recognized shape for its structured tools or messages to be inspected.

The checks cover reader-facing categories including tool clarity, execution bounds, schema-description alignment, context boundaries, instruction specificity, output contracts, message-role structure, and Python pipeline hygiene.

H1.6: tool descriptions without a differentia

Per-tool schema validation assesses one definition at a time. Within one parsed input, H1.6 instead compares tool definitions with each other and reports a pair when, under LintLang's term-and-synonym model, one or both descriptions provide no distinguishing term. Both tools can be individually valid, so per-tool validation has nothing to report. A mutual finding means neither description distinguishes itself; domination means one tool's terms are all covered by the other, and the finding names which description to repair. Directory scans do not aggregate tool definitions across files or infer a shared namespace.

Findings print the sub-code: ~ [MEDIUM] H1.6 tool:find_tickets vs tool:search_tickets. pattern_id stays H1; JSON output adds a code field holding the most specific identifier.

H1.6 is MEDIUM, so --fail-on fail does not block on it. Matching uses a finite English synonym lexicon, so pairs that say the same thing in different words or a different sentence shape are missed. The absence of an H1.6 finding is not evidence that no such pair exists.

Use narrow, intentional paths. Directory scans can discover Markdown and Python files that were not written as agent configuration; use .lintlangignore or --exclude where needed.

For the exact H-series identifiers:

lintlang patterns

lintlang patterns lists the H1-H7 structural detectors only. Python pipeline findings report as P1 and P2 in scan, JSON, and SARIF output.

See the full technical reference for detector details.

Where it fits

syntax and schema validation
        ↓
LintLang static language checks
        ↓
runtime agent evaluation
        ↓
domain and security review

LintLang is useful during authoring and pull-request review, before runtime testing. It does not:

• determine whether an instruction is factually or semantically correct;
• observe an agent selecting or executing tools;
• prove that a finding causes a runtime failure;
• certify an agent as safe or production-ready;
• replace runtime evaluation or human review.

Suggestions are review aids, not guaranteed meaning-preserving fixes.

Optional instruction preflight

Secondary capability: provider-neutral instruction preflight inspects one present instruction plus explicit context.

More

• Technical reference
• Product scope and invariants
• Changelog
• Contributing
• Report an issue or disputed finding
• Security policy

License

Apache License 2.0

LintLang is maintained by Hermes Labs.