llmlint-cli 0.3.8

LLM-as-judge linter: enforce code-quality checks deterministic linters can't express, by driving real coding harnesses through oneharness.

llmlint

The next generation of linting: an LLM as a judge. llmlint enforces the code-quality checks a human reviewer normally makes — adherence to architectural patterns, coding-style intent, alignment to organization objectives — that deterministic linters can't express. It is additive to your existing linters, not a replacement: keep using deterministic tools for everything they can already check, and reach for llmlint only for the judgment calls.

Each check is a rule: a statement about your code that is judged true (holds) or false (a violation). A single fast Rust binary, llmlint batches your rules into as few harness calls as it can, then drives a real coding harness (Claude Code, Codex, Cursor, …) through oneharness to read the relevant files and decide, and reports the violations — with file and line numbers where they can be pinned down. Because the gate is "just a config file," llmlint drops into CI next to your other linters.

By default llmlint reports the failing rules (with the locations it could pin down) and a one-line summary — passing, skipped, and not-relevant rules are just counted:

Add -v to itemize every rule (passed, skipped, and not relevant too) and to print the oneharness debug view — the exact oneharness run … command and the raw result for each judge — to stderr, so the report on stdout stays clean:

The -v debug view (oneharness command + raw result per judge, on stderr)

These are real captures of the CLI, rendered from the actual colorized output by just screenshots and gated by screencomp.

The exit code is unaffected by verbosity (0 all-pass, 1 a violation, 2 the run couldn't complete); operational errors are always shown. Use --format json for the full machine-readable report.

The human report is colorized — green PASS, red FAIL/ERROR — when stdout is a terminal. Coloring follows the NO_COLOR convention and a --color <auto|always|never> flag: auto (the default) colors only an interactive terminal, always forces it (e.g. through a pager or to capture a screenshot), never disables it. --format json is never colorized.

While the judges run, llmlint draws a live progress view on stderr — rules resolving one by one as their judges return (the GIF above) — then clears it and prints the report. Like the color, it is audience-aware: --progress <auto|always|never> (default auto) shows it only for an interactive human — a terminal, not CI, and not an AI coding agent (Claude Code, Codex, Cursor, detected via their environment variables). Piped, redirected, in CI, or captured by an agent, the view is fully suppressed so it never spams a log or an agent's context — the report on stdout, and --format json, are byte-for-byte the same either way.

How it works

1. You declare rules (and optionally agents that group them) in a YAML config — like any other linter.
2. For each agent, llmlint renders a system prompt from a template (the rules + the target file paths) and calls oneharness run with a generated JSON Schema for structured output. oneharness constrains and validates the harness's answer, so llmlint gets a checked verdict per rule, not prose.
3. The harness reads the target files on demand with its own tools to gather evidence, then returns { "rule_name": { "holds": bool, "violations": [...] } }.
4. llmlint aggregates (majority vote across judges when configured), reports, and exits non-zero if any rule was violated.

llmlint shells out to oneharness — it is a runtime prerequisite (see Install).

Install

llmlint needs the oneharness binary on your PATH.

# One step via pip: prebuilt binaries for llmlint AND oneharness (a dependency)
pip install llmlint-cli

# Or piece by piece:
# 1) oneharness (the harness driver)
curl -fsSL https://raw.githubusercontent.com/nickderobertis/oneharness/main/scripts/install.sh | sh
#    (or: cargo install --git https://github.com/nickderobertis/oneharness --locked)

# 2) llmlint
curl -fsSL https://raw.githubusercontent.com/nickderobertis/llmlint/main/scripts/install.sh | sh
#    (or: cargo install llmlint --locked)
#    (or, without a crates.io release: cargo install --git https://github.com/nickderobertis/llmlint --locked)

llmlint doctor      # confirms oneharness is reachable

The installer honors LLMLINT_VERSION / LLMLINT_INSTALL_DIR (or the --version / --to flags), works on Linux, macOS, and Windows under a POSIX shell (Git Bash / MSYS / WSL), and refuses a binary it cannot verify. Each tagged release publishes prebuilt binaries for those platforms, each with a SHA-256 checksum and a keyless Sigstore build-provenance attestation bundle (.sigstore.json); on native Windows PowerShell, use cargo install llmlint --locked.

Via pip. Each release also ships per-platform wheels wrapping the same prebuilt binary, so anywhere Python is present, pip install llmlint-cli (or uv tool install llmlint-cli / pipx install llmlint-cli) is a seconds-fast binary install with no Rust toolchain — handy in restricted-egress environments where package registries are reachable but github.com is not. It depends on oneharness-cli, so the oneharness runtime prerequisite comes along automatically — one pip install is a complete, working setup. That includes tool-isolating installers: uv tool install and pipx link only llmlint itself onto your PATH, but llmlint also looks for oneharness beside its own executable — where those tools place dependency binaries — so no extra flags are needed. (The PyPI package is llmlint-cli — PyPI reserves names too similar to existing projects — but the installed binary is llmlint.) Wheels are published with PyPI Trusted Publishing and carry PEP 740 attestations — the same Sigstore build provenance as the GitHub release assets.

Behind a mirror. In a network that can reach a release-proxy mirror but not github.com, point the archive download at it:

LLMLINT_RELEASE_BASE_URL=https://mirror.example/llmlint \
  curl -fsSL https://raw.githubusercontent.com/nickderobertis/llmlint/main/scripts/install.sh | sh

The archive comes from the mirror, but its integrity is checked against a trust root the mirror does not control. If a Sigstore verifier is installed — cosign, the official sigstore Python client, or gh — the installer downloads the .sigstore.json bundle from the mirror and verifies it offline — no GitHub API — against the keyless signature bound to this repo's release workflow; the trusted digest comes from the signed attestation, so a mirror cannot forge it. With no verifier installed it falls back to a .sha256 fetched from canonical GitHub (LLMLINT_CHECKSUM_BASE_URL overrides that root) — but it refuses a checksum that shares the mirror's origin, since a tampered mirror would just serve a matching tampered checksum. If nothing independent of the mirror can vouch for the archive, the install aborts.

A verifier is one install away even where github.com itself is unreachable, because all three ship through package registries:

pip install sigstore                                        # PyPI
npm install -g @sigstore/cli                                # npm
go install github.com/sigstore/cosign/v2/cmd/cosign@latest  # Go module proxy

(go install fetches through proxy.golang.org and integrity-checks against the sum.golang.org transparency log — it never contacts github.com.) So the pattern for a restricted-egress host or sandbox image is: provision one verifier from whichever registry is reachable, then run the installer with LLMLINT_RELEASE_BASE_URL pointing at your mirror — every byte comes from the mirror or a registry, and the signature still chains to the Sigstore root. Elsewhere, brew install cosign, a distro sigstore package, or the sigstore/cosign-installer GitHub Action in CI all work too.

You also need a coding harness installed and authenticated (e.g. Claude Code). See oneharness list / oneharness detect --all.

Quick start

llmlint init                 # write a starter llmlint.yml (config-lint plugin on)
llmlint init --with-template # ...and embed the prompt template to customize
$EDITOR llmlint.yml          # write your rules
llmlint                      # lint the configured files
llmlint src/api/**/*.rs      # ...or lint specific files
llmlint --format json        # machine-readable output

Configuration

llmlint.yml (discovered by walking up from the working directory; override with -c/--config, repeatable). Discovery is nested in both directions. Walking up, every config found (one per directory) is merged, nearest first, so a config beside the files being linted, a project config above it, and a user-level config higher still layer together — the most-local config wins each top-level scalar, every config contributes its rules, and a more distant config fills only the gaps (the same nearest-wins precedence as plugins). Walking down, a config in a subdirectory governs its own part of the project: its files globs are rooted at that directory (a frontend/llmlint.yml with *.txt matches frontend/'s files, never a same-named file elsewhere), so you can keep per-area rules next to the code they check. A subtree config scopes rules, not session-wide settings (model, timeout, template, rationales come from the working directory and up); --config replaces the whole walk with no cascade. llmlint init writes it with a leading # yaml-language-server: $schema=… modeline pointing at llmlint's published JSON Schema, so editors with the YAML language server (e.g. VS Code's YAML extension) give completion and validation as you write. Add the same line to a hand-written config to opt in.

version: 1                     # this config's published version (used when it is consumed as a plugin)

# Files linted when none are passed on the CLI. Omit the whole block (or leave
# `include` empty) to lint every file in the tree from the current directory;
# `exclude` and `.gitignore` still apply.
files:
  include: ["src/**/*.rs"]
  exclude: ["**/generated/**"]

# Require a short `rationale` for every verdict (default true). See Rationales below.
rationales: true

# Default base for `--diff` (when `--diff-base` isn't passed). Any git revision —
# a branch, tag, commit, or `A..B`/`A...B` range. Set it to your default branch so
# a CI quality gate reviews whatever the current branch changed; unset = `HEAD`.
diff_base: main

# Pull in shared rule sets / plugins with one line each. An entry is a local
# path or a URL (`http(s)://`, `file://`); pin a URL to a version with `@`.
plugins:
  - "https://raw.githubusercontent.com/nickderobertis/llmlint/main/assets/config_lint.yml@1"  # bundled: lints this config's own rules
  - "https://example.com/org-rules.yml@1.2.3"   # pinned; fetched + cached once
  - "./team-rules.yml"

# Agents group rules and add reviewer context + harness/model/batch config.
# YAML anchors let you share prompt text with zero framework support.
agents:
  architecture:
    harness: claude-code       # any id from `oneharness list`; omit to use oneharness's own default
    model: opus
    batch_size: 15             # rules per judge run (default 20)
    prompt_template: |         # appended to the master template before render
      You are a senior software architect reviewing service boundaries.

rules:
  - name: handlers_delegate_to_services   # unique, terse, descriptive
    description: |
      true when every HTTP handler delegates business logic to a service layer.
      false when a handler performs business logic (DB queries, domain rules)
      inline.
    agent: architecture        # optional; omit to use the default agent
    # override: true           # optional; extend a same-named plugin rule, inheriting unset fields
    judges: 3                  # optional; independent judges, majority wins (default 1)
    rationale: true            # optional; override the session-wide `rationales` for this rule
    relevance: true            # optional; when to evaluate — see Relevance below (default true)
    files:                     # optional; override the target files for this rule
      include: ["src/api/**"]

Nested & per-directory configs

Configs nest — discovery walks both up from the working directory and down into its subtree, merging every llmlint.yml it finds. This lets you layer a user-level config, a project config, and per-area configs that live next to the code they govern, with no extra wiring.

~/.llmlint.yml                 # user-level defaults (model, rationales…)
my-project/
├── llmlint.yml                # project rules + settings  (run from here)
├── backend/
│   └── llmlint.yml            # rules for backend/**, globs rooted at backend/
│       #   files: { include: ["**/*.py"] }
│       #   rules: [{ name: no_print_debugging, … }]
└── frontend/
    └── llmlint.yml            # rules for frontend/**, globs rooted at frontend/
        #   files: { include: ["**/*.ts"] }
        #   rules: [{ name: no_inline_styles, … }]

Running llmlint from my-project/ evaluates all of these together:

• no_print_debugging runs only on backend/**/*.py, and no_inline_styles only on frontend/**/*.ts — each subtree config's files globs are rooted at its own directory, so **/*.py under backend/llmlint.yml means backend/**/*.py, never a stray .py elsewhere.
• The project's own rules and settings apply across the whole run; a more-local config wins each setting and can override a rule from a config above it.
• Session settings (model, timeout, prompt template, rationales) come from the working directory and up — a subtree config scopes rules, it doesn't retune the whole run. Run from my-project/backend/ instead and that config becomes the most-local one, layering under the project and user configs.

Passing explicit files (llmlint backend/svc.py) narrows this to just the configs that govern those files: a subtree config is consulted only when a passed file lives under it, and each rule judges only the passed files inside its own directory. So llmlint backend/svc.py never loads frontend/llmlint.yml (nor fetches its plugins, nor collides with a rule it happens to share a name with) — you get exactly the rules that apply to what you asked to lint.

Use llmlint config to see the merged result and llmlint config --sources (or llmlint where rules.<name>) to trace any rule, agent, or setting back to the exact file it came from. To bypass discovery entirely, pass explicit configs with -c/--config (repeatable) — that roots every glob at the working directory with no cascade.

Writing good rules

• Phrase each rule as a positive invariant. holds = true means the code complies; holds = false is a violation that llmlint reports and fails on.
• Make the verdict unambiguous. Often a plain statement of the property is enough — every public item has a doc comment already says what passes and, by negation, what fails; there's no need to bolt on a "false when…" clause that only restates the inverse. Spell out the violating case only when it carries meaningful detail — concrete examples, easily-confused edge cases — that the positive statement leaves unclear. When you do state both, keep them mutually exclusive. The bundled config-lint plugin (the config_lint.yml URL above) lints your config for exactly this, plus descriptive (non-placeholder) names that match what each rule checks.
• Names are unique, terse, and descriptive (^[A-Za-z][A-Za-z0-9_]*$); they become the JSON keys of the structured output.
• Scope a rule by file type or location with files, not relevance. When a rule applies only to files you can name by path — a file type (*.py) or a location (src/**) — put that subset in the rule's files globs. A path scope is deterministic and costs no judge tokens, and it keeps the verdict about the property, not about applicability. Reserve relevance for conditions on a file's content or changes that a glob can't express.
• Scope a rule to the changes it applies to with relevance (see below) instead of bolting "…or not applicable" onto the description — that keeps the true/false outcome clean and lets llmlint tell "didn't apply" apart from "true".
• Keep each description and relevance concise. A judge call batches an agent's rules into one prompt, so tokens one bloated rule spends dilute every other rule in that batch. State the invariant in the fewest words that keep it unambiguous; add length only when it buys clarity (concrete examples, tricky edge cases). config-lint checks this too.

The prompt template

llmlint renders the judge's system prompt from a minijinja (Jinja2-style) template. The bundled default lives in assets/default_template.md; embed a copy to customize with llmlint init --with-template, or set prompt_template yourself. The top-level prompt_template replaces the master template; an agent's prompt_template is appended to it before rendering, so reviewer context you add per-agent sees the same variables.

These variables are in scope when a template renders:

Variable	Type	Description
files	list of strings	The target file paths for this run — relative to the working directory, always forward-slashed (so a Windows run reads the same as Linux/macOS).
rules	list of objects	The rules in this batch. Each has .name (the identifier, also the JSON key in the structured output), .description (the invariant to judge), .rationale (whether this rule wants a justification), .relevance (the relevance condition string, or unset for an always-evaluated rule), .require_line_attribution (whether every violation must cite a file + line), and .files (the subset of files this rule applies to).
file_rules	list of objects	Per-file applicability — one entry per target file, in the same order as files. Each has .file (the path), .mode ("include" or "exclude"), .rules (the rule names to apply or, when mode == "exclude", to skip — whichever list is shorter), and .diff (that file's unified diff, present only under --diff when the file changed). The default template's "Target files" section is built from this.
diffs	list of objects	Per-file changed-line diffs — one entry per changed file, present only under --diff (empty otherwise). Each has .file (matching its entry in files) and .diff (the unified diff text). The default template inlines these per file via file_rules; kept separately for custom templates.
rationales	bool	True when any rule in this batch wants a rationale — gate the rationale guidance on it.
relevance	bool	True when any rule in this batch carries a relevance condition — gate the relevance guidance on it.
line_attribution	bool	True when any rule in this batch requires line attribution — gate the line-attribution guidance on it.

## Target files
{% for f in files %}- {{ f }}
{% endfor %}
## Rules to evaluate
{% for r in rules %}### {{ r.name }}

{{ r.description }}
{% endfor %}

A run is one (agent, file set, judge) batch, so rules is that batch's slice (see batch_size), not necessarily every rule in the config.

Rationales

By default each judge must justify every verdict with a short rationale. The structured output for each rule is ordered deliberately — the judge echoes the rule name, writes the rationale, then commits to the result (holds + any violations):

{
  "no_inline_sql": {
    "name": "no_inline_sql",                       // 1. anchor on the rule
    "rationale": "raw SQL built inline in db.rs:42, not via the query layer",  // 2. reason
    "holds": false,                                // 3. conclude
    "violations": [{ "file": "src/db.rs", "line": 42, "message": "inline SQL" }]
  }
}

Reasoning before concluding (and naming the rule first) keeps each verdict consistent and targeted — it leans on the model's next-token prediction so the holds follows from the evidence just written, not the other way round. Beyond that, rationales buy you:

• Auditability — a durable record of why each verdict landed, carried in --format json for every rule (pass or fail).
• Debugging — when a verdict looks wrong, you see the judge's reasoning, not just a bare pass/fail.
• Reliability — verdicts are measurably steadier when the judge must commit to evidence first.

The cost is extra output tokens on every request. Turn rationales off to save tokens:

rationales: false            # session-wide default (CLI --no-rationales overrides it)

rules:
  - name: handlers_delegate_to_services
    description: ...
    rationale: true           # …but keep them for this high-stakes rule

Precedence, lowest to highest: the session default rationales (default true) → a per-rule rationale → the --rationales / --no-rationales CLI flags (which set the session default for the run; a per-rule rationale still wins). In the human report, a rule's rationale is shown for every failure by default, and for every evaluated rule at -v. The default prompt template asks for rationales that are terse and pithy — the fewest tokens that still cite the evidence — so the token cost stays small. See Cost vs performance to trade it for a cheaper run.

For a multi-judge rule (judges: N), the report and --format json show each judge's result and rationale, not just one representative — so you can see exactly where the judges agreed or split:

Relevance

Not every rule applies to every change. Rather than make each description carry its own "…or not applicable" escape hatch — which muddies the true/false outcome and hides why a rule passed — declare when a rule should be evaluated with relevance:

rules:
  # Always evaluated (the default). The judge may not opt out.
  - name: public_items_are_documented
    description: ...
    # relevance: true            # implicit

  # Never evaluated — disabled deterministically, with no judge call.
  - name: legacy_only_check
    description: ...
    relevance: false

  # Conditionally evaluated. The judge decides whether the condition holds for
  # the change *before* the verdict; if it doesn't, the rule is "not relevant".
  - name: errors_are_contextualized
    description: |
      TRUE when every returned error adds context about the operation that
      failed. FALSE when an error is propagated with no added context.
    relevance: the change adds or modifies error handling

For a conditional rule the structured output gains a relevant boolean, decided before the verdict — so a not-applicable rule is distinguishable from a true one:

// Not relevant: the object ends after `relevant`; the rationale explains why.
{ "errors_are_contextualized": {
    "name": "errors_are_contextualized",
    "rationale": "the change only renames a struct field; no error handling touched",
    "relevant": false } }

// Relevant: proceed to the verdict as usual.
{ "errors_are_contextualized": {
    "name": "errors_are_contextualized",
    "rationale": "every `?` propagation wraps with `.context(...)`",
    "relevant": true,
    "holds": true } }

A not-relevant rule is neither a pass nor a violation — it never fails the build. The human report counts it in a … not relevant summary segment and, at -v, itemizes it as a dim N/A <rule> (not relevant) line with the reason; --format json carries "outcome": "not_relevant" and a not_relevant summary count. For a multi-judge rule, relevance is decided by majority first, then the verdict is tallied over the judges that found it relevant.

Ignore directives

Suppress a rule at a specific place with an inline comment in the target file — the same idea as # noqa / // eslint-disable, but strict: a directive must name the specific rule(s) and give a reason.

let q = format!("SELECT * FROM users WHERE id = {id}"); // llmlint: ignore[no_inline_sql] one-off migration, not user-facing

# llmlint: ignore-file[public_items_are_documented] generated stubs, documented upstream

// llmlint: ignore-block[no_inline_sql] legacy query layer, migration tracked in JIRA-42
fn legacy_queries() { /* … */ }
// llmlint: ignore-end[no_inline_sql]

• llmlint: ignore[rule, ...] <reason> is line-scoped — it covers the line it sits on (a trailing comment) or the line right below it (a comment on its own line).
• llmlint: ignore-file[rule, ...] <reason> is file-scoped — it covers the whole file.
• llmlint: ignore-block[rule, ...] <reason> … llmlint: ignore-end[rule, ...] is block-scoped — it covers every line between the open and its matching close. The closing ignore-end names the same rule(s) and needs no reason. Blocks track each rule independently, so rules opened together may be closed at different points and blocks for different rules may overlap.

Use whatever comment syntax the file's language uses (//, #, /* … */, <!-- … -->); llmlint keys off the reserved llmlint: ignore / llmlint: ignore-file / llmlint: ignore-block / llmlint: ignore-end prefix.

Two layers, by design. llmlint deterministically validates each directive's structure before any judge runs — it must name specific, configured rule(s) and carry a reason (except ignore-end, which only closes a block). A directive with no brackets, an empty list, an unknown or misspelled rule, or no reason is a hard file:line: error (exit 2), so a typo fails loudly instead of silently suppressing nothing. Block pairing is checked too: an unclosed ignore-block, an ignore-end with no open block, or re-opening a rule already open is a hard error. Actually honoring a well-formed directive is the judge's job: the default prompt tells it to skip a named rule's violation at the directive's location. (A custom prompt_template should carry the same guidance if you want directives honored.) Because the prefix is reserved, a linted file that merely documents the feature must use real rule names or avoid the literal llmlint: ignore[…] form.

This structural check is deterministic and free — no model call — so it is also exposed as its own command, llmlint check-ignores. Run it in your tight, fast linter loop (next to cargo fmt / clippy, in a pre-commit hook, or as a quick CI step), where it catches a typo'd or reason-less directive in milliseconds. The full llmlint run performs the same check as a pre-flight, so the two never disagree — check-ignores just gives you the fast feedback without waiting on (or paying for) a judge.

Batching

Model calls are the slow, paid part, so llmlint packs rules into as few as it can. Rules group by agent, then split into batches of at most batch_size (default 20) — one oneharness run per batch, over the union of its files, each rule scoped to its own files in the prompt. Multi-judge rules fan out per judge (judge j runs the rules with judges >= j). Fewer, fuller batches, fewer round-trips.

Judges and voting

judges: N runs a rule through N independent judges and takes the majority verdict. N must be odd (1, 3, 5, …) so the vote can't tie — an even count is a config error. Only rules that opt in pay the extra cost: judge 1 runs all rules, judge 2 only the rules with judges >= 2, and so on.

Cost vs performance (token usage)

Defaults favor judgment quality over cost: rationales on, a thorough prompt, every file read in full. Trade some back for fewer tokens, roughly by impact:

• judges — each extra judge is a full extra pass (Judges). Keep it at 1 except for high-stakes rules.
• rationales: false — drops the per-verdict justification, output tokens on every rule (Rationales). Re-enable rationale: true per rule; --no-rationales for one run.
• Fewer agents, bigger batch_size — every batch re-sends the prompt and re-reads its files (Batching). Merge rules onto one agent; split only for a different harness, model, or reviewer context.
• Read less — narrow files.include/exclude; --diff reviews only changed lines; FILES/--rule/--agent lint a subset.
• require_line_attribution off unless you need pinned locations — on can trigger localize re-prompts.
• oneharness.schema_max_retries — caps re-asks on a schema-invalid answer.
• model — dollars, not tokens: a cheaper model per agent or run.

llmlint check-ignores spends no tokens at all.

Results logging & history

The terminal report is deliberately terse — failing rules and a summary. But the full results (every rule's verdict, votes, per-judge breakdown, violations, and rationales) are worth keeping. Results logging is on by default: each lint/lint-config run is also written to disk as one JSON record under an auto-generated, time-sortable id, and the run prints that id on stderr:

$ llmlint
FAIL no_inline_sql
     src/db.rs:12: inline SQL

2 rules: 1 passed, 1 failed, 0 skipped
See full results with `llmlint history 20260704T153000Z-1a2b3`

Inspect what was logged with llmlint history:

llmlint history                       # list recent runs (most recent first)
llmlint history latest                # full results of the most recent run
llmlint history <id>                  # full results of a specific run
llmlint history <id> --status fail    # only the failing rules
llmlint history <id> --rule my_rule   # only one rule
llmlint history <id> --path           # just the JSON record's path (for scripting)
llmlint history <id> --format json    # the raw record
llmlint history --format json         # a JSON array of run summaries

Records live in the platform per-user data directory by default (~/.local/share/llmlint/history on Linux, ~/Library/Application Support / %LOCALAPPDATA%\llmlint\data\history on macOS/Windows; $XDG_DATA_HOME is honored). Only the newest 100 runs are kept; older records are pruned after each run. All three knobs are configurable:

history:
  enabled: true          # false turns logging off entirely
  max_runs: 100          # how many recent runs to keep
  dir: .llmlint/history  # where records go (relative to nothing in particular; an absolute path is clearer)

The LLMLINT_HISTORY_DIR environment variable overrides dir, and --no-history (or LLMLINT_NO_HISTORY=1) disables logging for a single run. Like the other top-level settings, history is a cwd-and-up session setting (a subtree config never retunes it) and traces through llmlint config --sources / llmlint where history.dir.

oneharness passthrough

llmlint lets oneharness discover its own oneharness.toml by default. To force a specific oneharness config, use --oneharness-config <path> (or oneharness.config in the llmlint config); it is forwarded via oneharness's --config. Override the binary with --oneharness-bin or $LLMLINT_ONEHARNESS_BIN.

Plugins (shared rule sets)

plugins pulls other llmlint configs into this one — their rules and agents are merged in. For the top-level settings (template, files, oneharness, rationales), the nearer config to the root wins: your config's settings take precedence over a plugin's, a plugin's over its own plugins', and an earlier-listed plugin over a later sibling. A plugin only fills in a setting the including config left unset, so a shared plugin can ship sensible defaults without overriding what you set locally. The CLI overrides all of them (see Commands). Each entry is a config file:

• a local path (./team-rules.yml), resolved relative to the including file;
• a URL — http(s):// (fetched over HTTPS) or file:// (read directly).

Resolution is transitive: a pulled-in config's own plugins are pulled in turn, and so on. Diamonds and cycles are de-duplicated (each config loads once), and the chain is bounded at a depth of 100 to fail fast on a pathological graph.

By default a rule name is unique across the whole merged config — declaring the same name twice is an error. To adjust a rule a plugin gave you without restating it, re-declare it with override: true and set only the fields you want to change; every other field (including the description) is inherited from the plugin's rule:

plugins:
  - "https://example.com/org-rules.yml@1"   # ships `no_inline_sql`, 1 judge

rules:
  # Keep the org rule's text, but vote it across 3 judges and scope it tighter.
  - name: no_inline_sql
    override: true
    judges: 3
    files:
      include: ["src/db/**"]

The override must be set on the nearer-root config, and there must be exactly one base rule (the same name declared without override) for it to extend — an override with nothing to override is an error, so a typo'd name can't silently do nothing. When several configs override the same base, the nearest-root override wins each field.

URL fetching is built in (a pure-Rust HTTPS client — no curl or other external tools, no system OpenSSL) and honors the standard HTTP(S)_PROXY / NO_PROXY env vars. The bundled config-lint plugin ships inside the binary and resolves offline.

A URL may be pinned to a version with an @ suffix matching the plugin config's own top-level version: @1 accepts any 1.x, @1.2 any 1.2.x, @1.2.3 exactly that. The pin is both an assertion (a mismatch is a hard error) and the cache key: a pinned URL is fetched once into the cache and reused on later runs without refetching — bump the pin to pull a new version. An unpinned URL is fetched every run.

The cache lives under $XDG_CACHE_HOME/llmlint/plugins (override with LLMLINT_CACHE_DIR). Set LLMLINT_PLUGIN_REFRESH=1 to force a refetch.

Linting your llmlint configs

llmlint ships with a config-lint rule set that lints llmlint config files themselves — that every rule's description yields a clear, unambiguous verdict, its name is descriptive (non-placeholder) and matches what the description checks, a conditional rule uses relevance instead of bolting "…or not applicable" onto the description, and a rule scoped to a file type or location uses files globs rather than relevance. It's the Writing good rules guidance, enforced (and each rule is phrased to pass its own checks). Each finding cites the config file + the offending rule's line (require_line_attribution). There are two ways to use it:

• As a plugin — add the bundled URL to your plugins (it's on by default in llmlint init), and its rules run against your config files on every normal llmlint run, alongside your own rules:

  plugins:
    - "https://raw.githubusercontent.com/nickderobertis/llmlint/main/assets/config_lint.yml@1"
  

  The URL ships inside the binary and resolves offline (no network, no cache), so it works disconnected and needs no pin bump to stay current.

• As a subcommand — llmlint lint-config is the lint command with that plugin included by default, so you don't have to add it to your config. It first runs the deterministic ignore-directive (comment) check over the config files, then judges each config's rules. Point it at specific files (llmlint lint-config path/to/llmlint.yml) or let it discover every llmlint config in the tree. Handy in CI as a standalone "is my config well-authored?" gate.

Finding where something is defined

Once configs merge across files and plugins, a rule, agent, or setting in the effective config can come from any of them. Two commands trace an item back to the file (or plugin URL) you'd edit to change it.

llmlint where <path> answers one lookup and prints just the source, so it composes in scripts. The path mirrors the config structure:

$ llmlint where oneharness.model            # a top-level setting
./shared/team.yml
$ llmlint where agents.security             # an agent
./shared/team.yml
$ llmlint where rules.no_inline_sql         # where a rule is defined
https://example.com/org-rules.yml@1
$ llmlint where rules.no_inline_sql.judges  # the file an override set a field in
./llmlint.yml
$ editor "$(llmlint where rules.no_inline_sql.judges)"

Because an override resolves field by field, a single rule can draw its description from the plugin that defined it and its judges from your config — where rules.<name>.<field> points at the file that actually set that field (or the definition site when no override did). An unknown name lists what's available, and a setting left at its built-in default says so, both exiting non-zero.

For the whole picture at once, llmlint config --sources adds a sources block:

{
  "config_files": ["./llmlint.yml", "./shared/team.yml", "https://example.com/org-rules.yml@1"],
  "sources": {
    "settings": { "version": "./llmlint.yml", "oneharness.model": "./shared/team.yml" },
    "agents":   { "security": "./shared/team.yml" },
    "rules": {
      "no_inline_sql": {
        "source": "https://example.com/org-rules.yml@1",  // where the rule is defined
        "fields": { "judges": "./llmlint.yml" }            // a field an override moved
      }
    }
  },
  "config": { /* … the merged config … */ }
}

A rule with no cross-file override has no fields entry; settings and agents are each kept whole from the nearest-root config that set them, so they have a single source.

Commands & exit codes

• llmlint [FILES...] — lint (the default). --format human|json, --agent, --rule, --max-parallel, --timeout, --cwd. Target individual rules with --rule NAME (repeatable) or a whole group with --agent NAME; an unknown rule/agent name is an exit-2 error that lists the available names. Every top-level setting also has a flag that wins over the config: --rationales/--no-rationales, --model NAME, --schema-max-retries N, --prompt-template PATH, plus --oneharness-bin/--oneharness-config. Pass --diff [<backend>] to add each changed file's diff to the judge prompt so it reviews only the changed lines; bare --diff uses the git backend (compared against HEAD). Add --diff-base <REF> to compare against a different git revision instead of HEAD — a branch, tag, commit, or A..B/A...B range — so --diff --diff-base main reviews exactly what the current branch changed versus main (the PR-review case). The base can also be set once in config as diff_base: (the flag overrides it).

• llmlint check-ignores [FILES...] — validate the structure of inline llmlint: ignore directives in the target files, deterministically and with no model call (-c/--config, --cwd; pass FILES to scope it, e.g. the changed files in a pre-commit hook). This is the same pre-flight lint runs, split out for the fast static-check loop: exit 0 when every directive is well-formed, exit 2 (located file:line:) on a typo'd / reason-less / unbalanced one.

• llmlint lint-config [FILES...] — lint llmlint config files with the bundled config-lint rules, without adding the plugin to your own config. It's the lint engine with that plugin forced on: it first runs the deterministic comment (ignore-directive) check, then judges each config's rules. Shares lint's flags (--format, --model, --timeout, --cwd, --diff, …); the config source is fixed, so --config/--agent/ --rule aren't taken. Same exit codes as lint.

• llmlint init — write a starter config (--with-template, --global, --force).

  

• llmlint config — print the merged config and the ordered list of sources that contributed, as JSON. Add --sources to also trace every rule, agent, and setting back to the file (or plugin URL) it came from — see Finding where something is defined.

  

• llmlint where <path> — print the single source of one config item: a setting (oneharness.model, version), agents.<name>, rules.<name>, or a rule field rules.<name>.<field>. See Finding where something is defined.

• llmlint doctor — check that oneharness is installed and reachable.

  

• llmlint history [ID] — inspect logged run results. With no id, list recent runs; with an id (or latest), show that run's full results. --status pass|fail|skipped|not_relevant and --rule NAME (both repeatable) drill into part of a run; --path prints just the record's file path; --format json emits the raw record (or a JSON array when listing); --dir/--cwd/--limit tune where and how much.

Exit codes: 0 all rules hold · 1 at least one violation · 2 usage, configuration, or harness error (could not complete the lint).

Development

just bootstrap   # toolchain components + fetch (from a clean clone)
just check       # full gate: fmt, clippy -D warnings, tests + 95% coverage, docs
just test-e2e    # the e2e binary journeys in isolation
just deps-check  # cargo deny + cargo machete
just lint-live   # opt-in: ad-hoc lint against the REAL oneharness + a real harness
just live-claude # opt-in: live e2e — built llmlint → real oneharness → real harness

Tests drive the real llmlint binary against a hermetic mock-oneharness fixture. The live tier (just live-claude, and the ad-hoc just lint-live) drives the whole stack end to end against a real, authenticated harness — the only thing that makes real model calls, and out of the check gate. It runs on PRs in its own workflow across Linux/macOS/Windows, so a missing CLI, auth, or oneharness is a hard failure, not a skip. See AGENTS.md and tests/AGENTS.md.

License

MIT — see LICENSE.