veneer

Static analysis for the failure modes of AI-generated Python — hallucinated imports, vacuous tests, silent exception handling.

$ veneer check .

AI001  ingest/loader.py:12       'fastparse' is not on PyPI — import will fail at runtime
AI002  api/handlers.py:203       except Exception block swallows the error with no handling
AI003  tests/test_billing.py:44  test test_refund_succeeds contains no assertion

What this is

A linter for patterns that are cheap for a parser to see and expensive for a human reviewer to see. Generated code has type hints, docstrings, clean naming and sensible structure — and sometimes a phantom import underneath. The defects live in the gap between looks right and is right.

This does not detect AI-generated code. It detects patterns that happen to be frequent in it. Some findings will be on human-written code, and that is expected. The audience is people who use AI assistants and want a safety net.

Install

pip install pyveneer

The distribution is pyveneer — veneer was already taken on PyPI. The command and the import package are both veneer.

Zero runtime dependencies. Python 3.9+.

Usage

veneer check .                      # check the current tree
veneer check src/ --select AI001    # one rule only
veneer check . --offline            # skip all network lookups
veneer check . --format json        # machine-readable
veneer check . --statistics         # counts per rule, not per finding
veneer check . --include-vendored   # also scan nested repos and vendor/
veneer rules                        # list available rules

What gets scanned

veneer checks one project. The walk stops at a nested project — a git submodule or separate checkout, or a conventional vendoring directory (vendor/, third_party/, _deps/, subprojects/ and friends) — because that code belongs to somebody else and would be judged against the wrong pyproject.toml.

Skipping is never silent: veneer names every tree it stopped at, and --include-vendored (or skip-vendored = false) scans them. Pointing veneer straight at one also works — an explicit path outranks the skip.

$ veneer check .
no findings
skipped 1 nested project (--include-vendored to scan it):
  third_party/pybind11

--statistics combines with --format json, which is the shape to reach for when aggregating many repositories:

for repo in */; do veneer check "$repo" --statistics --format json; done

Exit codes: 0 clean, 1 findings present, 2 internal error.

Rules

Each rule reports only what it can defend. AI004 reports documented-but-absent parameters and never the reverse, because partial documentation is a normal style. AI005 skips parallel test suites and examples/ directories, where duplication is the point. AI006 ignores anything involving a call, since any callable may return None. AI002 skips except queue.Empty: pass, which is control flow rather than a swallowed error.

AI001 and slopsquatting

Attackers query LLMs at scale, collect package names that get hallucinated repeatedly, and register those names on PyPI with malicious payloads. The model invents the name, the attacker owns it, the developer installs it.

So AI001 reports two distinct things:

• Absent from PyPI → error. The import fails at runtime.
• Present, undeclared, first published under 90 days ago → warning. Worth verifying before you install it.

Network responses are cached to ~/.cache/veneer/ for 7 days. --offline skips all lookups, and reports nothing it could not verify.

CI

GitHub Action

Findings appear as inline annotations on the pull request diff, not as a log buried in CI output.

permissions:
  contents: read
  security-events: write   # required to upload SARIF

jobs:
  veneer:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: Dante-Berth/veneer@v0.1.0

Inputs: paths, select, ignore, offline, sarif-file, upload-sarif, fail-on-findings, python-version.

pre-commit

repos:
  - repo: https://github.com/Dante-Berth/veneer
    rev: v0.1.0
    hooks:
      - id: veneer
      # or, with no network access:
      # - id: veneer-offline

SARIF anywhere else

veneer check . --format sarif --output veneer.sarif --exit-zero

--exit-zero matters in CI: it lets the report upload before a separate step decides whether to fail the build.

Suppression

except ValueError:  # noqa: AI002
    pass

• # noqa: AI002 on the reported line, or bare # noqa
• # veneer: disable-file anywhere in a file
• [tool.veneer] in pyproject.toml:

[tool.veneer]
ignore = ["AI003"]
exclude = ["migrations/*"]
per-file-ignores = { "tests/*" = ["AI002"] }
skip-vendored = false                 # scan nested repos and vendor/ too

• --select / --ignore override the config file

Precision

Six rules at 95% precision beats thirty at 60%. A linter that produces false positives gets uninstalled once and never reinstalled, so every rule biases hard toward under-reporting. If a pattern is ambiguous, it is skipped.

Rules are validated against real third-party repositories before they ship, not only against fixtures. See phase0/FINDINGS.md for the validation run that shaped AI001 and AI003 — including the version where AI003 produced 35 findings and every one was wrong.

A later survey across 37 repositories cut AI001 from 52 findings to 13 and AI003 from 69 to 31, by teaching AI001 about native extension modules a project builds itself and AI003 about fixtures, non-test classes and smoke tests. If you hit a false positive anyway, it is a bug worth filing — and suppression is there so you are never blocked on the fix.

License

MIT