alibi

Cross-check the views of your attack surface and find the endpoints that cannot corroborate each other.

An endpoint should be able to account for itself. It is in the code, so a contract should describe it. It is in the contract, so something should implement it. It takes real traffic, so it had better exist somewhere. When one view knows about an endpoint and the others do not, that gap is the finding.

alibi runs OWASP noir, reads its JSON, and compares the views against each other.

Why this is a separate tool

Noir already reads five independent views of the same surface:

What it does not do is compare them. That is the whole job here, and it needs no change to noir — alibi runs it once per view and joins the results.

The per-view part matters. Noir deduplicates by (method, url) across every analyzer, so a Flask route and an OpenAPI path spelled identically collapse into one endpoint carrying one technology. That is right for a discovery tool — it is one endpoint — but it erases the corroboration this tool is built to measure, and it erases it in the worst possible direction: the better two views agree, the more of them vanish. Casdoor scans as 372 code endpoints and 9 documented ones; scan its swagger/ directory alone and the specification has 235.

--only-techs restricts the detector pool, so one scan per view keeps each one whole. Which technology speaks for which view is views.yml; which technologies exist is whatever noir list techs reports.

alibi parses no API formats of its own. Its only input is noir's JSON.

Install

Requires noir 1.0.0 or newer on PATH -- that is the release where noir list techs became a subcommand, and that catalog is what assigns every technology to a view. Development tracks the current noir release. An older binary is refused by name rather than left to fail on its first catalog read.

$ uv tool install noir-alibi     # or: pipx install noir-alibi
$ alibi scan ./my-service

Use

$ alibi scan                                      # the working directory
$ alibi scan ./service ./contracts ./prod.har     # or wherever the views live

Every path is a source, scanned once per view. Point it at whatever you have — a source tree, a spec directory, a single capture file — and the views you are missing switch their rules off rather than flooding the report.

alibi  ·  1 source  ·  377 endpoints

  code 372   doc 235

  230 corroborated -- vouched for by more than one view

  19 endpoints nearly matched another view -- these may be matching failures, not real gaps

SHADOW  Shadow API -- Implemented, but no contract describes it
  134 findings  ·  4 critical, 57 high, 62 medium, 11 low

  critical POST    /api/upload-groups        router.go:87
           upload paths carry more consequence than reads
  critical POST    /api/upload-permissions   router.go:208
  ...
  ... and 122 more (SHADOW in full: -f json)

TWO SURFACES?
  The doc view is 97% under /api, and 37 of these findings are outside it.
  If that is a separate surface the contract never covered, narrow the scan:
    alibi scan <paths> --ignore '^/(?!api/)'
  If it is the same surface left undocumented, they are the findings that matter most.

Groups stop at twelve — the ordering is worst-first, so the tail is the least informative part, and -f json has all of it.

In CI

- run: alibi scan . ./contracts -f sarif > alibi.sarif
- uses: github/codeql-action/upload-sarif@v3
  with: { sarif_file: alibi.sarif }

Or gate directly: alibi scan . ./contracts --fail-on high exits non-zero when a finding reaches that severity. A scan noir could not read in full reports executionSuccessful: false, so a degraded run does not pass as a clean one.

How endpoints are matched

Noir keeps each framework's own route syntax rather than inventing a common one, so the same endpoint arrives spelled several ways:

python_flask   /api/users/<int:user_id>
aiohttp        /users/{id}
java_spring    /api/catalog/{id}
oas3           /v1/pets/{petId}
rails          /posts/:id
nginx          /admin/.*

The rule that makes these comparable: a path parameter's name is not part of its identity. {petId} and <int:user_id> describe the same slot; only its position and whether it spans a / matter. Names are kept as evidence and reported, but never reach the key.

Findings say how the match was made:

What keeps it honest

A tool like this dies by reporting hundreds of findings on its first run, or by reporting progress nobody made. Six things push back:

Rules do not fire without both views. Scan a codebase with no contracts anywhere and every endpoint technically qualifies as an undocumented shadow API. Those findings say nothing except that you did not supply any documentation, so a rule only runs when every view it reasons about was actually in the scan. The report names the rules that sat out.

Near misses are reported as doubt, not as findings. "In code, not in the docs" is indistinguishable from "in both, but alibi failed to line them up." So an endpoint that lands in one view is checked against the others for a near miss — same path with a different verb, or one segment apart where one side has a parameter and the other a literal. Findings carrying a near miss are demoted and flagged for review. That count sits next to the totals, because every finding is only as trustworthy as it is small.

Views that never met are one diagnostic, not hundreds of findings. Argo CD registers /api in Go and documents 198 paths beneath it, so its code and its specification share not one endpoint. Read literally that is 58 shadow APIs and 198 phantom contracts, none of them real. Zero corroboration between two populated views means the comparison did not work — a mount point standing in for the routes beneath it, or a stack noir could not read — so the rules are held back and the reason is printed instead. Paths that turn out to have many endpoints from other views beneath them are labelled as probable mounts.

A missing view and an empty one mean opposite things. Noir reports what it could not read, and alibi prints that above the findings. NetBox ships a 12.35MB OpenAPI document with 308 paths; noir skips it for exceeding its file-size cap, and without that report alibi states the project documents nothing — not merely incomplete, but the wrong answer stated confidently.

A rule that stopped running has not resolved anything. Recording scans and comparing them reintroduces the same mistake at a distance: forget the contracts directory on one run and SHADOW evaluates nothing, which to a naive difference looks exactly like every shadow API having been closed. On the five-view fixture, dropping one argument turned seven standing findings into "resolved". Snapshots record which rules evaluated, differences only consider rules that ran in both scans, and the rest are named under NOT COMPARED.

An absence is only evidence when the signal exists. Noir's auth taggers cover the frameworks they know. In a stack they do not cover, nothing carries an auth tag, and treating that as "unauthenticated" would promote every finding and drain the severity column of meaning. Adjustments that fire on a missing tag require that tag to appear somewhere in the scan first.

Rules

Severity then shifts on what noir's taggers found: personal data, file uploads, no sign of authentication, or a method that changes state.

Both the view map (views.yml) and the rules (rules.yml) are data, not code.

Gateways are not endpoint lists

One location /api/ stands for everything beneath it, so gateway and infrastructure rules answer does this reach that endpoint rather than does this contain it. Compared as sets, every prefix rule looks like a route nobody implemented and every implemented route looks unreachable.

Coverage is deliberately generous. Noir reports the path a rule matches but not whether it matches as a prefix or exactly (location = /x, an Ingress pathType: Exact), so exactness cannot be recovered — and treating every rule as a prefix suppresses findings rather than inventing them.

The report says how much of the code each routing view reaches, because whether "34 endpoints no gateway reaches" is real depends on whether that config is the one fronting the service. No threshold separates those honestly: Argo CD's e2e test fixture reaches 39% of its code and NetBox's real config reaches 100%.

Traffic has to have been watched

A HAR capture records requests that happened. A Postman collection records requests somebody meant to make. ORPHAN, LIVE_UNDOC and COLD all reason about what ran, so they require a view somebody actually watched and say so when they sit out.

Non-web surface stays out

Noir reports CLI arguments, Kafka topics and mobile deep links in the same list. cli://gitops-engine/agent flattened into HTTP becomes /agent — it collides with any web route of that name and gets asked whether a gateway routes to it. The protocol belongs to the endpoint's identity; http and https are one space and everything else keeps its own.

Suppression

Some gaps are the intended state. Put an .alibi.yml beside the source:

ignore:
  - path: "^/internal/"
    why: internal-only admin surface
  - rule: UNEXPOSED
    path: "^/debug/"
    why: not fronted by the gateway in this repo

Or pass --ignore REGEX for a one-off. Suppressed findings are counted and the count is printed — a tool that quietly drops findings is worse than one that prints too many, because there is no longer any way to tell what it withheld.

Status

Early, but all five views are compared.

Measured against five repositories:

Casdoor is the cleanest case: 230 of its 235 documented endpoints matched the code, with no path normalization failures at all. All 19 near misses were the same path under a different verb — noir registering every method on a Go catch-all handler, not a matching problem.

NetBox is the instructive one. It holds two surfaces in one repository: a server-rendered web UI and a DRF-router REST API that only the second is documented. Scanned whole it reports 746 findings, most of them the true but useless observation that a web UI is not in an API specification. Scoped to the surface the contract describes, it collapses to what was actually worth saying:

$ alibi scan ./netbox --ignore '^/(?!api/)'

→ 3 shadow APIs: /api/plugins, /api/schema/redoc, /api/schema/swagger-ui, all three genuinely served and genuinely absent from the schema. The 397 phantoms that remain are the bulk operations NetBox's own router subclass adds to every list endpoint, which no urlconf walk can see.

The other three are held back, each for a reason worth knowing:

• Argo CD registers /api in Go and documents 198 paths beneath it — the same surface at two granularities.
• authentik assembles its URLconf at runtime by importing every installed app's urls module, which no static reader can follow.
• flipt mounts a gRPC gateway; its Go source holds one route.

Which is this tool's ceiling, stated plainly: it compares what noir can read, and a view read at the wrong granularity is worse than one not read at all. Most of the machinery above exists to tell those apart rather than to report them as defects.

License

MIT