intel: the intelligence sidecar

Propose-only, off the hot path, opens no listener. It reads samples and quarantine through the admin API (contracts/admin.openapi.yaml), scores structural drift, types unknown fields from value shapes, generates a parser proposal, asks the data plane to dry-run it, and posts it for a human. It never touches the vault, the parser store or the pipeline, and no code path can call approve, reject, rollback or replay (a test asserts the request log).

cd intel
python3.13 -m venv .venv && . .venv/bin/activate          # 3.11+ required
pip install -e '.[test]'
pytest                                                     # 100+ tests, warnings are errors

python -m ulpf_intel --admin http://127.0.0.1:9000 --state-dir ./intel-state   # the loop
python -m ulpf_intel --once                                # one pass and exit
python -m ulpf_intel --healthcheck                         # GET /healthz; exit 0 healthy, 1 not (no shell in the image)
python -m ulpf_intel --egress-check                        # exit non-zero if ANY outbound attempt succeeds

--egress-check is the air-gap probe: a public DNS lookup and TCP connects to 1.1.1.1:443 and 8.8.8.8:53, 2 s each. All three are expected to fail; on a machine with internet access it correctly reports FAIL.

What each module does

The loop, per source

1. Fetch recent samples (parsed and quarantined). Fewer than min_samples (50): do nothing.
2. Compare with the stored baseline. No baseline and a high quarantine rate: "never parsed".
3. Score >= 0.35, or quarantine rate > 0.2: post one DriftAlert (reused while open).
4. At least min_cluster (50) quarantined records of one shape: generate a proposal, dry-run it via POST /admin/parsers/dryrun, post it. One proposal per alert.
5. A proposal that misses its thresholds (match_rate >= 0.95, mean_coverage >= 0.9) is still posted, with acceptance: warnings, so the UI shows it as needing edits. It is never dropped.
6. The baseline is refreshed only from healthy passes, never while the source is drifting.

Measured (fixtures, reference evaluator; the data plane's dry-run is authoritative and not built yet)

Known limits

• ASA direction. On ASA the first address is the foreign host: the source of an inbound connection, the destination of an outbound one. The proposal reads the words inbound/outbound, then cue words (src, dst, from), ->, port behaviour, and last position, flags every weak decision with the other path as an alternative, and the reviewer confirms. It agrees with the manifest on 1725/2000 records; the 275 it gets wrong are Teardown TCP (302014), which carries no direction word at all (only the matching Built message with the same connection id would say). A test pins the split.
• CEF/LEEF are tested on SYNTHETIC samples only (no CEF/LEEF file exists in the fixture corpus), and the severity banding (0-3 Low ... 9-10 Very-High to OCSF 2..5) follows the CEF specification and is stated to the reviewer in every proposal. The optional local-LLM step names only anonymous csv columns; it does not improve mappings or free-text slot names.
• The reference evaluator has no render-back, so render_back_ok_rate comes only from the data plane's dry-run.
• Timestamps without a zone use --timezone (default +00:00) and every such proposal says so in its warnings.
• "Stale" marking of proposals whose base version moved is done by the data plane on approve (409); the sidecar only avoids re-proposing.