yubel 0.8.1

the ever-watchful guardian — all-seeing dynamic security

A cloud-native, multi-target DAST orchestrator with a correlation brain.

Yubel runs the best open-source dynamic security engines — ZAP, Nuclei, Nikto, Wapiti, testssl.sh, sqlmap, dalfox, katana, httpx, schemathesis, graphw00f, graphql-cop, kube-hunter — against web apps, REST/GraphQL APIs, cloud assets, containers and Kubernetes, then does what no single scanner can: it correlates their output into corroborated findings and synthesizes multi-step attack chains. Runs installed (pip), in Docker, as a Kubernetes Job/CronJob, or in CI.

_{web · REST/GraphQL APIs · cloud · containers · Kubernetes  |  consensus · attack chains · systemic correlation · baseline diff · OWASP/CWE/MITRE}

🛡️ Air-gapped by design

Yubel needs no LLM and no cloud to work. Its core makes zero outbound calls — it only ever talks to the targets you point it at — so results never leave your perimeter. Add --offline to harden the engines further — today that disables nuclei's OAST/interactsh callbacks and its template update check; the other engines are not covered yet, so treat it as defence in depth, not a guarantee. Runs fully inside regulated, isolated and on-prem networks where AI-driven tools simply can't go. Deterministic, reproducible, auditable — the same scan yields the same result, every time.

Why Yubel

No single scanner is best at everything. ZAP excels at deep web crawls (Yubel does not pass credentials to it yet — see the AUTH column in docs/engines.md); Nuclei is unbeatable for templated checks and fast fuzzing; testssl.sh owns TLS; kube-hunter is the way to dynamically pentest a cluster; schemathesis tears apart OpenAPI contracts. Real programs already run several of these — badly glued together with bespoke scripts, incompatible reports and no shared severity model.

Yubel is the glue, done properly:

• One config, many engines. Describe your targets once; Yubel routes each to every engine that can handle it.
• One normalized model. Every engine's output becomes a Finding on a shared 5-level severity scale, then duplicates across engines are merged (and credited).
• One set of reports. JSON, a self-contained HTML dashboard, Markdown, and SARIF for GitHub code scanning — from the same run.
• Runs anywhere. Laptop, Docker, Kubernetes Job/CronJob, or a CI pipeline with a real fail-gate.
• Honest and resilient. A missing engine is skipped, not fatal. One engine crashing never kills the scan. Every execution is recorded.

Yubel does not reinvent scanning — it stands on the shoulders of the OSS ecosystem catalogued in docs/LANDSCAPE.md (382 tools surveyed).

What makes Yubel different

Most tools stop at "run scanner, print findings". Because Yubel sees every engine's output for a target at once, it adds an analysis layer that single scanners structurally cannot:

• Cross-engine consensus. A finding independently reported by two or more engines is flagged corroborated and its confidence is upgraded — cutting false positives by using the engines as a jury, deterministically (no probabilistic LLM validator). Duplicates merge, keeping the worst severity and crediting every reporter.
• Attack-chain synthesis. Yubel recognizes when separate findings combine into a real exploitation path and promotes it to its own high-impact finding — 13 rules today, e.g. SSRF + cloud host → IMDS credential theft, XSS + non-HttpOnly cookie → account takeover, anonymous K8s API + exposed kubelet → cluster takeover, JWT alg=none + admin → auth bypass, request smuggling + cache poisoning. No isolated scanner reports these.
• Systemic correlation across targets. When the same weakness class shows up on two or more targets, Yubel raises a single systemic finding — "fix centrally, resolve everywhere". A tool that sees one app at a time structurally cannot.
• "Why we believe this" evidence trail. Every finding carries a deterministic, reproducible rationale (which engines, corroboration, taxonomy, risk) — auditable reasoning instead of a model's guess.
• Baseline diff. Point --baseline at a prior run and every finding is tagged new / existing / regressed, with fixed issues tracked separately. --fail-on-new gates CI on newly introduced risk only, so a backlog never blocks a pipeline.
• Taxonomy + risk scoring. Every finding is mapped to OWASP Top 10 (2021), OWASP API Top 10 (2023), CWE and MITRE ATT&CK, given a composite 0–100 risk score, and rolled up into a per-target A–F grade and an OWASP coverage matrix.
• Noise clustering. 200 "missing header" hits collapse into one finding with an instance count, so real issues aren't buried.

Yubel is the conductor, not another instrument — it can even orchestrate an AI-agent scanner as just one more engine, while its own core stays deterministic and offline. See a rendered report: docs/sample-report.html.

How it works

Quick start

pip install yubel     # core (orchestrator + reporters + analysis)

yubel setup                # show what's missing and how to install it
yubel setup --install      # actually install what it can (brew/pip/go)
yubel engines              # what's registered, available, and carries auth
yubel selftest             # synthetic run — validates the pipeline, no network
yubel scan -t https://example.com --fail-on high -o report/

Prefer everything bundled? The Docker image ships the orchestrator and the engines:

docker run --rm -v "$PWD/out:/out" ghcr.io/ggeorgeazevedo/yubel:latest \
  scan -t https://example.com -o /out
open out/yubel.html

On Linux without Homebrew, yubel setup --install cannot fetch nikto, testssl or zap — use the Docker image, or install those three by hand. Per-engine options, timeouts and the authentication matrix live in docs/engines.md.

Targets it understands

sqlmap is intrusive and runs only when named explicitly (-e sqlmap) or with --include-intrusive. grpc exists in the target enum but no engine covers it yet — a config that targets it now fails validation instead of writing an empty report. Which engines carry your credentials is a separate question: run yubel engines and read the AUTH column.

A real config

yubel init            # writes a documented yubel.yaml
yubel scan -c yubel.yaml -o report/

parallelism: 4
fail_on: high            # CI gate: exit non-zero if any finding >= high
targets:
  - { name: shop, type: web, url: https://shop.example.com,
      auth: { kind: bearer, token: ${SHOP_TOKEN} } }
  - { name: payments, type: api, url: https://api.example.com,
      openapi: https://api.example.com/openapi.json }
  - { name: graph, type: graphql, url: https://api.example.com/graphql }
  - { name: prod-cluster, type: kubernetes, host: 10.0.0.10, k8s_mode: remote }
output: { dir: report, formats: [json, html, markdown], sarif: true }

Secrets stay out of git via ${ENV_VAR} expansion.

Deploy

kubectl apply -f deploy/k8s/configmap.yaml
kubectl apply -f deploy/k8s/job.yaml
kubectl logs -f job/yubel-scan

helm install yubel deploy/helm/yubel \
  --set mode=cronjob --set schedule="0 2 * * *"

- uses: ggeorgeazevedo/yubel@v0
  with:
    target: https://staging.example.com
    fail-on: high
- uses: github/codeql-action/upload-sarif@v3
  if: always()          # a failing gate must not swallow the findings
  with: { sarif_file: yubel-report/yubel.sarif }

permissions:
  contents: read
  security-events: write     # required by upload-sarif

steps:
  - uses: ggeorgeazevedo/yubel@v0
    with:
      target: https://staging.example.com
      fail-on: high
      upload-sarif: true

Baseline / trend scanning

yubel scan -c yubel.yaml -o report/                      # run 1
cp report/yubel.json baseline.json
# ... later, in CI ...
yubel scan -c yubel.yaml --baseline baseline.json \
           --fail-on critical --fail-on-new -o report/   # fails only on NEW criticals

Reports

• yubel.html — editorial, self-contained assessment report: executive summary with an A–F risk grade, an attack-paths section, an OWASP Top 10 coverage matrix, and risk-scored findings with corroboration and taxonomy chips. Works offline, light/dark, print-friendly.
• yubel.sarif — SARIF 2.1.0 (with security-severity from the risk score + OWASP/MITRE tags) → GitHub Security → Code scanning.
• yubel.json — full machine-readable result: findings, enrichment, per-engine run log and the baseline diff.
• yubel.md — Markdown summary (exec summary, attack paths, OWASP coverage, findings) for PRs/tickets.

Pipeline

target ─▶ DISCOVERY ── katana / httpx crawl the target first
                    └─ up to crawl_max_urls (150) discovered URLs are seeded
                       into the parameter scanners  [--no-crawl to skip]
        ─▶ registry.select_for(target) ─▶ [engines for this type, minus deny-list,
                                           minus intrusive/opt-in]
        ─▶ ThreadPool (bounded by parallelism)
        ─▶ each engine: build_command → run in temp workdir → parse → [Finding]
        ─▶ collect ─▶ dedupe (merge cross-engine dups, keep worst severity)
        ─▶ ANALYSIS ── taxonomy (OWASP/CWE/MITRE) + risk score
                    ├─ consensus (corroboration → confidence uplift)
                    ├─ cross-target correlation (same flaw on N targets = 1 fix)
                    ├─ cluster noise (info/low repetition → 1 finding)
                    ├─ attack-chain synthesis (composite findings)
                    ├─ remediation KB (deterministic fix per finding)
                    ├─ confirmed / needs-review tier
                    └─ baseline diff (new / existing / regressed / fixed)
        ─▶ reporters (json/html/md/sarif) ─▶ fail-gate exit code

Discovery is where coverage is decided. crawl_max_urls defaults to 150 and the cap is logged, never a silent truncation. --no-crawl turns the phase off; --crawl-headless runs katana with a headless browser for JS-heavy SPAs.

Intrusive engines (currently sqlmap) are off by default and only run when you pass --include-intrusive or name them explicitly with -e sqlmap.

Adding an engine

1. Subclass Engine in src/yubel/engines/, declaring name, supports, binary — and header_flag with the tool's add-a-header flag. Without header_flag the engine never receives credentials and will scan anonymously while reporting a normal result. Check the spelling against the tool's --help and set header_style (colon or json) to match.
2. Implement build_command() and parse() (return normalized Findings).
3. Register it in engines/registry.py.
4. Run python3 scripts/gen_engines.py and commit docs/engines.md. If your adapter reads a new options key, add a description for it in that script — CI fails on an undocumented option.

Availability probing, target routing, CLI listing, parallelism, auth and reporting then come for free. See engines/nuclei.py for a compact reference and CONTRIBUTING.md.

Releasing (how the badges go green)

The badges light up automatically once the repo is on GitHub:

• CI / CodeQL — run on every push/PR (green after the first successful run).
• License / Stars — populate as soon as the repo is public.
• Release / PyPI / Docker — populate when you cut a release:

git tag vX.Y.Z && git push origin vX.Y.Z

That tag triggers release.yml (builds the wheel, publishes to PyPI via Trusted Publishing, and creates a GitHub Release) and docker.yml (pushes the image to ghcr.io). One-time setup: on PyPI add a Trusted Publisher for this repo (Project → Publishing → GitHub) — no API token needed. Your repo's Community Standards checklist (Insights → Community Standards) is already 100%: README, LICENSE, CODE_OF_CONDUCT, CONTRIBUTING, SECURITY, issue templates and a PR template are all present.

Architecture

See docs/ARCHITECTURE.md for the full design and docs/LANDSCAPE.md for the surveyed ecosystem.

⚠️ Legal & ethical use

DAST is active testing: it sends real attack traffic. Only scan systems you own or are explicitly authorized to test. Intrusive engines can modify data. You are responsible for your use of Yubel. See SECURITY.md.

License

Apache-2.0. Yubel orchestrates third-party engines that carry their own licenses (GPL, Apache, MIT, proprietary, …) — installing or bundling them is your responsibility; see docs/LANDSCAPE.md for each tool's license.