hypergumbo

hypergumbo is a local-first CLI that generates surveys and sketches from source code. The goal of this project is to efficiently help developers and LLMs understand a codebase.

pip install hypergumbo

Requires Python 3.10+. For optional extras (embeddings, gitleaks, grammars), run hypergumbo add-extras after installing.

Intel Mac users: Some tree-sitter packages lack x86_64 wheels. See docs/INTEL_MAC.md for a Docker-based workaround.

git clone https://github.com/iterabloom/hypergumbo
hypergumbo hypergumbo/

Output:

# hypergumbo

hypergumbo is a local-first CLI that generates surveys and sketches from source code. The goal of this project is to efficiently help developers and LLMs understand a codebase. > Requires Python 3.10+. For optional extras (embeddings, gitleaks, grammars), run `hypergumbo add-extras` after installing. > Intel Mac users:

> **Audited IO surface.** hypergumbo's own filesystem / network / subprocess activity is enumerated per CLI subcommand category in [SECURITY.md](SECURITY.md), with machine-verifiable claims authored at `docs/hypergumbo.claims.yaml`. Re-run the audit locally via `hypergumbo verify-claims --claims docs/hypergumbo.claims.yaml`. The wrapper-function discipline used to make path-bounded claims structurally verifiable is documented as the recommended pattern for any hypergumbo user (any language) wanting the same precision.

## Overview
Python (91%), Markdown (4%), Yaml (3%)
728 files    (383 non-test + 345 test)
~320,798 LOC (~129,172 non-test + ~191,626 test)

## Structure

` ` `
hypergumbo/
├── .agent
│   └── [and 6 other items]
├── .gitea
│   ├── SQUASH_TEMPLATE.md
│   └── [and 1 other items]
├── .githooks
│   ├── commit-msg
│   └── [and 9 other items]
├── docs
│   ├── CACHE.md
│   └── [and 22 other items]
├── packages
│   ├── hypergumbo-core
│   │   ├── src
│   │   │   └── hypergumbo_core
│   │   │       ├── analyze
│   │   │       │   ├── base.py
│   │   │       │   └── [and 3 other items]
│   │   │       ├── __main__.py
│   │   │       ├── cli.py
│   │   │       ├── ir.py
│   │   │       └── [and 26 other items]
│   │   ├── tests
│   │   │   ├── test_framework_patterns.py
│   │   │   └── [and 94 other items]
│   │   └── [and 2 other items]
│   ├── hypergumbo-tracker
│   │   ├── src
│   │   │   └── hypergumbo_tracker
│   │   │       ├── cli.py
│   │   │       └── [and 13 other items]
│   │   └── [and 5 other items]
│   └── [and 4 other items]
├── scripts
│   ├── lib
│   │   └── forgejo-api.sh
│   └── [and 33 other items]
├── tests
│   ├── test_bakeoff_deep_reflect.py
│   └── [and 2 other items]
├── conftest.py
├── pyproject.toml
├── setup.py
└── [and 21 other items]
` ` `

## Frameworks

- pytest
- pytorch
- transformers

## Tests

345 test files · cargo test, pytest, unittest

*~95% estimated coverage (2693/2847 functions called by tests)*

## Configuration
[...]

See full example output

Use -t to control the token budget:

hypergumbo . -t 1000   # brief overview (structure only)
hypergumbo . -t 4000   # good balance for most LLMs
hypergumbo . -t 8000   # detailed with many symbols

Two Outputs

Sketch (hypergumbo .) — Token-budgeted Markdown sized for LLM context windows. Ranks symbols by graph centrality (★ = most connected).

Survey (hypergumbo survey) — Full JSON with all symbols, edges, and provenance tracking. Use this for programmatic analysis.

CLI Commands

hypergumbo [path]              # Markdown sketch (default)
hypergumbo survey [path]       # Full JSON survey
hypergumbo slice --entry X     # Subgraph from entry point
hypergumbo io-boundaries       # Find all I/O (filesystem, network, subprocess, env, IPC, browser storage)
hypergumbo verify-claims ...   # Verify security claims against analysis
hypergumbo routes [path]       # List HTTP routes
hypergumbo search <query>      # Search symbols
hypergumbo symbols [path]      # Browse symbols with connectivity
hypergumbo explain <symbol>    # Detailed symbol info
hypergumbo test-coverage       # Analyze test coverage (transitive)
hypergumbo catalog             # List analysis passes

Useful flags:

hypergumbo . -x                                # exclude test files (cleaner output)
hypergumbo . --no-source                       # omit source code (included by default)
hypergumbo . --no-progress                     # hide progress indicator (sketch/run only; on by default)
hypergumbo io-boundaries --show-external-potential   # opt into the (large) external-potential bucket (text output)
hypergumbo --help --all                        # comprehensive help for all commands

Project-local taint catalogs

verify-claims ships with paranoid defaults auto-derived from the built-in IO primitive catalog. Projects can supply their own trust zones, sanitizers, and label maps:

hypergumbo verify-claims --claims claims.yaml \
    --taint-sources    myrepo/taint/sources.yaml \
    --taint-sinks      myrepo/taint/sinks/ \
    --taint-sanitizers myrepo/taint/sanitizers.yaml

Each flag accepts a YAML file or a directory (globbed as *.yaml), and is repeatable. The same paths can be declared inside the claims YAML under extra_catalogs: {sources, sinks, sanitizers} — relative paths resolve against the claims-file directory. User entries whose (module, name, kind) triple matches a built-in replace it; sanitizers concatenate.

Results are automatically cached in ~/.cache/hypergumbo/. Just run:

hypergumbo .    # auto-runs analysis if no cache exists, then generates sketch

The cache auto-invalidates when source files change. See docs/CACHE.md for details.

See hypergumbo --help for all options.

What It Understands

• Language analyzers: Python, JS/TS, Java, Rust, Go, C/C++, and many more
• Linkers: Tier 2 edge-recovery passes across four subcategories — Protocol (HTTP, WebSocket, message queues, SQL), Bridge (JNI, wasm_bindgen, Tauri IPC, language-pair FFI), Framework (gRPC, GraphQL, React components, DI resolution, ORM), Infrastructure (containment, inheritance, module imports). Full catalogue.
• Framework patterns: FastAPI, Django, Rails, Spring Boot, Phoenix, Express, and many more
• I/O boundary detection: Maps every call chain that reaches the filesystem, network, subprocess, environment, IPC, or browser-local storage — across FFI boundaries
• Taint-flow analysis: Traces data from sensitive sources (environment variables, received network input, crypto outputs, key material) to sinks in six trust zones (host_fs, network, host_env, ipc, browser_storage, relay), with sanitizer awareness
• Supply chain tiers: Classifies code as first-party, internal, external, or derived for dependency-aware analysis

How It Works

1. Profile: Scan the repo for languages, file counts, LOC
2. Analyze: Run language-specific analyzers to extract symbols and edges
3. Link: Connect symbols across language boundaries (JS fetch → Python route)
4. Enrich: Detect frameworks via YAML pattern matching
5. Classify: Assign supply chain tiers (first-party, internal, external, derived)
6. Trace I/O: Map call chains to I/O boundaries; run taint-flow analysis
7. Output: Generate Markdown sketch or JSON survey

The Internal Representation

All analyzers produce the same IR types:

• Symbol: A code element (function, class, method) with name, location, and stable ID
• Edge: A relationship between symbols (calls, imports, extends, implements)
• Span: Source location (file, line, column)

This uniform IR is what allows all language analyzers and linkers (Protocol / Bridge / Framework / Infrastructure — see ADR-3bbb) to work together coherently.

Architecture

packages/
├── hypergumbo-core/           # CLI, IR, slice, sketch, linkers
│   └── src/hypergumbo_core/
│       ├── cli.py             # Entry point
│       ├── ir.py              # Symbol, Edge, Span
│       ├── sketch.py          # Token-budgeted Markdown
│       ├── slice.py           # Subgraph extraction
│       ├── linkers/           # Tier 2 edge-recovery passes (Protocol/Bridge/Framework/Infrastructure)
│       └── frameworks/        # Framework detection (YAML patterns)
├── hypergumbo-lang-mainstream/  # Python, JS, Java, Go, Rust, etc.
├── hypergumbo-lang-common/      # Haskell, Elixir, GraphQL, etc.
├── hypergumbo-lang-extended1/   # Zig, Solidity, Agda, etc.
├── hypergumbo-tracker/           # Structured work tracker for agent governance (MPL-2.0)
└── hypergumbo/                  # Meta-package (installs all above)

Key design choices:

• Registry pattern: Analyzers and linkers self-register via decorators
• Two-pass analysis: First collect symbols, then resolve edges (enables cross-file references)
• Provenance tracking: Every edge records which analyzer/linker created it
• YAML-driven patterns: Framework detection is declarative, not hardcoded

Development

git clone https://github.com/iterabloom/hypergumbo.git
cd hypergumbo
python3 -m venv .venv && source .venv/bin/activate
./scripts/dev-install
source .venv/bin/activate  # reload to enable pytest alias
pytest                      # runs smart-test (affected tests only)

dev-install installs all packages, git hooks, and the pytest/smart-test wrapper. 100% test coverage required.

See CONTRIBUTING.md for PR workflow (including fork-based workflow for external contributors), smart test selection setup, and coverage requirements. Agent instructions live in AGENTS.md.

Links

• docs/USE-CASES.md — Practical workflows and examples
• docs/VERIFY-CLAIMS-SCOPE.md — What verify-claims can and cannot see: the verdict ladder and exit-code contract, the caveat vocabulary, and what a clean verdict does not mean. Read this before gating CI on it.
• docs/RELEASE-NOTES-8.X.md — User-facing release notes for the current (8.x) line; earlier lines are 7.x, 6.x, 5.x
• CHANGELOG.md — Implementation history
• docs/LANGUAGES.md — Supported languages
• docs/LINKERS.md — Linkers catalogue (Protocol / Bridge / Framework / Infrastructure)
• docs/FRAMEWORKS.md — Framework patterns
• docs/hypergumbo-spec.md — Detailed specification
• docs/CITATIONS.md — Paper citations for embedding models
• docs/CACHE.md — Caching architecture
• docs/agent-supervisor.md — Operator guide for scripts/agent-supervisor (the tmux-session watchdog for autonomous agents)
• SECURITY.md — Vulnerability reporting
• hypergumbo-tracker README — Standalone tracker for AI agent governance

License

AGPL-3.0-or-later