okfgen

Point it at a repo, database, docs site, or open-data portal.
Get a portable, agent-ready knowledge graph in seconds — no LLM, no API key, no lock-in.

▶ Explore the live interactive knowledge graphs →

okfgen is a deterministic reference implementation of both sides of the Open Knowledge Format (OKF) — Google's vendor-neutral standard for representing the knowledge around your data and systems as just markdown files with YAML frontmatter (announcement).

• Producers turn a source system, database, docs site, or live open-data portal into a bundle.
• Consumers read a bundle back out — a viewer, a search index, a reasoning agent.

It extracts structured facts straight from the source (schemas, file structure, READMEs, dependency manifests, page headings). No LLM and no API key are required; an optional --llm flag adds Claude-powered enrichment where you want it.

              PRODUCERS                              CONSUMERS
   git repo  ─┐                          ┌─  visualize  → interactive HTML graph
   database  ─┤                          ├─  search     → full-text index
  open data  ─┼─►  generate ─► BUNDLE ─► ┼─  ask        → reasoning agent
   local dir ─┤        │       (.md +    └─  validate   → conformance check
   web docs  ─┘        ▼      frontmatter)
                    enrich  (pass 2: join paths, backlinks, citations)

Quickstart (30 seconds)

Zero-install with uv — turn this directory into a knowledge graph and open it:

uvx okfgen generate . -o my-okf
uvx okfgen visualize my-okf -o my-okf/graph.html
# then open my-okf/graph.html

Or with pip:

pip install okfgen
okfgen generate . -o my-okf && okfgen visualize my-okf -o my-okf/graph.html

🎥 See the demo above, or explore the live gallery — every graph is fully interactive.

Why okfgen

• One command, any source → a knowledge graph. Code, databases, docs, and live open-data portals — the same tool, the same output.
• Deterministic, offline, no API key. Reproducible facts, not LLM hallucinations. Runs in air-gapped environments. The LLM is strictly opt-in.
• Open format, zero lock-in. Output is plain markdown + YAML you can read, diff in git, and grep — not a proprietary database.
• Agent-ready. Search, a citation-backed reasoning agent, and a portable JSON index make bundles first-class context for RAG and AI agents.
• Provenance built in (OKF v0.2). Every concept records how it was produced (generated) and what it derives from (sources) — with real credibility signals where the source exposes them: publisher, view counts, and freshness dates from live open-data portals.
• A viewer you can email. The visualizer is a single self-contained HTML file — no backend, no CDN, data never leaves the page.
• Reference implementation of an open standard. Tracks the OKF v0.2 spec; every bundle it emits passes its own conformance validator.

How it compares

Install

pip install okfgen                 # core: git, local, web, schema (zero deps)
pip install "okfgen[all]"          # + BigQuery, Firebase, MCP, PyYAML

Optional extras: [bigquery], [firebase], [mcp], [yaml], [dev]. For development from a clone: pip install -e '.[dev]'.

Producers — make a bundle from your data

okfgen generate https://github.com/psf/requests.git   # a source system (git)
okfgen generate ./my-project                          # a source system (local)
okfgen generate schema:./warehouse.schema.json        # a database (offline)
okfgen generate schema:./ddl.sql                      # a database (SQL DDL)
okfgen generate bq:my-gcp-project                     # BigQuery datasets/tables
okfgen generate firebase:my-firebase-project          # Firestore collections
okfgen generate https://docs.mytool.dev/              # a documentation site
okfgen generate ckan:https://portal/dataset/some-set  # a live CKAN open-data portal
okfgen generate socrata:https://data.cityofnewyork.us/d/erm2-nwe9  # a live Socrata dataset

Cloud sources use Application Default Credentials (gcloud auth application-default login). Output goes to ./<name>-okf/.

The enrichment agent (pass 2)

Producers draft concepts; the enrichment agent enriches them — exactly the two-pass pattern from the OKF blog. Deterministically, it infers join paths between tables from foreign-key naming (customer_id → customers) and wires backlinks so the graph is navigable both ways:

okfgen enrich ./my-okf                 # in place
okfgen enrich ./my-okf -o ./enriched   # to a new directory
okfgen enrich ./my-okf --llm           # also rewrite descriptions via Claude

Consumers — read a bundle back out

The OKF value proposition is producer/consumer independence: any consumer works on any bundle, regardless of who produced it.

# Viewer: a self-contained interactive graph (no backend, no CDN, data stays local)
okfgen visualize ./my-okf -o graph.html

# Search index: full-text, TF-IDF ranked
okfgen search ./my-okf "weekly active users"
okfgen search ./my-okf --export index.json      # portable JSON index

# Reasoning agent: retrieves concepts, follows join links, answers with citations
okfgen ask ./my-okf "how do orders relate to customers?"
okfgen ask ./my-okf "..." --llm                 # phrase answer via Claude

# Conformance validation
okfgen validate ./my-okf --strict

# Agent Skill: turn a source (or bundle) into a SKILL.md + reference files
okfgen skill ./my-repo -o my-skill              # from any source
okfgen skill ./my-okf  -o my-skill --llm        # from a bundle; sharpen the description

okfgen ask shows its work — the retrieved concepts, the links it traversed, and the citations behind the answer — so the reasoning is auditable.

okfgen skill produces an Agent Skill folder: a SKILL.md (with a triggering description and lean instructions) plus the OKF bundle alongside as reference files — progressive disclosure, so an agent loads detail only when it needs it.

Use it inside your AI agent (MCP)

okfgen ships an MCP server, so Claude Desktop, Claude Code, Cursor, and any Model Context Protocol client can produce and reason over OKF bundles without leaving the agent.

pip install "okfgen[mcp]"
okfgen-mcp            # stdio MCP server

Register it (e.g. Claude Desktop claude_desktop_config.json, or Cursor's MCP settings):

{
  "mcpServers": {
    "okfgen": { "command": "okfgen-mcp" }
  }
}

Exposed tools: okfgen_generate, okfgen_search, okfgen_ask, okfgen_validate, okfgen_visualize, okfgen_skill, okfgen_list_source_types. Now an agent can say "catalog this database and tell me how orders join to customers" and get grounded, cited answers.

Sample bundles

Browse the sample knowledge graphs online: https://bushans.github.io/okfgen/

Ready-to-browse bundles live in samples/bundles/. Open any graph.html in a browser, or point the consumers at them. The same visualizers are published to GitHub Pages from docs/ (regenerate with python samples/build_pages.py).

• Three offline, reproducible bundles (database, source system, docs site): python samples/build_samples.py
• One live public-data bundle — Toronto Beaches Water Quality from the Toronto Open Data CKAN portal: python samples/build_live_samples.py

See samples/README.md for details.

Output layout

<name>-okf/
├── index.md            # root listing + okf_version: "0.2"
├── log.md              # generation / enrichment log (ISO-dated)
├── overview.md         # the root "Project" / "Data Project" concept
├── dependencies.md     # parsed manifests (git/local)
├── docs/…              # documentation concepts
├── modules/…           # per-directory code concepts (git/local)
├── datasets/… tables/… # database / BigQuery concepts
├── collections/…       # Firestore concepts
├── pages/…             # web page concepts
└── graph.html          # (after `visualize`) the interactive viewer

Every concept carries the required type frontmatter field plus recommended title/description/resource/tags, and an OKF v0.2 generated: { by, at } provenance stamp. Bodies use the conventional OKF # Schema, # Examples, # Computation, # Joins headings.

Design notes

• Deterministic by default. git/local/web/schema run on the standard library alone (zero third-party deps). Cloud SDKs and the LLM are optional extras, loaded lazily and off unless you ask.
• Producer/consumer split. Consumers depend only on markdown + frontmatter (okfgen/consumer.py), never on producer internals.
• Scriptable. Every command prints its primary output path to stdout and logs to stderr: BUNDLE=$(okfgen generate ./repo).

Development

pip install -e '.[dev]'
pytest

New to the project? TESTING.md is a step-by-step VS Code walkthrough: environment setup, running the test suite, and driving every producer/consumer command locally.

Contributing

Contributions are welcome — new source adapters especially. See CONTRIBUTING.md for the dev setup and a guide to adding a producer in a few lines.

License

Apache-2.0.