myKG — Knowledge Graph Extractor

myKG automatically generates a confidence-scored knowledge graph from a set of mixed documents — Markdown, plain text, PDF, Word, PowerPoint, Excel, HTML, and images — grounded in an induced RDFS/OWL ontology.

Contents

• Features
• Quick Start
• Command line
• Articles & Tutorials
• Configuration
• Troubleshooting
• Extract Pipeline
  • Running
  • Sessions
  • Pipeline Steps
• Outputs
  • Re-running from a Specific Step
  • Orphan-Connection Pass
• Advanced Options
  • Human Review Gate
  • Locked Base Schema
  • SKOS Thesaurus
  • Website / Repo Fetching
  • Standalone Document Conversion
  • Append Mode
    • Incremental Schema Growth
  • Merging Sessions
  • Walkthrough Report
  • Obsidian Vault Export
  • Neo4j LOAD CSV Export
• MCP (Model Context Protocol) Server
• Using mykg with Claude Code
  • Agent mode (Claude Code skill)
  • claude-cli profile
• Roadmap
• Development
  • Testing
  • Healthiness check
• Design
• License

Features

MyKG builds trustworthy knowledge graphs through a self-evolving ontology that continuously adapts, maintains consistency, and assigns confidence scores to knowledge, keeping information grounded and reliable as it grows.

Ontology-Guided Extraction

• Schema-guided knowledge graph generation — the extracted graph is always grounded in a formal RDFS/OWL schema: concept types, property names, domain/range constraints, and the is-a hierarchy are explicit and inspectable before any entity is extracted

• AI coding assistant friendly — designed for smooth use alongside AI coding assistants such as Claude Code; run extractions, inspect outputs, and iterate on your knowledge graph without leaving your coding environment; see Using mykg with Claude Code

• Second brain for AI coding assistants — the Obsidian vault output turns your extracted knowledge graph into a directory of wikilinked Markdown notes that any AI coding assistant can read as project context; point Claude Code, Cursor, or Copilot at output/obsidian_vault/ and ask questions, trace relationships, and get answers grounded in your own documents

• MCP server for desktop AI apps — run mykg mcp-serve to expose your knowledge graph via the Model Context Protocol; integrates with Claude Desktop, Cherry Studio, and any MCP-compatible client — 13 query tools let LLMs search entities, explore relationships, find paths, traverse the graph, and read wiki notes directly from your extracted knowledge; see MCP Server

• Incremental updates — append new files to an existing session, extracting only what changed. Optionally grow the schema from new documents while preserving existing concepts and properties

• Resumable pipeline — every stage persists intermediate state; re-enter at any step after a crash or edit

• Session isolation — each run is fully self-contained; inputs, intermediate state, outputs, and logs co-located

• Cross-session merge — combine two independently-produced graphs into one unified knowledge graph

• Bring your own ontology — supply a --base-schema TTL file (RDFS or OWL) to lock in classes and properties from an existing formal ontology; the LLM expands it with domain-specific concepts but will not rename, remove, or contradict your authoritative vocabulary. Please note that this mechanism is controlled by the LLM, and may not be strictly enforced. Add --freeze-schema to skip LLM schema induction entirely and extract from the documents strictly against your ontology verbatim — no surprise types, no invented properties

• SKOS thesaurus support — pass --thesaurus to load a SKOS vocabulary; skos:exactMatch terms are collapsed silently, skos:closeMatch terms trigger a warning — giving the schema merger richer synonym awareness than string matching alone

• Verifiable TTL ontology — after Pass 1, the induced schema is exported as a valid RDFS/OWL Turtle file (intermediate/schema.ttl) that can be opened directly in ontology editors such as Protégé. The TTL is validated by rdflib (syntax + semantic checks: domain/range refer to declared classes, no conflicting ranges) before any extraction begins

• Human-in-the-loop ontology design — pause after schema induction with --review, edit the schema, and resume extraction. Edit schema.json directly or refine schema.ttl in Protégé and feed it back with --freeze-schema

Input

• Mixed-format corpora — point mykg extract-graph at any directory; supported extensions are converted to Markdown automatically before ingest:

  Format	Extensions	Backend
  Markdown	.md	passthrough (consumed as-is)
  Plain text	.txt	renamed to .md in-process
  PDF, Word, PowerPoint, Excel, images	.pdf .docx .doc .pptx .xlsx .png .jpg .jpeg	MinerU in an ephemeral uv-managed Python 3.12 venv — nothing is installed into your active environment
  HTML	.html .htm	markdownify in-process; anchors and image tags stripped
  Websites, GitHub repos	any URL	Crawlee in an ephemeral uv venv — produces an mykg_web_fetch/ folder

  Anything outside the allowlist (e.g. .svg, .css, .php assets next to an HTML bundle) is logged and skipped, never silently dropped. The allowlist is configurable via preprocess.extensions in mykg_config.yaml.

  Incremental conversion — unchanged source files are skipped on re-run. Adding one PDF to a corpus and re-running only re-converts that PDF. Force a full re-conversion with mykg extract-graph --from-step preprocess.

Graph & Output

• Provider-agnostic — works with Anthropic (Claude), OpenAI (GPT), Google Gemini, Ollama (local), OpenRouter, or the claude CLI
• Five output families — JSONL for Neo4j/NetworkX/RAG, Turtle RDF for OWL toolchains, NetworkX multi-format for graph analysis, Obsidian vault for linked personal knowledge management, and an optional Neo4j LOAD CSV bundle (plain-header CSVs + paste-and-run Cypher script for Neo4j Browser / cypher-shell)
• Obsidian vault — second brain for AI coding assistants — every extracted entity becomes a wikilinked Markdown note in output/obsidian_vault/; open it in Obsidian to navigate the graph with backlinks and Graph View, or point your AI coding assistant (Claude Code, Cursor, Copilot) at the vault folder so it can answer questions, trace relationships, and reason over your knowledge base in natural language
• Interactive HTML graph — node/edge filtering, search, hover popups; opens directly in a browser
• Confidence scoring — every extracted attribute, node, and edge carries a 0.0–1.0 confidence score

Quick Start

Requires Python 3.11+ (developed on macOS; automated CI runs the test suite on Ubuntu and Windows), and one of: an Anthropic/OpenAI/Gemini/OpenRouter API key, Ollama running locally, or the claude CLI.

Install from PyPI

Install mykg, then run the interactive setup wizard — it asks for your provider, model, and API key and writes mykg_config.yaml and .env.mykg in one step.

pip install mykg
mykg init

Then extract a knowledge graph from your notes:

mykg extract-graph my_notes/

Open mykg_sessions/<timestamp>/output/knowledge_graph.html in your browser to explore the result.

Install from source

Install uv, clone the repo, sync dependencies, run the setup wizard, then extract.

git clone https://github.com/SenolIsci/mykg && cd mykg
uv sync && uv run mykg init --force

Then extract a knowledge graph from your notes:

uv run mykg extract-graph my_notes/

For Ollama (local inference, no API key needed), pull a model and select the ollama-local profile when mykg init prompts you.

ollama pull llama3.3
mykg init
mykg extract-graph my_notes/

Command line

All Commands

mykg extract-graph my_notes/        # any directory: .md, .txt, .pdf, .docx, .html, images

It uses a two-pass LLM pipeline: Pass 1 induces a global RDFS/OWL schema from your document corpus; Pass 2 extracts typed entity and relationship instances per file against that schema. Non-Markdown inputs (.txt .pdf .docx .doc .pptx .xlsx .png .jpg .jpeg .html .htm) are converted to Markdown automatically before extraction. The result is exported to multiple formats: JSONL for property-graph consumers such as Neo4j, Turtle RDF for OWL toolchains, seven NetworkX formats for graph analysis and visualization, an Obsidian vault — a second brain of wikilinked Markdown notes your AI coding assistant (Claude Code, Cursor, Copilot) can read and reason over directly — and optionally a Neo4j LOAD CSV bundle with a paste-and-run Cypher script for one-step import into Neo4j Browser or cypher-shell.

Articles & Tutorials

Walkthroughs and case studies on Medium:

• How I Turned My Website Into a Knowledge Graph with myKG
• How to Build a Second Brain You Can Actually Trust
• Build an LLM Wiki for Your AI Agents
• From Documents to a Living Knowledge Graph: Introducing myKG

Configuration

All configuration lives in a single mykg_config.yaml file discovered automatically from the working directory (or any parent). There are no hardcoded defaults in the code — the YAML is the sole source of truth.

mykg init           # interactive: choose provider, model, paste API key
                    # writes mykg_config.yaml and .env.mykg in one step
mykg init --force   # overwrite an existing config
mykg init --profile openrouter-free --model google/llama-4-maverick --api-key sk-or-...  # non-interactive

The wizard walks you through three prompts:

1. Profile — choose your LLM provider (OpenRouter, Anthropic, OpenAI, Gemini, Ollama, Claude CLI, or Agent / Claude Code skill)
2. Model — accept the default or type any model slug for that provider (skipped in agent mode — the host Claude Code session is the LLM)
3. API key — paste your key (skipped for Ollama, Claude CLI, and agent mode)

LLM Providers

Switch provider by setting profile: at the top of mykg_config.yaml.

API Keys

myKG reads API keys from environment variables. Set them by exporting directly or by creating a .env.mykg file in your project directory (loaded automatically on startup).

Option A — export in your shell:

export ANTHROPIC_API_KEY=sk-ant-...

Option B — create a .env.mykg file:

# .env.mykg
ANTHROPIC_API_KEY=sk-ant-...

For source installs you can also copy sample.env.mykg to .env.mykg as a starting template.

Troubleshooting

Token Budgets

Each profile's llm: and pipeline: blocks carry a chain of token-budget values sized for that model's context window:

• llm.context_window — the model's total context limit
• llm.max_output_tokens — the output cap reserved for each LLM response
• pipeline.pass1.batch_token_target and pipeline.pass2.concat_batch_token_target / batch_token_target — input budget per LLM call, sized to (context_window − max_output_tokens) × 0.95
• pipeline.chunking.window_tokens / overlap_tokens — chunk size and overlap for splitting large files, sized to roughly batch_token_target / 4 and window_tokens × 0.10

The shipped values are tuned per profile (e.g. claude-cli/anthropic-claude assume a 200K context window, openrouter-free/ollama-local assume 64K). If you switch to a different model — especially on ollama-local or openrouter-free — check that model's actual context window and rescale these values, otherwise window_tokens + max_output_tokens may exceed what the model can actually handle, causing truncated or failed responses.

Use the bundled context-calculator tool to recompute the chain for a new model:

# Compute a full token-budget chain from context window + output cap:
context-calculator --context 64000 --max-output 32000

# Or measure your actual corpus and suggest values for the active profile:
context-calculator --from-config --input-dir my_notes/

--from-config reads the active profile from mykg_config.yaml, measures token counts across your input files, and writes suggested values to mykg_config_candidate.yaml for review before copying them into mykg_config.yaml.

Diagnosing Truncated/Rejected LLM Calls

On JSON parse or validation errors in Pass 1/2, check run.log for a mykg.llm.retry warning just above the failure — always logged, regardless of logging.llm_log:

• output truncated (finish_reason=...) — hit max_output_tokens. Lower pipeline.chunking.window_tokens or raise llm.max_output_tokens, then recompute with context-calculator (see Token Budgets).
• context length exceeded, request rejected: ... — input didn't fit the context window. Lower batch_token_target (pipeline.pass1/pipeline.pass2).

Pass 2 Prep Mode (pass2.prep_mode)

Controls how source files are packed into Pass 2 LLM calls (set per profile in mykg_config.yaml):

• batch_chunks (default) — packs chunks across files into token-bounded batches, ignoring file boundaries; best throughput and extraction density.
• concat — merges whole small files into directory-grouped batches (one LLM call per window) for cross-file context.
• per_file — one file per extraction unit; cleanest provenance (every entity traces to one source file). Fully updates a modified file on --append — its old nodes are cleanly replaced on re-extraction. (batch_chunks with batch_per_file: true gives the same guarantee; mixed-batch modes may retain stale copies — see docs/architecture.md.)

See docs/architecture.md for the full comparison.

Hitting API Rate Limits (HTTP 429)

A 429 surfaces in the log as a retry warning like:

[WARNING] mykg.llm.retry — OpenAI 429 rate-limit (attempt 1/5) — retrying in 2.0s

429 is a "Too Many Requests" error. If you see repeated 429 errors during pass1, pass2, or the orphan-connection pass, your account's requests-per-minute limit is lower than the number of concurrent calls mykg is making. Each profile sets these independently under pipeline::

• pass1.max_workers — concurrent schema-induction batch calls
• pass2.max_workers — concurrent per-file extraction calls
• orphan_pass.max_workers — concurrent orphan-connection calls

Lower these (e.g. from 8 down to 2–4) in the active profile to reduce concurrent requests. This is especially likely on openrouter-free (free-tier models have very low per-minute caps), on gemini with a free-tier key (5 requests/minute/model — drop pass1/pass2/orphan_pass to 1–2), and on lower-tier anthropic-claude/openai accounts. llm.retry_429_max / llm.retry_429_base_delay control automatic backoff on a 429, but a persistent 429 is a signal to reduce max_workers, not just retry harder. claude-cli is unaffected — it is serial by design (max_workers: 1); it doesn't hit API rate limits since there's no API call. agent-claude-code is not API rate-limited either (no API key involved), but it is not serial — its default profile sets pass1/pass2/orphan_pass max_workers > 1 (configurable, like any other profile), since the skill dispatches multiple subagents per wave.

Also check your quota/credits. Some providers return 429 when your account has exhausted its token quota or spending balance, not only for request cadence. If lowering max_workers doesn't help and the 429s persist from the very first call, check that your account still has available tokens/credits (e.g. the OpenAI/Anthropic billing dashboard, or your OpenRouter balance). No max_workers value will clear a 429 caused by a depleted balance — top up or switch to a profile with quota (e.g. ollama-local for local inference, or claude-cli which bills via your Claude Pro/Max plan instead of the API).

Non-Reproducible Extraction (Different Graph Each Run)

If the same corpus produces a different schema or a different set of nodes and edges on each run, sampling temperature is the usual cause.

myKG sends temperature: 0.0 on every LLM call so extraction is repeatable — it is a structured-output task, not a creative one, and provider defaults (typically ~1.0) work against that. Two cases fall outside it, and in both the provider's own sampling applies:

• Models that reject an explicit temperature — OpenAI's o-series and gpt-5 reasoning families — have it omitted automatically and use their own fixed sampling.
• claude-cli and agent have no temperature control at all: claude -p exposes no such flag, and the host Claude Code session servicing the agent inbox has no sampling dial. These two profiles accept the setting and ignore it, so extraction on them is not reproducible in the sense above.

If you need reproducibility, switch to a profile whose provider exposes temperature (anthropic-claude, openai on a non-reasoning model, gemini, ollama-local, openrouter-free).

Changed in 0.4.5 — earlier versions sent no temperature at all and inherited each provider's default. If you were relying on that, extraction output will shift. To restore the old behaviour, set an explicit empty temperature: in the active profile's llm: block; a null value means "send nothing".

Extract Pipeline

Reads a directory of mixed format files and produces a typed knowledge graph in three output formats. The pipeline runs 12 sequential steps; all intermediate state is persisted so any step can be re-entered without repeating upstream work.

Running

mykg extract-graph <input_dir> [OPTIONS]
# source installs: uv run mykg extract-graph <input_dir> [OPTIONS]

<input_dir> is any directory containing your source files. Subdirectories are included recursively. Only files matching the configured extensions are copied into the session:

• .md — always included (the pipeline's native format)
• All extensions listed under preprocess.extensions in mykg_config.yaml (.pdf, .docx, .doc, .pptx, .xlsx, .png, .jpg, .jpeg, .html, .htm, .txt by default)

Everything else (.py, .json, .yaml, lock files, etc.) is ignored. Hidden directories (.venv, .git, etc.) and the sessions folder are also excluded automatically, so you can safely point extract-graph at the project root or any parent directory.

Options

Examples

# New run — auto-creates a timestamped session
mykg extract-graph my_notes/

# Resume a session with 4 parallel Pass 2 workers
mykg extract-graph my_notes/ --session 2026-05-17T18-31-07 --workers 4

# Run once against a different profile/model without editing mykg_config.yaml
mykg extract-graph my_notes/ --profile openrouter-free --model google/llama-4-maverick

# Pause for schema review after Pass 1
mykg extract-graph my_notes/ --review
# → edit mykg_sessions/<name>/intermediate/schema.json
mykg approve-schema --session 2026-05-17T18-31-07
mykg extract-graph my_notes/ --session 2026-05-17T18-31-07 --review

# Re-run from assembly onward (reuses existing extractions)
mykg extract-graph my_notes/ --session 2026-05-17T18-31-07 --from-step assemble

# Lock a base ontology so the LLM won't rename its classes
mykg extract-graph my_notes/ --base-schema ontology/core.ttl

# Induce and inspect the schema first, extract in a second invocation
mykg extract-graph my_notes/ --pass1-schema-induction-only
# → review/edit mykg_sessions/<name>/intermediate/schema.json
mykg extract-graph my_notes/ --session <name> --pass2-kg-extraction-only

# Mix models across phases: a strong model for schema induction,
# a smaller/cheaper one for the laborious Pass 2 extraction
mykg extract-graph my_notes/ --profile openrouter-free --model google/gemma-4-26b-a4b-it:free --pass1-schema-induction-only

Per-run --profile / --model for phase-specific model selection. Because --profile and --model override the run without touching mykg_config.yaml, you can pick a different model for each phase of a two-phase extraction. Schema induction (Pass 1) benefits from a strong reasoning model, while the laborious per-file Pass 2 extraction can run on a smaller, cheaper model — just change --profile/--model between the --pass1-schema-induction-only and --pass2-kg-extraction-only runs.

Sessions

Every run automatically creates an isolated session folder:

mykg_sessions/
  2026-05-17T18-31-07/
    input/           ← archived copy of all input Markdown files
    intermediate/    ← all intermediate pipeline state
    output/          ← final outputs (JSONL, TTL, HTML, NetworkX)
    run.log          ← log file
    walkthrough.md   ← post-run report

Sessions are the primary unit of resumability. Pass --session <name> to resume from the last completed step. Pass --from-step <step> to force-restart from a specific point.

The sessions root is configurable via pipeline.paths.sessions_dir (default: mykg_sessions/ in the current directory).

Pipeline Steps

The pipeline runs 12 steps in sequence. All intermediate state is written to disk so any step can be re-entered without repeating upstream work.

Pass 1 internally runs four sequential stages: parallel batch induction → algorithmic merge → harmonization LLM call → quality review LLM call. pass1.max_schema_proposals (default 50) caps the number of batches dispatched to the LLM; set to -1 to dispatch all batches on large corpora.

Outputs

Property Graph (JSONL)

nodes.jsonl — one JSON line per entity:

{
  "id": "person-alice",
  "type": "Person",
  "confidence": 0.94,
  "source_files": ["team.md"],
  "attributes": {
    "name":  {"value": "Alice",          "confidence": 1.0},
    "email": {"value": "alice@acme.com", "confidence": 0.88}
  },
  "aliases": ["Alice Smith", "A. Smith"]
}

edges.jsonl — one JSON line per relationship:

{
  "id": "works_at-abc123",
  "type": "works_at",
  "from": "person-alice",
  "to": "org-acme-corp",
  "confidence": 0.96,
  "method": "llm_extraction",
  "attributes": {
    "role":       {"value": "Engineer", "confidence": 0.91},
    "start_date": {"value": null,       "confidence": 0.0}
  }
}

Missing attributes are never dropped — they are represented as {"value": null, "confidence": 0.0}.

The method field distinguishes edges extracted by Pass 2 (llm_extraction) from edges inferred by the orphan pass (orphan_inferred).

RDF / OWL (Turtle)

knowledge_graph.ttl — pure RDFS/OWL triples, no edge metadata:

@prefix ex: <http://mykg.local/schema/> .
@prefix :   <http://mykg.local/data/> .

ex:Person  a rdfs:Class .
ex:works_at  rdfs:domain ex:Person ;  rdfs:range ex:Organization .

:person-alice  a ex:Person ;  rdfs:label "Alice" .
:person-alice  ex:works_at  :org-acme-corp .

Load in Protégé, query with SPARQL (Fuseki, GraphDB), or reason with HermiT/Pellet.

Interactive HTML

knowledge_graph.html — self-contained D3.js force-directed graph. Open in any browser, no server required. Supports:

• Filter nodes and edges by type
• Filter by confidence threshold
• Search by name
• Hover popups with full attribute values
• Resizable sidebar

NetworkX Formats (networkx_output/)

Node/edge attributes are exported as attr_<name>_value / attr_<name>_confidence scalar pairs for GML compatibility.

Obsidian Vault (obsidian_vault/)

One .md note per extracted entity, grouped into subdirectories by concept type. Each note has YAML frontmatter (id, type, confidence, sources), an attributes section, outgoing and incoming wikilink relationship sections, and a source files list. An index.md at the vault root summarizes node counts per type with links to every entity.

Open output/obsidian_vault/ as a vault in Obsidian to get Graph View, backlink navigation, and full-text search across the extracted entities.

Neo4j LOAD CSV Bundle (neo4j_csv/)

Optional, off by default. Enable with --neo4j-csv on the command line, or set pipeline.export.neo4j_csv_enabled: true in mykg_config.yaml.

When enabled, step_validate_graph writes a self-contained Neo4j import bundle next to the other outputs:

output/neo4j_csv/
  nodes_<Label>.csv             ← one per concept type (Person, Organization, …)
  relationships_<TYPE>.csv      ← one per property (WORKS_AT, KNOWS, …)
  import_browser.cypher         ← paste-and-run for Neo4j Browser
  import_shell.cypher           ← for `cypher-shell -f`
  README.md                     ← bundle-local quick-reference

Plain CSV headers (id,name,name_confidence,_node_confidence,_parents,_source_files,...) — no :ID / :LABEL decorations, so the same files work for both Neo4j and any other CSV-aware tool.

Two ways to import the bundle:

# Flow A — Neo4j Browser
# 1. Copy *.csv into your DBMS's import/ directory
# 2. Paste the contents of import_browser.cypher and press play

# Flow B — cypher-shell (set dbms.security.allow_csv_import_from_file_urls=true first)
cypher-shell -u neo4j -p <pw> -f output/neo4j_csv/import_shell.cypher

Both scripts use idempotent MERGE against a _MykgNode uniqueness constraint, so re-running updates the graph in place. Requires Neo4j 5+. No Python driver, no plugin, no APOC — the scripts use only core Cypher.

See Neo4j LOAD CSV Export below for configuration details and the standalone CLI fallback.

Re-running from a Specific Step

Use --from-step to delete a step's outputs and all downstream outputs, then re-run from that point.

SESSION=2026-05-17T18-31-07

# Re-run from Pass 2 (reuse the existing schema)
mykg extract-graph my_notes/ --session $SESSION --from-step pass2

# Re-run only assembly + export (reuse raw extractions)
mykg extract-graph my_notes/ --session $SESSION --from-step assemble

# Re-run both orphan stages
mykg extract-graph my_notes/ --session $SESSION --from-step orphan_score

# Orphan LLM pass only — full clean sweep
mykg extract-graph my_notes/ --session $SESSION --from-step orphan_connect_fullsweep

# Orphan LLM pass only — additive (preserves prior confirmed edges)
mykg extract-graph my_notes/ --session $SESSION --from-step orphan_connect_incremental

# Re-run only merge/harmonize/quality-review (reuse existing Pass 1 batch proposals)
mykg extract-graph my_notes/ --session $SESSION --from-step merge_proposals

Five re-entry patterns:

Orphan-Connection Pass

After assembly, nodes with zero edges are "orphans" — present in the graph but unreachable by traversal. The orphan pass reconnects them in two stages:

Stage 1 — orphan_score (no LLM): Uses chunk_node_index.json to find nodes that co-occur in the same source chunk as each orphan. Candidates are scored by co-occurrence frequency and filtered by schema type compatibility. Written to orphan_candidates.json.

Stage 2 — orphan_connect (LLM): One LLM call per source chunk. The prompt includes the full chunk text, all orphan IDs from that chunk, co-occurring connected nodes, and all schema properties. Confirmed edges carry "method": "orphan_inferred" and are merged directly into edge_metadata.json.

Unconnectable orphans (no resolvable source chunk) are logged as orphan_unconnectable advisory events in orphan_log.json.

Configure via pipeline.orphan_pass.* in mykg_config.yaml. Disable entirely with pipeline.orphan_pass.enabled: false.

Advanced Options

Human Review Gate (--review)

Pause after Pass 1 to inspect and edit the induced schema before Pass 2 runs:

mykg extract-graph my_notes/ --review
# → pipeline halts; edit mykg_sessions/<name>/intermediate/schema.json
mykg approve-schema --session <name>
mykg extract-graph my_notes/ --session <name> --review   # resumes from Pass 2

Locked Base Schema (--base-schema)

Lock certain classes and properties so the LLM cannot rename, remove, or restructure them:

mykg extract-graph my_notes/ --base-schema ontology/base.ttl

Both RDFS and OWL ontologies are accepted — owl:Class, owl:ObjectProperty, and owl:DatatypeProperty are recognized alongside their RDFS equivalents, so you can use an OWL ontology exported from Protégé directly without conversion. Locked entries can still receive additional attributes proposed by the LLM. Near-duplicate LLM proposals are collapsed into the locked entry with a warning.

Frozen Schema (--freeze-schema)

Use the base schema verbatim — skip Pass 1 LLM induction entirely. The LLM extracts instances against exactly the classes and properties you provide, with no additions:

mykg extract-graph my_notes/ --base-schema ontology/base.ttl --freeze-schema

This is useful when you have a complete ontology and want strict extraction — no LLM-invented types, no surprise properties. Only the concepts and relationships in your TTL will appear in the output. Requires --base-schema. Mutually exclusive with --append and --append-with-grow-schema.

Comparison with --base-schema alone:

SKOS Thesaurus (--thesaurus)

Resolve near-duplicate concept names during schema merge using a SKOS vocabulary:

mykg extract-graph my_notes/ --thesaurus ontology/terms.skos.ttl

• skos:exactMatch → silent collapse
• skos:closeMatch → collapse with warning in merge_log.json
• skos:broader / skos:narrower → advisory hints only

Website / Repo Fetching (mykg fetch-web)

Crawl a website, or shallow-clone a GitHub repo, into a folder that's a ready-made extract-graph input:

# Crawl a website (same-domain, robots.txt-respecting)
mykg fetch-web https://example.com
mykg extract-graph ./mykg_web_fetch/example.com/

# Shallow-clone a GitHub repo (git clone --depth 1, no Crawlee/venv)
mykg fetch-web https://github.com/SenolIsci/mykg
mykg extract-graph ./mykg_web_fetch/github.com_SenolIsci_mykg/input/

# Fetch multiple seeds (one URL per line; mix of sites and GitHub repos)
mykg fetch-web --url-list urls.txt --output ./mykg_web_fetch/batch/

• GitHub URLs (https://github.com/<owner>/<repo>) are detected automatically and routed to git clone — no Crawlee, no venv.
• Everything else is crawled with Crawlee inside an ephemeral uv venv (same pattern as the MinerU venv used by preprocess), respecting robots.txt, --max-pages, --max-depth, and a configurable request delay/concurrency.
• Resumable — fetch_manifest.json records a SHA-256 per page; re-running skips unchanged pages. --force re-fetches everything.
• Output dir defaults to ./<fetch.output_dir>/<seed-domain>/ (configurable via fetch.output_dir in mykg_config.yaml, default mykg_web_fetch); --output overrides it and is required with --url-list.
• Downloaded assets (PDFs, images, etc.) are filtered through the same preprocess.extensions allowlist used by extract-graph, so anything Crawlee saves is something preprocess already knows how to convert.

All knobs live under fetch: in mykg_config.yaml — see docs/architecture.md for the full crawl/clone sequence diagrams. Run mykg fetch-web --help for the complete flag list.

From Claude Code, the /mykg skill handles fetch requests in plain English — no flags to remember:

/mykg fetch https://example.com and extract
/mykg download the repo: https://github.com/SenolIsci/mykg
/mykg fetch these urls: <url1> <url2> ... and extract

All work: the skill picks the right fetch-web invocation (single page, GitHub clone, or --url-list batch with an auto-generated temp file for inline URLs), runs it, and — for the "and extract" intents — chains straight into extract-graph on the fetched output (one fresh session per seed for multi-seed fetches), confirming with you before the LLM-bearing extraction step.

Standalone Document Conversion (mykg parse-docs)

extract-graph already converts non-Markdown inputs (PDF, DOCX, images, …) to Markdown automatically via the preprocess step — on both the initial run and on --append. You only need parse-docs when you want to convert documents on their own, without running a pipeline or creating a session: inspecting MinerU output, building a Markdown corpus to commit, or feeding another tool.

# Convert a single file
mykg parse-docs --input report.pdf --output ./md/

# Convert every non-.md file under a directory (recursive; structure preserved)
mykg parse-docs --input raw_docs/ --output ./md/

# Convert only specific files (relative to --input; repeatable)
mykg parse-docs --input raw_docs/ --output ./md/ --file a.pdf --file sub/b.docx

• MinerU in an ephemeral venv — conversion runs MinerU inside a throwaway uv-managed venv that is built per invocation and deleted on exit; nothing is installed into mykg's own interpreter. The multi-GB install is paid once per call and reused across every file in that call.
• Extension allowlist — candidate files are filtered through preprocess.extensions from mykg_config.yaml (the same allowlist extract-graph uses). .html/.htm are always hard-skipped (MinerU cannot convert HTML — use extract-graph, which routes HTML through markdownify). Pass --no-filter to send every non-.md file to MinerU regardless of suffix.
• Large corpora — use --file-list <path> (one rel-path per line) instead of repeated --file flags to avoid the OS argv-size limit. --file and --file-list are mutually exclusive.
• No session — parse-docs is a pure file-to-file utility: it does not create a session or touch mykg_sessions/. Per-file failures are logged and the run continues, exiting non-zero at the end if any file failed.

Run mykg parse-docs --help for the complete flag list. From Claude Code, /mykg convert pdfs in ./inbox to ./md maps to the right invocation automatically.

Append Mode

Re-run the pipeline on new or modified files without re-running Pass 1:

mykg extract-graph my_notes/ --session <name> --append

The input directory may contain PDF, DOCX, HTML, TXT, and image files alongside .md — newly-added non-Markdown files are converted automatically during the append run (the same incremental preprocess step the initial run uses, subject to preprocess.enabled). No separate mykg parse-docs step is needed. Only new or changed source files are converted; unchanged ones are skipped by content hash.

Reconciling Edits and Deletions (--sync)

--append adds; --sync reconciles. Plain --append extracts new files only — it detects modified and deleted files and warns about them, but leaves the graph untouched. Add --sync to act on them:

mykg extract-graph my_notes/ --session <name> --append --sync

All four combinations, and what each does with the three kinds of change:

The three flags are orthogonal axes: --append decides which files are processed, --sync decides whether existing entries are reconciled, and -with-grow-schema decides whether the schema may grow.

--sync requires --append. --append-with-grow-schema already implies --append, so the last row needs only the two flags shown. --update is shorthand for --append --sync:

mykg extract-graph my_notes/ --session <name> --update

Reach for the fourth row when the new vocabulary lives in a document you edited rather than one you added — without --sync, the locked Pass 1 only sees new files, so a concept introduced by an edit never reaches the LLM.

⚠️ Behaviour change: before the release 0.4.3, plain --append re-extracted modified files. It no longer does — add --sync. If your edits stop showing up in the graph, that is why; the run will have logged a warning naming the files.

Deleting a file removes the nodes and edges that came only from it; nodes shared with a surviving file are recomputed, so their confidence and attribute values no longer reflect the removed document. Running --sync with nothing to reconcile costs nothing — it degrades to exactly plain --append, so it is safe to leave in a script.

Exact deletion needs pass2.prep_mode: per_file or batch_chunks with pass2.batch_per_file: true. On the shipped default one LLM call covers several files, so a co-batched sibling may retain some of the deleted file's nodes.

Multiple source folders. Each folder you append to a session is registered with its own subtree of session/input/, so two fetch-web crawls that both produce index.md no longer overwrite each other. --sync is scoped to the folder you name — reconciling one folder can never delete another's files, which is why no confirmation prompt or size cap is needed.

Incremental Schema Growth (--append-with-grow-schema)

Plain --append freezes the schema — Pass 1 is skipped, so new entity types and relationships are never induced. Use --append-with-grow-schema when you add documents that introduce concepts the current schema doesn't cover:

mykg extract-graph my_notes/ --session <name> --append-with-grow-schema

This runs a locked Pass 1 over only the changed files: the LLM may add new concepts and properties but cannot rename, remove, or restructure existing ones. When the schema grows, a surgical back-fill re-extracts the old chunks most likely to contain instances of the new types (configurable via append.grow_schema_backfill_top_k_chunks_per_type, default 10; set 0 to disable). When the new documents don't introduce new types, the run collapses to a plain --append at no extra cost — and with nothing to reconcile at all, the locked Pass 1 is skipped entirely (zero LLM calls). Note that "changed files" means new files unless you also pass --sync.

The flag implies --append (no need to pass both) and is mutually exclusive with --from-step and --base-schema (the session's existing schema.ttl is auto-loaded as the locked base).

Mixed-format inputs: Both --append and --append-with-grow-schema automatically preprocess newly-added non-Markdown files (PDF, DOCX, HTML, TXT, images). Just drop the new files into the input directory and append — the preprocess step runs incrementally, converting only the new or changed sources (unchanged files are skipped by content hash) and feeding the converted Markdown straight into extraction. No separate mykg parse-docs step is required.

Merging Sessions

Combine two independently-produced sessions into a unified knowledge graph:

mykg merge-graphs <session-A> <session-B> [OPTIONS]

# Example
mykg merge-graphs 2026-05-01T10-00-00 2026-05-15T14-30-00

# Resume a merge (last incomplete step auto-detected)
mykg merge-graphs A B --output-session <merged-name>

Options:

What happens:

1. Both schemas are merged via the same three-stage chain as Pass 1 (algorithmic union → LLM harmonization → LLM quality review)
2. All file-keyed structures are namespaced (session_a/<filename>, session_b/<filename>) before merging
3. Nodes are deduplicated across sessions: same type + canonical name → single node, regardless of source session
4. Re-extraction strategy (none / surgical / full) handles properties absent from one session's schema
5. source_map.json records full file provenance; merge_manifest.json records schema deltas and strategy used
6. walkthrough.md includes a Merge Provenance section with before/after counts and node/edge breakdowns

Configure the re-extraction strategy:

merge_graphs:
  reextraction_strategy: surgical   # none | surgical | full

Obsidian Vault Export

Every run writes a linked Markdown vault to output/obsidian_vault/ by default. Open that folder in Obsidian to explore the extracted knowledge graph with Graph View and backlinks.

Vault structure:

output/obsidian_vault/
  index.md                  ← overview: node count per type, links to every entity
  Person/
    person-alice-smith.md   ← one note per entity
    person-bob-jones.md
  Organization/
    organization-acme-corp.md
  ...

Each entity note contains:

---
id: person-alice-smith
type: Person
confidence: 0.94
sources:
  - team.md
---

# Alice Smith

## Attributes
- **role**: Engineer (0.91)
- **email**: alice@acme.com (1.0)

## Relationships

### Outgoing
- [[Acme Corp]] — works_at (0.96)

### Incoming
- [[Bob Jones]] — manages (0.88)

## Source Files
- team.md

Wikilinks ([[...]]) are Obsidian-native — clicking them in the app navigates to the linked entity note, and the Graph View shows the full relationship network automatically.

Config:

pipeline:
  export:
    obsidian_enabled: true          # default — set false to skip vault export
    obsidian_vault_dir: obsidian_vault   # subfolder name inside output/

Or use --obsidian-vault on the command line for a one-off run without editing config.

Neo4j LOAD CSV Export

Optional bundle for one-step import into Neo4j 5+. Off by default. When enabled, every run writes the bundle to output/neo4j_csv/ alongside the other outputs.

Bundle contents (see neo4j_csv/ above for the full layout):

• One nodes_<Label>.csv per concept type with plain headers (id,name,name_confidence,...)
• One relationships_<TYPE>.csv per property (rel-type names sanitized to upper snake_case)
• import_browser.cypher — paste-and-run for Neo4j Browser (relative file:/<name>.csv URIs)
• import_shell.cypher — for cypher-shell -f (absolute file:/// URIs)
• README.md — bundle-local quick-reference with paste instructions

The scripts use:

1. A uniqueness constraint on (_MykgNode {id}) — created on first run, IF NOT EXISTS thereafter
2. MERGE for every node and edge — idempotent, safe to re-run
3. IN TRANSACTIONS OF 1000 ROWS — handles large bundles without OOM
4. Per-label domain labels (:Person, :Organization) plus the shared :_MykgNode label that carries the constraint

Config:

pipeline:
  export:
    neo4j_csv_enabled: false        # default — set true to enable
    neo4j_csv_dir: neo4j_csv        # subfolder name inside output/

Or use --neo4j-csv on the command line for a one-off run without editing config.

Walkthrough Report

A human-readable summary is written to mykg_sessions/<name>/walkthrough.md after every run:

# Regenerate the walkthrough for an existing session
mykg walkthrough --session 2026-05-17T18-31-07

Disable with pipeline.report.enabled: false.

MCP (Model Context Protocol) Server

Serve any completed session as an MCP server for LLM-powered Q&A:

# Serve the latest session (stdio transport — for Claude Desktop)
mykg mcp-serve

# Serve a specific session
mykg mcp-serve --session 2026-06-25T19-16-18

# Serve via streamable HTTP for web clients (Cherry Studio, etc.)
mykg mcp-serve --transport streamable_http --port 3100

# Serve a specific session via streamable HTTP
mykg mcp-serve --session 2026-06-25T19-16-18 --transport streamable_http --port 3100

Claude Desktop (stdio) — the client launches mykg as a subprocess, no manual server start needed. Add to claude_desktop_config.json. Replace /path/to/your/project with the absolute path to your project folder:

{
  "mcpServers": {
    "mykg": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/your/project",
        "run",
        "mykg",
        "mcp-serve"
      ]
    }
  }
}

To serve a specific session:

{
  "mcpServers": {
    "mykg": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/your/project",
        "run",
        "mykg",
        "mcp-serve",
        "--session",
        "2026-06-25T19-16-18"
      ]
    }
  }
}

Claude Code (stdio) — add a .mcp.json file to your project root so Claude Code launches the MCP server automatically at session start:

{
  "mcpServers": {
    "mykg": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/your/project",
        "run",
        "mykg",
        "mcp-serve"
      ]
    }
  }
}

To serve a specific session:

{
  "mcpServers": {
    "mykg": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/your/project",
        "run",
        "mykg",
        "mcp-serve",
        "--session",
        "2026-06-25T19-16-18"
      ]
    }
  }
}

Restart required. Claude Code reads .mcp.json at session start. After creating or editing this file, restart your Claude Code session for the server to connect. You will be prompted to approve the mykg MCP server on first use.

After a pipeline run (extract-graph, --append, etc.), the in-memory graph becomes stale. Call the mykg_reload tool to refresh it without restarting — or restart the Claude Code session to reload from scratch.

If you also have the /mykg skill installed, the skill will prefer MCP tools for read-only queries when the server is connected, and fall back to direct file reads when it is not.

Streamable HTTP (Cherry Studio, MCP Inspector, or any HTTP-based MCP client) — start the server first with mykg mcp-serve --transport streamable_http --port 3100, then connect your client:

{
  "mcpServers": {
    "mykg": {
      "type": "streamableHttp",
      "url": "http://localhost:3100/mcp"
    }
  }
}

The MCP server exposes 14 tools: mykg_search_nodes, mykg_get_node, mykg_get_neighbors, mykg_find_path, mykg_get_schema, mykg_list_node_types, mykg_query_subgraph, mykg_get_stats, mykg_query_graph (BFS/DFS traversal), mykg_hub_nodes, mykg_orphan_nodes, mykg_read_note (Obsidian vault LLM wiki notes), mykg_list_sessions, and mykg_reload (refresh the in-memory graph after pipeline runs).

Configuration — default transport, host, and port are set per profile in mykg_config.yaml:

    mcp:
      host: localhost              # bind address for streamable HTTP (ignored for stdio)
      port: 3100                   # port for streamable HTTP (ignored for stdio)
      transport: stdio             # default transport: stdio | streamable_http

To default to streamable HTTP, change transport: streamable_http in your active profile. CLI flags (--transport, --host, --port) override the config.

Terminal query (mykg query)

mykg query "<question>" [--session NAME] [--mode bfs|dfs] [--depth N] [--token-budget N]

Query the knowledge graph directly from the shell — no MCP server required. mykg query finds up to 3 seed nodes matching your question by name, alias, or attribute value, traverses outward through the graph to a bounded depth, and prints a text context window of the visited nodes and the relationships between them. It is the terminal-callable equivalent of the MCP mykg_query_graph tool, and it is read-only: it operates on the latest completed session unless --session is passed.

• question (positional, required) — the free-text query.
• --session NAME — session under mykg_sessions/; defaults to the latest completed session (newest dir with output/nodes.jsonl).
• --mode bfs|dfs — traversal strategy (default bfs).
• --depth N — traversal depth limit (default 2).
• --token-budget N — approximate token budget bounding the returned context (default 2000).

mykg query "who is Alice" --depth 2

# Knowledge Graph Context: who is Alice
Seeds: <ids> | Mode: bfs | Depth: 2
Nodes visited: N | Edges found: M

## Nodes
- [Type] Name (node-id) conf=0.95 (attr=val, ...)

## Relationships
- from-id --[edge_type]--> to-id (conf=0.90)

If nothing matches, it prints No nodes found matching '<question>'. Try mykg_search_nodes for more flexible search. and exits 0.

Using mykg with Claude Code

myKG ships with two complementary integrations for running extractions from inside Claude Code:

• Agent mode (agent-claude-code profile + bundled skill) — the pipeline writes LLM tasks to a session-local inbox folder and a Claude Code skill dispatches subagents to answer them. Parallel by default.
• claude-cli profile — the pipeline shells out to the claude -p binary for each LLM step. Serial only.

Pick the claude-cli for a drop-in claude-as-LLM experience; pick agent-claude-code when you want agent skill and parallel subagent dispatch.

Agent mode (Claude Code skill)

Agent mode is a different way to run myKG inside Claude Code: instead of claude -p subprocesses, the pipeline writes LLM tasks to a session-local inbox folder and a Claude Code skill dispatches subagents to answer them. Pick agent mode over claude-cli when you want parallel subagent dispatch from inside an active Claude Code session.

Why pick agent mode

• No API key needed. Uses your existing Claude Pro/Max plan via the skill subagents — same as claude-cli, but without invoking the claude -p binary.
• Inspectable LLM I/O. Every prompt lands as intermediate/agent_inbox/<id>.task.json and every answer as intermediate/agent_outbox/<id>.answer.json. Replay or edit any step by hand.
• Parallel by default. The skill dispatches up to pass2.max_workers subagents per wave in a single message — not serial like claude-cli. Pass-2 chunks complete in parallel waves.

Install and configure

pip install mykg          # or: uv tool install mykg
mykg init --profile agent-claude-code
# ...then restart Claude Code so the skill loader picks up the new entry.

mykg init --profile agent-claude-code writes mykg_config.yaml, copies the bundled skill into ~/.claude/skills/mykg (honoring $CLAUDE_CONFIG_DIR if set), and adds a managed <!-- BEGIN mykg-section --> ... <!-- END mykg-section --> block to the project's CLAUDE.md. A .mykg_skill_version stamp file is written next to the skill so future runs can detect drift; the CLAUDE.md block tells Claude Code where the wiki lives, how to find the most-recent session, and how to extend the graph with new documents (no separate setup required).

Upgrade after pip install -U mykg:

mykg init --reinstall-skill --reinstall-claude-md

This atomically refreshes the bundled skill (copy to .tmp → os.replace) without touching your mykg_config.yaml, and replaces the content between the CLAUDE.md markers with the version shipped in the current package — any user content outside the markers is preserved. Either flag can be used alone (--reinstall-skill only / --reinstall-claude-md only). The copy-based skill install is deliberately not a symlink — symlinks fail on Windows without Developer Mode, dangle if mykg is uninstalled, and don't sync through OneDrive. The cost (live edits don't auto-propagate) only matters for mykg developers, who pass --reinstall-skill between edits.

The agent: block in the generated mykg_config.yaml configures the inbox/outbox paths and poll interval:

profile: agent-claude-code

profiles:
  agent-claude-code:
    provider: agent
    agent:
      inbox_dir: agent_inbox        # relative to <session>/intermediate/
      outbox_dir: agent_outbox
      poll_interval_seconds: 2
    pipeline:
      pass2:
        max_workers: 8              # how many subagents the skill dispatches per wave

Invoke from inside Claude Code

The skill exposes one slash command — /mykg — that accepts free-form intent. You describe what you want; the skill figures out which mykg CLI command to run, reads the live --help to validate flags, confirms expensive actions, and (for extract-graph) drains the LLM inbox in parallel waves.

Examples:

Any flag mykg accepts on the CLI works here too — the skill reads --help rather than maintaining its own list, so --from-step orphan_connect, --workers 8, --obsidian-vault, etc. all flow through.

mykg init and mykg merge-graphs are intentionally not wrapped: init is interactive (run from a shell once per machine), and merge-graphs has additional design questions and will be added in a follow-up.

Full design and contract: docs/agent-mode.md. Skill source: src/mykg/data/skills/mykg/SKILL.md.

claude-cli profile

myKG ships with a claude-cli profile that runs extractions through the locally-installed claude CLI.

Setup

Install the claude CLI, then install mykg and run the setup wizard — select [5] Claude CLI when prompted.

npm install -g @anthropic-ai/claude-code
pip install mykg && mykg init
mykg extract-graph my_notes/

How it works

The claude-cli provider calls claude -p as a subprocess for every LLM step (Pass 1 schema induction, Pass 2 extraction, orphan connection, name normalization). All pipeline features — session isolation, resumability, orphan recovery, cross-session merge — work identically to API-based providers.

Key constraints of the claude-cli profile:

• max_workers must be 1 — the claude CLI is serial by design; parallel workers will queue
• The effort and model fields in mykg_config.yaml map directly to --effort and --model flags passed to claude -p

Using myKG from inside Claude Code Session

You can run myKG extractions as a tool call from within a Claude Code session. This is useful for building knowledge graphs from notes or documentation while you work:

# From any Claude Code session terminal:
mykg extract-graph ./docs/ --session my-docs-kg

# Then reference the output in your session:
# mykg_sessions/my-docs-kg/output/nodes.jsonl
# mykg_sessions/my-docs-kg/output/knowledge_graph.ttl

Claude Code can then read nodes.jsonl or edges.jsonl as well as the Obsidian vault directly to answer questions about the extracted graph, or load knowledge_graph.ttl into a SPARQL tool for structured queries.

Roadmap

• in preparation.

Development

Installation

git clone https://github.com/SenolIsci/mykg && cd mykg
uv sync

Testing

# All non-live tests (fast, no API key needed)
uv run pytest -m "not live" -v

# All tests including live API integration tests
# Requires OPENROUTER_API_KEY in environment or .env.mykg (see sample.env.mykg)
uv run pytest -m live -v

# Single file
uv run pytest tests/test_assembler.py -v

# With coverage (HTML report at htmlcov/index.html)
uv run pytest -m "not live"
open htmlcov/index.html

Healthiness check

A dry-run smoke test that answers one question: is everything alive and right? It probes every configured LLM endpoint, then exercises the major capabilities — fetch-web, parse-docs (MinerU), extract-graph, query, walkthrough — end to end through the real CLI, and prints a summary table.

# Just the endpoints — seconds
uv run pytest tests/test_healthiness.py -k endpoints -v --no-cov

# Everything except MinerU — the usual quick check
uv run pytest tests/test_healthiness.py -m "live and not mineru" -v --no-cov

# Full check including PDF/XLSX conversion via MinerU
uv run pytest tests/test_healthiness.py -m live -v --no-cov

# Point the capability stages at a specific provider
MYKG_E2E_PROFILE=gemini uv run pytest tests/test_healthiness.py -m live -v --no-cov

Each endpoint is checked twice: that it is reachable, and that it can return the JSON contract Pass 2 depends on — a provider can answer a ping and still fail every real extraction. Providers with no key configured are skipped, never failed, and failures are classified into an actionable cause (quota or balance exhausted, api key rejected, model not available, …) rather than a bare traceback.

  ENDPOINTS         reach   contract
  openrouter-free   ok      ok        2 node(s), 1 edge(s)
  anthropic-claude  ok      ok        2 node(s), 1 edge(s)
  gemini            FAIL    —         quota or balance exhausted → top up, or switch profile
  ollama-local      ok      ok        2 node(s), 1 edge(s)

Artifacts land in a pytest temp directory — the crawled HTML, the MinerU-converted Markdown, and a full session under mykg_sessions/ — so a health check never touches your real sessions. The gitignored _healthiness_run symlink in the repo root points at the most recent run.

Two things to expect. A failing endpoint is the slow one: the cause is only known after the adapter exhausts its 429 backoff ladder, which is what lets the report tell "rate limited, recovered" apart from "quota exhausted" — lower llm.retry_429_max for a faster verdict. And on free tiers (Gemini allows 5 requests/minute/model, OpenRouter's free models are similarly capped) the extraction stages log 429 backoff warnings; the adapters retry and recover, which is documented behaviour, not a failure.

The whole file carries the live marker, so it is excluded from CI and from pytest -m "not live". Design notes and measured results are in docs/healthiness-check.md.

Linting and Formatting

uv run ruff check src/ tests/          # lint
uv run ruff check --fix src/ tests/    # auto-fix
uv run ruff format src/ tests/         # format

Token Budget Calculator

When switching to a model with a different context window:

context-calculator --context 128000 --max-output 16384

Outputs a ready-to-paste YAML snippet for the pipeline: block.

Profiling

python -m cProfile -o profile.out -m mykg.cli extract input_files/
uv run snakeviz profile.out

Design

Storage & runtime model — myKG uses JSONL files (nodes.jsonl, edges.jsonl) as its persistent graph storage and builds an in-memory NetworkX DiGraph for all graph operations (traversal, search, path-finding, hub/orphan detection, MCP server queries). There is no external database dependency — the graph loads from flat files at startup and all computation happens in-process. This keeps the tool self-contained and portable: a session folder is everything you need.

For a thorough description of the architecture, algorithm, data models, and design decisions, see docs/architecture.md.

License

MIT — see LICENSE.