hyperextract 0.8.0

Smart Knowledge Extraction CLI

Transform documents into structured knowledge with one command.

📖 English Version · 中文版

"Stop reading. Start understanding."
"告别文档焦虑，让信息一目了然"

📰 What's New

v0.8.0

• 🔄 Document Upsert — re-feed an attributed document and its previous version is rolled back automatically: removed facts disappear, shared keys re-merge from surviving sources. (#84)
• 📁 Per-File Source Attribution — he parse ./docs/ attributes each file by its name automatically; roll back or audit any single file later. An explicit --source still overrides.
• ⏱️ Spatiotemporal Provenance — temporal/spatial/spatio-temporal graphs fully support source attribution and rollback, with deterministic (MERGE_FIELD) replay tests.

v0.7.0

• 🗑️ Two-Tier Knowledge Deletion — he remove --node/--edge hard-deletes by key; he remove --edit-node --fact removes a single wrong fact via LLM rewrite (key-invariance check, dry-run, automatic backups). (#84)
• 📜 Source Attribution & Provenance — he feed --source / he parse --source record each document's raw contributions; he remove --document rolls back exactly what one document contributed, re-merging keys shared with surviving sources. (#84)
• 📈 Incremental Everything — feed/parse/removal/edit now patch the vector index in place (only affected vectors re-embedded); he feed skips documents whose content hash is unchanged (--refeed to force). (#84)
• 👁️ he info --sources — see which documents contributed to a knowledge abstract and how much. (#84)
• 🧪 he template validate — catch semantic template errors before paying for LLM calls: 9 diagnostic rules, --json for CI. (#77)
• 📊 GraphML & CSV Export + 🌐 OrcaRouter provider + 🔐 config.toml saved 0600 + 🔗 Obsidian wikilink fix. (#85, #71, #86, #87)

v0.6.0

• 🗑️ Knowledge Deletion (two-tier) — he remove / ka.remove_nodes() hard-delete by key (orphan edges pruned); ka.edit_node() removes a single wrong fact via LLM rewrite, with dry-run, key-invariance checks, and automatic backups. Graph, hypergraph, and temporal/spatial KAs. (#84)
• 🧪 he template validate — catch semantic template errors (missing time_field, unknown identifier fields, bad placeholders…) before calling an LLM: 9 diagnostic rules, --json for CI, --all for directories. (#77)
• 📊 GraphML & CSV Export — he export graphml for desktop graph tools, he export csv for spreadsheets (hypergraphs get a hyperedges table). (#85)
• 🔐 Config File Permissions — ~/.he/config.toml is now saved 0600 (owner-only) instead of world-readable. (#86)
• 🔗 Obsidian Link Fix — wikilink aliases no longer break on titles containing [ ] | # ^. (#87)

v0.5.0

• 🛡️ Chunk-Level Fault Isolation — One failed chunk (rate limit, timeout, unparseable output) no longer discards the rest of a multi-chunk extraction. Graph-family extraction degrades to a partial result + warning. (#78)
• ⚡ MCP Python SDK 2.x Support — he-mcp now works with both mcp 1.x and 2.x; hyperextract[mcp] resolves mcp<3 (protocol 2026-07-28). (#72, #82)
• 🔀 Directed-Edge Fix — relation_members now preserves (source, target) direction instead of sorting endpoints, so A→B and B→A stay distinct edges. Custom endpoint field names are supported. (#74)
• 🔑 DeepSeek API Key Fix — create_llm("deepseek") now reads DEEPSEEK_API_KEY on the OpenAI-compatible path. (#76)
• 🎓 Education Templates — Two new presets: education/course_concept_graph and education/curriculum_structure. (#80)
• 🌐 OrcaRouter Provider — One key for 150+ models (OpenAI, Anthropic, Google, DeepSeek, Qwen…) via create_client("orcarouter"). (#71)
• 🧭 Smaller Fixes — Graph_RAG.search returns a consistent 3-tuple in default mode; he talk -i honors --top-k; clearer docs on onboarding, dev install, and structured output. (#70, #73, #57)

Earlier

• 🤖 DeepSeek Provider — Use deepseek-v4-flash / deepseek-v4-pro directly via create_client(llm="deepseek"). V4 thinking mode is auto-disabled for structured extraction compatibility. (#67, v0.4.0)
• 🔌 MCP Server — Query your knowledge abstracts from Claude Desktop and IDE agents with he-mcp. (PR #40)
• 🧠 Anthropic Claude Support — Use claude-opus-4-8, claude-sonnet-4-6, and claude-haiku-4-5 directly as your LLM provider. (PR #38)
• 📝 Obsidian Export — Turn any graph into an Obsidian vault with Markdown notes linked by [[wikilinks]]. (PR #37)
• 🧹 he clean — Remove a KA's index or the whole knowledge abstract in one command. (PR #39)
• 🔧 Reliability Fixes — True mean for multi-chunk embeddings, capped OpenAI-compatible batch sizes, and resolved multi-word llm_* merge strategies. (PRs #35, #36, #41)

See the full changelog in the GitHub releases.

Hyper-Extract is an intelligent, LLM-powered knowledge extraction and evolution framework. It radically simplifies transforming highly unstructured texts into persistent, predictable, and strongly-typed Knowledge Abstracts. It effortlessly extracts information into a wide spectrum of formats—ranging from simple Collections (Lists/Sets) and Pydantic Models, to complex Knowledge Graphs, Hypergraphs, and even Spatio-Temporal Graphs.

✨ Core Features

🎯 What Can You Do With It?

he parse paper.pdf -t general/academic_graph -o ./paper_kb/
he show ./paper_kb/

he parse earnings.md -t finance/earnings_graph -o ./finance_kb/
he search ./finance_kb/ "What are the key risk factors?"

from hyperextract import create_client
llm, emb = create_client(
    llm="vllm:Qwen3.5-9B@http://localhost:8000/v1",
    embedder="vllm:bge-m3@http://localhost:8001/v1",
    api_key="dummy",
)

🚀 Supported Platforms & Models

Hyper-Extract uses LangChain structured output with function calling. The model must support tool/function calling.

Embedding models (semantic search) work with any OpenAI-compatible endpoint: text-embedding-3-small, text-embedding-v4 (Bailian), bge-m3 (local vLLM).

DeepSeek note: DeepSeek V4 models default to "thinking" mode, which Hyper-Extract auto-disables so structured extraction works. Set DEEPSEEK_API_KEY. DeepSeek has no embeddings API — pair it with an OpenAI-compatible embedder:

from hyperextract import create_client
llm, emb = create_client(llm="deepseek", embedder="openai:text-embedding-3-small")

Anthropic note: Claude is used for the LLM (set ANTHROPIC_API_KEY). Anthropic has no embeddings API, so pair it with an OpenAI-compatible embedder:

from hyperextract import create_client
llm, emb = create_client(llm="anthropic", embedder="openai:text-embedding-3-small")

Requires the extra: pip install 'hyperextract[anthropic]'.

📖 Full guide: Provider System & Local Model Support

⚡ 30-Second Quick Start

1. Install:

# Install uv first (if you haven't)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Install Hyper-Extract CLI
uv tool install hyperextract
# or: pipx install hyperextract

2. Configure your provider (pick one):

OpenAI:

he config init -p openai -k YOUR_OPENAI_API_KEY

Anthropic (Claude):

he config llm -p anthropic -k YOUR_ANTHROPIC_API_KEY
he config embedder -p openai -k YOUR_OPENAI_API_KEY

DeepSeek:

he config llm -p deepseek -k YOUR_DEEPSEEK_API_KEY
he config embedder -p openai -k YOUR_OPENAI_API_KEY

Bailian (Alibaba Cloud):

he config init -p bailian -k YOUR_BAILIAN_API_KEY

Local vLLM:

he config llm -p vllm -u http://localhost:8000/v1 -k dummy -m Qwen/Qwen3.5-9B
he config embedder -p vllm -u http://localhost:8001/v1 -k dummy -m BAAI/bge-m3

3. Extract, query & visualize:

# Extract knowledge from a document
he parse examples/en/tesla.md -t general/biography_graph -o ./output/ -l en

# Query it
he search ./output/ "What are Tesla's major achievements?"

# Visualize
he show ./output/

# Export to an Obsidian vault (Markdown notes + [[wikilinks]])
he export obsidian ./output/ -o ./vault/

Which provider should I use? OpenAI and Bailian provide both LLM and embedding models in one API. Anthropic and DeepSeek are LLM-only (pair them with an OpenAI embedder for search/chat). Local vLLM is free but requires a GPU. DeepSeek is the most cost-effective option (~$0.001-0.005/page vs ~$0.01-0.05/page for OpenAI gpt-4o-mini).

uv pip install hyperextract

from hyperextract import Template

ka = Template.create("general/biography_graph")

with open("examples/en/tesla.md") as f:
    result = ka.parse(f.read())

result.show()

📈 Why Hyper-Extract?

🧩 Supported Knowledge Structures

From simple to complex — pick the right structure for your data:

Example — AutoGraph visualization:

📋 What's under the hood? (Architecture & Templates)

Hyper-Extract follows a three-layer architecture:

• Auto-Types — 8 strongly-typed data structures (Model, List, Set, Graph, Hypergraph, Temporal Graph, Spatial Graph, Spatio-Temporal Graph)
• Methods — Extraction algorithms: KG-Gen, GraphRAG, LightRAG, Hyper-RAG, Cog-RAG, and more
• Templates — 80+ presets across 6 domains. Zero-code setup.

Template example (Graph type):

language: en
name: Knowledge Graph
type: graph
tags: [general]
description: 'Extract entities and their relationships.'
output:
  entities:
    fields:
    - name: name
      type: str
    - name: type
      type: str
    - name: description
      type: str
  relations:
    fields:
    - name: source
      type: str
    - name: target
      type: str
    - name: type
      type: str
identifiers:
  entity_id: name
  relation_id: '{source}|{type}|{target}'

• Browse all 80+ templates
• Create custom templates

📚 Documentation & Resources

🔌 MCP Server

Expose your knowledge abstracts to MCP-capable assistants (Claude Desktop, IDE agents) via the Model Context Protocol — read + export only.

pip install 'hyperextract[mcp]'
he-mcp        # stdio MCP server

Tools: list_templates, info, search, ask (RAG), export_obsidian. Full guide: MCP Server docs.

🤝 Contributing & License

Contributions are welcome! Please submit Issues and PRs.
Licensed under Apache-2.0.

🔒 Security

This project has been security assessed by MseeP.ai.

AtomGit Mirror

AtomGit mirror - a synchronized AtomGit mirror of Agent Reach for easier access and cloning in China. Hosted on AtomGit: https://atomgit.com/yifanfeng97/Hyper-Extract