OntoGraphRAG

Turn unstructured documents into schema-consistent knowledge graphs, ask grounded questions over them, and measure what to trust.

OntoGraphRAG is an ontology-guided KG-RAG system. It builds Neo4j-backed knowledge graphs from raw text, retrieves over both graph structure and chunk vectors, and exposes answer-grounding, provenance, and uncertainty signals. Unlike free-form GraphRAG, extraction is constrained to a schema you supply, so the same concept lands in the same type across every document.

Quick start

Requires Python 3.11+, Neo4j 5.0+, and one LLM provider key (or local Ollama). Node.js 18+ is only needed for source checkouts that rebuild the frontend.

# 1. Install
python -m pip install "ontographrag @ git+https://github.com/julka01/OntoGraphRAG.git"

# 2. Start Neo4j
docker run -d --name ontographrag-neo4j \
  -p 7474:7474 -p 7687:7687 -e NEO4J_AUTH=neo4j/password neo4j:5

# 3. Configure (copy .env.example to .env and fill in keys)
export NEO4J_URI=bolt://localhost:7687 NEO4J_USERNAME=neo4j NEO4J_PASSWORD=password
export OPENROUTER_API_KEY=...   # or OPENAI_API_KEY, GEMINI_API_KEY, ...

# 4. Check readiness and launch the web app
ontograph doctor
ontograph serve                  # → http://localhost:8004

Source checkout (development, frontend changes, or benchmarks):

git clone https://github.com/julka01/OntoGraphRAG.git && cd OntoGraphRAG
uv sync && source .venv/bin/activate
cd frontend && npm install && npm run build && cd ..
docker compose up -d neo4j
python -m ontographrag.cli serve

CLI

ingest, ask, and explore are thin wrappers around the server endpoints; use --server and --api-key (or ONTOGRAPHRAG_API_KEY) for remote/secured servers.

API

Served with the GUI on port 8004. Interactive docs at http://localhost:8004/docs, schema at /openapi.json. If APP_API_KEY is set, requests need X-API-Key: <key> (or ?api_key=); health endpoints and static assets stay public.

POST /create_ontology_guided_kg takes multipart file (PDF/TXT/CSV/JSON/XML, ≤ 50 MB), provider, model, embedding_model, optional ontology_file (.owl/.rdf/.ttl/.xml), max_chunks, kg_name, enable_coreference_resolution.

POST /chat takes JSON question, provider_rag, model_rag, optional kg_name, document_names, session_id. The response info block carries sources, chunk/entity/relationship counts, and the two per-answer trust signals: structural_support (graph-path support) and grounding_support (evidence entailment). The legacy confidence field remains for compatibility.

curl -X POST http://localhost:8004/create_ontology_guided_kg \
  -F "file=@report.pdf" -F "kg_name=demo" \
  -F "provider=openrouter" -F "model=openai/gpt-4o-mini"

curl -N http://localhost:8004/kg_progress_stream    # build progress (SSE)

curl -X POST http://localhost:8004/chat \
  -H "Content-Type: application/json" \
  -d '{"question": "What are the main findings?", "kg_name": "demo",
       "provider_rag": "openrouter", "model_rag": "openai/gpt-4o-mini"}'

Web UI

React + TypeScript SPA (frontend/, Vite) served at http://localhost:8004. Wheels ship built assets; source checkouts need npm run build once.

• Build KG — upload a document, pick provider/model, optionally attach an ontology; extraction progress streams live via SSE.
• Graph view — interactive force-directed network with node details, search (dims non-matches), and per-type filters.
• Named KGs — create, list, and switch between saved graphs.
• Chat — answers cite source chunks, highlight the entities used, and show inline trust pills: Structural (graph-path support) and Grounding (evidence entailment). History persists in localStorage.

What it does

• Ontology-guided extraction — entities and relations validated against your OWL/RDF/JSON schema; no synonym explosion or type drift across a corpus.
• Routed hybrid retrieval — entity-first linking, provenance-aware graph expansion, PPR-style chunk scoring, retriever-first and vector fallbacks, one interface shared with a vanilla-RAG baseline.
• Trust signals — structural (graph-path) and grounding (evidence-entailment) support per answer; full uncertainty suite in the evaluation pipeline.
• Provider-agnostic — OpenRouter, OpenAI, Gemini, DeepSeek, HuggingFace, local Ollama; selectable per request.

Benchmarks & experiment tracking

experiments/README.md covers the benchmark runner (vanilla RAG vs KG-RAG, uncertainty suite, retrieval lock-in study), datasets, and flags. ontograph datasets shows expected local paths, and ontograph prepare <dataset> helps with local dataset setup (including direct downloads where available); ontograph evaluate wraps the runner.

Every benchmark run logs to Weights & Biases automatically (entity WANDB_ENTITY, project mirage-kg-evaluation): manifest config, per-question tables, per-config AUROC/AUREC summaries, and metric charts. Authenticate with wandb login or WANDB_API_KEY; set WANDB_MODE=offline (or disabled) to run without an account. Local artefacts under results/runs/<run_id>/ are always written either way.

Configuration

.env.example documents the core variables: Neo4j connection, provider keys, embeddings, W&B, API security, and server host/port. Advanced behaviour is controlled by env-var families:

Provider keys: openrouter → OPENROUTER_API_KEY (free-tier models available), openai → OPENAI_API_KEY, gemini → GEMINI_API_KEY, deepseek → DEEPSEEK_API_KEY, huggingface → HF_API_TOKEN, ollama → none (set OLLAMA_HOST if non-default). Benchmark sweeps and retrieval thresholds are CLI-flag driven — see experiments/README.md.

Architecture

KG build: ingest → chunk (extraction windows + retrieval sub-chunks) → ontology load → per-chunk LLM extraction → cross-chunk relation pass → entity harmonisation (synonym merge, most-specific type) → provenance stamping → specificity stats (hub down-weighting) → embedding (name-centred entity vectors) → Neo4j write, tagged by kgName, progress via SSE.

RAG query: entity-first seeding (alias match + per-entity ANN) → question-local provenance-aware traversal → PPR-style graph scoring → retriever-first graph expansion when anchoring is weak → vector fallback → evidence organised into chain-style blocks → answer synthesis with Structural/Grounding trust signals.

ontographrag/
├── api/app.py               # FastAPI app, all endpoints, serves UI assets
├── cli.py                   # `ontograph` CLI
├── kg/                      # builders (extraction/harmonisation), loaders (Neo4j),
│                            #   chunking, csv_processor, utils
├── rag/                     # systems (KG-RAG + vanilla baseline), guardrails,
│                            #   reranking, retrieval_sampling
├── schemas/models.py        # Pydantic models
└── providers/model_providers.py  # LLM + embedding providers
experiments/                 # benchmark runner + uncertainty suite
frontend/                    # React + TypeScript web UI
MIRAGE/rawdata/              # local ignored benchmark data (downloaded separately)

Docker

docker compose up -d neo4j   # Neo4j only (development)
docker compose up -d         # Full stack (Neo4j + API)
# Neo4j Browser → http://localhost:7474 (bolt://localhost:7687)

Documentation

Citation

@article{julka2026ontographrag,
  title   = {When Answer Agreement Fails:
             Retrieval-State Lock-In in Retrieval-Augmented Generation},
  author  = {Julka, Sahib},
  journal = {arXiv preprint arXiv:XXXX.XXXXX},
  year    = {2026}
}

License

MIT — see LICENSE. Issues: GitHub Issues.