GraphBench
GraphBench is an open-source Python library that benchmarks two graph-based RAG pipelines head-to-head on multi-hop question answering.
| Pipeline | Approach |
|---|---|
| GraphRAG | Louvain community detection + Mistral-7B |
| GNN-RAG | 3-layer GAT (PyTorch Geometric) + Mistral-7B |
Dataset: 500 HotpotQA distractor questions (250 bridge, 250 comparison) Knowledge Graph: 50k REBEL triples in Neo4j AuraDB
Installation
pip install graphbench-kg
Or from source:
git clone https://github.com/Rohanjain2312/graphbench.git
cd graphbench
poetry install
Quick Start
from graphbench.utils.neo4j_client import Neo4jClient
from graphbench.utils.faiss_client import FAISSClient
from graphbench.utils.llm_client import LLMClient
from graphbench.pipelines.graphrag_pipeline import GraphRAGPipeline
from graphbench.pipelines.gnnrag_pipeline import GNNRAGPipeline
from graphbench.gnn.model import GATModel
from graphbench.utils.checkpoint import load_checkpoint
from pathlib import Path
# Init shared clients
neo4j = Neo4jClient() # reads NEO4J_URI / NEO4J_USERNAME / NEO4J_PASSWORD from .env
faiss = FAISSClient.load() # reads FAISS_INDEX_PATH from .env
llm = LLMClient(backend="hf")
# GraphRAG pipeline
graphrag = GraphRAGPipeline(neo4j_client=neo4j, faiss_client=faiss, llm_client=llm)
print(graphrag.answer("Where was Marie Curie born?").predicted_answer)
# GNN-RAG pipeline (requires trained checkpoint + entity embeddings)
ckpt = load_checkpoint(Path("checkpoints/gat_best.pt"), map_location="cuda")
model = GATModel()
model.load_state_dict(ckpt["model_state_dict"])
gnnrag = GNNRAGPipeline(neo4j_client=neo4j, faiss_client=faiss, llm_client=llm,
gat_model=model, entity_embeddings=embedding_dict)
print(gnnrag.answer("Where was Marie Curie born?").predicted_answer)
Architecture
See docs/architecture.md for full details.
Benchmark Results
500 HotpotQA distractor questions · seed=42 · Mistral-7B-Instruct-v0.2 (fp16) · 60k REBEL triples
| Metric | GraphRAG | GNN-RAG |
|---|---|---|
| Exact Match (EM) | 3.2% | 5.0% |
| Token F1 | 10.5% | 12.8% |
| Latency p50 (ms) | 5,555 | 5,344 |
| Latency p95 (ms) | 6,388 | 6,199 |
GNN-RAG outperforms GraphRAG on all four metrics. The gap is largest on bridge questions (7.2% vs 3.6% EM), where multi-hop graph traversal benefits most from learned edge scoring.
Full results and per-type breakdown: docs/benchmark_results.md
Project Structure
graphbench/
├── graphbench/ # Core library
│ ├── ingestion/ # REBEL loading, triple extraction, FAISS/Neo4j writers
│ ├── pipelines/ # GraphRAG and GNN-RAG pipelines
│ ├── gnn/ # GAT model, dataset, trainer
│ ├── community/ # Louvain community detection and summarization
│ ├── benchmark/ # HotpotQA loader, evaluator, metrics
│ └── utils/ # Neo4j client, FAISS client, LLM client, config
├── notebooks/ # End-to-end Colab notebook
├── demo/ # Gradio HuggingFace Spaces demo
├── tests/ # Pytest test suite
└── docs/ # Documentation
Requirements
- Python 3.10+
- Neo4j AuraDB Free (or local Neo4j)
- GPU recommended for GNN training (Colab Pro)
- See
.env.examplefor required environment variables
Contributing
See CONTRIBUTING.md.
License
MIT © 2025 Rohan Jain
Metadata
Release files for graphbench-kg 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| graphbench_kg-0.2.0.tar.gz | 44.6 kB | Without provenance | Details for graphbench_kg-0.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| graphbench_kg-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for graphbench_kg-0.2.0-py3-none-any.whl |
Total release size: 103.0 kB
Release files / graphbench_kg-0.2.0.tar.gz
| Download URL | graphbench_kg-0.2.0.tar.gz |
|---|---|
| Size | 44.6 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
319deca845d8463050ac998f998e08fcc8e2b12865fc6cac4d907a61f58b70f4
|
|
BLAKE2b-256 checksum How to use checksums |
0b2e1ba6d54066cb7823776ee7f5474ae312106d254088dcd27383a86d78822e
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.3.2 CPython/3.12.5 Darwin/25.3.0
|
Release files / graphbench_kg-0.2.0-py3-none-any.whl
| Download URL | graphbench_kg-0.2.0-py3-none-any.whl |
|---|---|
| Size | 58.4 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
76cd5fc206ed687e210b366339cb0cda457d40ec46c2516f59d2e2c1bfbc60b1
|
|
BLAKE2b-256 checksum How to use checksums |
36637b60ab53af9abbcfe5b6d51150a6bb5b1b4a4f69b74153ba8f45ddf3b13d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/2.3.2 CPython/3.12.5 Darwin/25.3.0
|
Security for graphbench-kg 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.