graphrag-stage1

Turn unstructured scientific text into a typed, provenance-anchored knowledge graph.

graphrag-stage1 is the ingestion/enrichment engine for an ontology-driven GraphRAG system. It is a library, not an app or a service: you import it, hand it your own LLM client, and call functions. It ships no model of its own.

Standard RAG chops documents into chunks and stores a vector per chunk — throwing away the reasoning (cause/effect, negation, who-claimed-it, exact numbers). This keeps all of it, running a multi-pass pipeline modeled on a compiler.

text  ──▶  Stage 1  ──▶  Stage 2  ──▶  Stage 3  ──▶  RDF / JSON-LD / SHACL-validated N-Quads
          atomic facts   verified       ontology
          + facets       frames         mapping

• Stage 1 breaks a paragraph into atomic, self-contained facts, each typed along seven orthogonal facets (measurement? cause? prediction? negated? who claimed it?) and anchored to its exact character span in the source.
• Stage 2 turns each fact into a structured semantic frame — grounding-checked against the source text, with a computed confidence and a routing decision.
• Stage 3 maps verified frames onto a formal OWL ontology (BFO/CCO/IAO + your domain) and emits RDF ready for a graph store such as Amazon Neptune.

The design is neuro-symbolic and that is the point: deterministic code (regex cue extraction, a recall floor, schema and logic validators) provides guarantees; the LLM provides semantic judgement; symbolic validators act as gates. The model is never the only thing standing between the document and the graph. For example, if no number appears in the text, has_measurement is forced false regardless of what the model claimed.

Install

pip install "graphrag-stage1[ontology,grounding] @ git+https://github.com/gedi-onto/graphrag-stage1"

Extras are additive:

The foundational ontologies (IAO, CCO, RO, alignment + SHACL shapes, ~3.7 MB) ship inside the package — nothing to download. You supply only your domain ontology.

Requires Python 3.10+.

Quick start

from graphrag_stage1 import analyze_paper, OllamaClient
from graphrag_stage1.grounding import OakGrounder

results = analyze_paper(
    open("paper.txt").read(),
    client=OllamaClient(model="qwen3:8b"),        # or AnthropicClient() / OpenAIClient()
    domain_ontology="ontologies/Domain/ino_merged.owl",
    grounder=OakGrounder(cache_path=".grounding_cache.json"),
)

for r in results:                                  # one entry per paragraph, in order
    if "error" in r:
        continue
    for fact in r["stage1"]["statements"]:
        f = fact["facets"]
        print(f["proposition_type"], "/", f["relation"], "|", fact["text"])

It works on text, not PDFs — extract text upstream (Textract, PyMuPDF, GROBID).

Three things that will bite you

1. Pass a grounder, or Stage 3 types almost nothing. Without one, an entity that isn't already a class in your ontology can only generalise to a true ancestor — which in practice means BFO:entity, the ontology root. Measured on a real paper: 0/10 entities domain-typed without a grounder, 8/22 with one.

2. num_ctx × OLLAMA_NUM_PARALLEL (local Ollama). Ollama allocates the context window per parallel slot. The defaults are tuned (num_ctx=6144, STAGE2_BATCH_SIZE=5); raising num_ctx to 8192 with OLLAMA_NUM_PARALLEL=4 reserves 32K of KV cache and thrashes VRAM on a 16 GB card. Measured 1026s vs 101s on identical work. If you raise it, lower OLLAMA_NUM_PARALLEL.

3. max_concurrency is a budget, not a throttle. Set it to your provider's allowance. Past it you get HTTP 429s, and each burns the full backoff before failing — slower and you lose paragraphs.

Performance

Measured on examples/pmc_dendrobine.txt (qwen3:8b local, RTX 5080 laptop, defaults): ~25 s per real 70-word paragraph — an 80-paragraph paper in ~33 minutes. A bigger model is not faster; it produces a better graph but costs more decode time per call. Speed comes from parallel capacity (your quota or hardware), not model size.

Status & limitations

Being straight about where this is:

• 122 tests pass. Stages 1 and 2 are solid and well covered; Stage 3 is verified end-to-end from an installed wheel.
• Evaluation is the weak point. The reported metrics are 1.0, but against 8 hand-written gold cases. The release gate (release_gates.json) requires 500 adjudicated paragraphs across 50 documents and 8 domains; the corpus is downloaded (osti_corpus.json) but not yet adjudicated, so readiness_check.py correctly reports BLOCKED. Treat the numbers as directional, not proven.
• Figure / table / equation extractors are routed but not implemented. Stage 1 types such content by role and Stage 2 routes it to a specialist queue, but no handler consumes that queue yet. On a figure-heavy paper this is a meaningful share of content.
• No layout parsing (L0). The pipeline starts from pre-extracted text; provenance is character-offset based, not page/bbox.

Documentation

• docs/graphrag_stage1_user_guide.pdf — the field manual; start here.
• DOCUMENTATION.md — complete reference (API, config, performance).
• DESIGN.md — the architecture rationale.
• docs/CHOOSING_A_MODEL.md — model/provider trade-offs, measured.

License

MIT — see LICENSE.