sci-rag-kit 0.5.0

Retrieval-augmented generation, built around your scientific domain.

Sci RAG Kit is a project template for question answering over scientific documents. It parses papers and reports, indexes their passages, runs five kinds of search, and measures retrieval against questions with known answers. With LLM provider credentials, it also builds a concept graph and writes answers that cite the passages they use. REST and MCP clients share the same service. Your project supplies the documents, the vocabulary of its field, and the questions used to score the result.

Start a new project with two commands. sci-rag new asks a few questions and writes a configured, git-initialized project directory:

pipx install sci-rag-kit
sci-rag new

To read the kit before creating a project, clone this repository and run the demo corpus (see Set up). The documentation site has the guided path. The quickstart takes about ten minutes.

Components

• Ingestion. PDF, HTML, Markdown, and plain-text files become passages that keep their section headings and their tables whole, so a value pulled from a results table still carries the heading that says what it measured. Each document also records its authors, its source, and whether its text may be redistributed, and a file ingested twice is recognized and skipped.
• One database. Passages, vectors, a full-text index, and the concept graph all live in PostgreSQL with the pgvector extension. That is a deliberate choice over a separate vector store or graph database: it leaves one thing to run, one thing to back up, and one place where a passage and its graph entries commit together.
• A concept graph. With LLM provider credentials, the kit reads every passage and extracts the concepts and relationships declared in domain/domain.yaml, then clusters related concepts and writes a summary of each cluster. The graph helps retrieve evidence when an answer spans several documents.
• Five kinds of search. By meaning, by exact words, through the concept graph, through the cluster summaries, and through a model-written hypothetical answer. Each finds evidence the others miss: keyword search catches an identifier that embeddings blur, and the graph reaches a passage whose words never appear in the question. The five result lists merge into one ranking, and every result names the layer that found it.
• Cited answers. The model answers from the retrieved passages only and cites each one by number, so a reader can check any claim against its source. When the documents cannot support an answer, the kit reports the gap.
• Rights. Every document carries a license class. A request that restricts rights never sees passages outside its scope, because the filter runs inside each search before ranking, so a shared endpoint cannot leak a paywalled PDF held internally.
• Measurement. A file of questions with known answers lets the kit score retrieval, grade generated answers, and report what each search layer contributes on the corpus at hand. Every report records the documents and models that produced it, so a number can be traced back to what it measured.
• Serving. One process answers the command line, a REST API with interactive docs at /docs, and agents over MCP, the protocol Claude Code and similar tools use to call external systems. All three go through the same service, so they cannot drift apart. API keys carry scopes and rate limits.
• Models. Gemini by default, through a Google AI Studio key or a Vertex AI project. Claude and any OpenAI-compatible endpoint are one setting away for generation, and the model that grades answers can differ from the one that writes them. An offline mode runs everything except the graph and generated answers, with no credential at all.

Set up

Setup needs uv, Docker or a PostgreSQL 16 through 18 server with pgvector, and optionally LLM provider credentials such as a Google AI Studio API key.

Create the local configuration file. The second command matters: the file is about to hold a credential, and cp alone leaves it readable by every account on the machine.

cp .env.example .env
chmod 600 .env

In .env, set one of these:

Then install, start the database, and run the demo:

make setup     # install dependencies, start Postgres, create the tables
make demo      # ingest the demo corpus, run a retrieval, score it

make setup starts the selected database backend and creates every table. Docker is the template default. A project can instead use a conda-forge server, a system PostgreSQL such as Postgres.app, or a Cloud SQL development instance; Configure Postgres backend covers each.

With a credential configured, the graph and the answers work too:

uv run sci-rag answer "What conversion route suits rice straw given its ash content?"
make demo-cloud        # build the graph, ask a multi-document question, score each layer
uv run sci-rag serve   # REST at /docs, MCP at /mcp

The demo corpus is five short synthetic documents about agricultural residues, with plausible but fictional numbers. It exists so the pipeline runs end to end before a real corpus goes in.

Use your own documents

Put PDFs, HTML, Markdown, or text files in data/raw/, then run seven commands. Bring your own domain explains each one.

uv run sci-rag draft manifest --folder data/raw      # 1. describe the documents
uv run sci-rag manifest lint data/corpus.jsonl        # 2. check the description, decide rights
uv run sci-rag draft ontology --folder data/raw       # 3. name the concepts your field cares about
uv run sci-rag build --manifest data/corpus.jsonl     # 4. ingest, then build the graph
uv run sci-rag draft questions --count 10             # 5. draft questions with known answers
uv run sci-rag eval retrieval --ablation              # 6. measure
uv run sci-rag answer "a question in your field"      # 7. ask

The three draft commands write a file to review, and each works without LLM provider credentials: --print-prompt prints the prompt, and --from-file reads the reply back. Printed prompts can contain sampled passages from your corpus. Before sending one to an assistant, confirm that the documents' rights, privacy requirements, provider terms, and institutional policy allow that disclosure. Two things are never drafted: a document's rights, and the human labels that check the answer grader.

For a first pass with no manifest, uv run sci-rag build data/raw ingests a folder directly and builds the graph when a credential is present.

Commands

uv run sci-rag --help groups every command by stage, and the CLI reference lists every option. The commands most projects use daily:

Register the MCP server with a local agent:

claude mcp add my-corpus -- uv run --directory /path/to/your/repo sci-rag mcp

Repository layout

domain/            The field: concepts, prompts, test questions
data/raw/          The documents; data/corpus.jsonl describes them
data/demo/         The demo corpus (synthetic, CC0)
src/sci_rag/       The pipeline: ingest, embed, graph, retrieve, answer, evals, server, cli
migrations/        Database tables
tests/             Runs offline; database tests use a disposable database
infra/terraform/   Optional Google Cloud deployment
docs/              This documentation

Documentation

The full site is at sustainability-software-lab.github.io/sci-rag-kit.

Requirements and defaults

Python 3.11 or 3.12. PostgreSQL 16 through 18 with pgvector; Docker is the template default and matches CI. Embeddings come from Google's gemini-embedding-001 at 1536 dimensions. Generation defaults to a Gemini model; uv run sci-rag doctor prints the current one. Docling gives the best PDF parsing and is an optional extra (uv sync --extra docling) because of its size. Without it, pypdf handles PDFs with reduced table fidelity.

License

BSD 3-Clause. See LICENSE.