jupyterhub-user-guide-rag

A small RAG pipeline over the CSUF JupyterHub user guide: it ingests the guide into cited chunks, embeds them into a local vector store, and answers questions with citations back to the guide.

The pipeline is deliberately config-driven and reproducible — every build artifact under data/ is regenerated from source by the commands below, so nothing under data/ is committed (see .gitignore).

Repository layout

This repo (jupyterhub-user-guide-rag/) lives beside the guide content repo, not nested inside it:

rag-project/
├── jupyterhub-user-guide/        # the guide CONTENT (.md/.ipynb) — source of truth
└── jupyterhub-user-guide-rag/    # THIS repo (the RAG code)
    ├── ingest/                   # M1: guide → cited, heading-based chunks
    ├── rag/                      # M2: embed → retrieve → generate (with citations)
    ├── service/                  # length cap + anonymous Q&A logging, shared by both frontends
    ├── sidebar/                  # JupyterLab extension (server extension + panel UI)
    ├── eval/                     # eval set, LLM judge, scoring
    ├── config/                   # ingest.yaml, rag.yaml, service.yaml
    ├── jupyterhub_rag_bundle/    # empty here; the wheel build bakes config/ + data/index/ into it
    ├── pyproject.toml            # builds the `jupyterhub-rag` wheel (M4)
    ├── notebooks/                # M2 prototype notebook
    ├── tests/                    # tests for ingest/, rag/, service/
    └── data/                     # build artifacts (git-ignored, regenerated):
        ├── chunks/chunks.jsonl   #   M1 output
        └── index/                #   M2 vector store (LanceDB "chunks" table)

The anonymous Q&A log buffer is not under data/: it defaults to ~/.jupyterhub-rag/logs (override with $JUPYTERHUB_RAG_LOG_DIR), so it follows the user's home rather than the install location — which is what lets the same config work from a checkout and from the read-only wheel installed into a JupyterHub instance.

Installing as a package

The repo is also a wheel, which is how it reaches a JupyterHub instance:

python -m rag --build                # the index must exist first — it ships in the wheel
python -m build --wheel              # -> dist/jupyterhub_rag-*.whl
python -m build --wheel ./sidebar    # -> dist/jupyterhub_rag_sidebar-*.whl

The wheel carries the configs and the prebuilt index as package data, so pip installing it gives a working bot with no checkout present. For development, pip install -e . from here (then pip install -e "./sidebar[test]") satisfies the sidebar's dependency on the core without publishing anything.

Deployment into the CSUF JupyterHub image is documented in the csuf-dev repo's RAG-DEPLOYMENT.md.

The ingest step reads the guide via config/ingest.yaml's source.repo_path: ../../jupyterhub-user-guide, so the guide repo must sit at that sibling path (or you edit that config value).

The published guide site (citation link target)

Answers cite back to the published guide site, currently https://csuf.github.io/csuf-nautilus-docs. Ingest reads the guide's local vendored files, but derives each chunk's citation URL against this site (page slugs from its sitemap.xml, anchors from its rendered heading ids).

The single switch is site.base_url in config/ingest.yaml. Change it there and re-run ingest to point every citation at a different site. The original source site is kept there commented out as a backup:

base_url: "https://csuf.github.io/csuf-nautilus-docs"
# base_url: "https://noah-hw-kim.github.io/jupyterhub-user-guide"   # backup

Everywhere else the URL appears it is derived from or verifying against that one base_url — if you change the site, update these too so they stay consistent:

How the pipeline fits together

guide repo (.md/.ipynb)
      │  python -m ingest         ← M1, offline, reads local files only
      ▼
data/chunks/chunks.jsonl
      │  python -m rag --build    ← M2, embeds chunks (needs RAG_API_KEY + network)
      ▼
data/index/  (LanceDB "chunks" table)
      │  python -m service "…"    ← M3, length cap + logging wrap around rag/ (needs RAG_API_KEY + network)
      ▼                             (or the sidebar's /chat handler — same service/ core, see below)
answer + citations

• Ingest is fully offline — it reads the guide's local files and derives each chunk's citation URL deterministically; no token or network needed.
• Build and query call the NRP LLM gateway (https://ellm.nrp-nautilus.io/v1, models qwen3-embedding and gpt-oss) and therefore need a token.
• python -m rag --build is the only thing left under rag/'s own CLI (rag/__main__.py) — asking a question goes through service/ instead, which every frontend (CLI, sidebar) shares.

The API key

--build and the query both need an NRP token. service/token.py looks in two places, in order:

1. RAG_API_KEY in the environment — the app never loads .env itself, so the key can't end up committed alongside config. Export it yourself:

   export RAG_API_KEY=<your-NRP-token>
   

   A local .env (git-ignored) is a convenient place to keep it; load it with:

   export $(grep -v '^#' .env | xargs)
   

2. ~/.nrp/llm_token (mode 0600) — the fallback, and what the sidebar's token box writes. It exists because a student inside a running JupyterLab cannot set an environment variable for the server process that is already running; without it, using the panel would mean editing a shell profile and restarting the server first. Point it elsewhere with $JUPYTERHUB_RAG_TOKEN_FILE.

The token is read per request, so pasting one into the panel takes effect on the next question — no restart. python -m rag --build still reads RAG_API_KEY only.

Get a token at nrp.ai/llmtoken/ (see the guide's advanced-configurations/02-get-llm-token.ipynb).

Rebuild from scratch on JupyterHub

A fresh clone has no data/ (it's git-ignored), so you regenerate both the chunks and the index. On the Hub the conda base already provides lancedb, pyyaml, and requests, so no virtualenv is needed — use the default python.

# 0. Go to the project root that holds both repos
cd ~/rag-project

# 1. Make sure the guide CONTENT repo is present beside the code repo.
#    If it's already cloned, update it:
git -C jupyterhub-user-guide pull
#    …otherwise clone it into exactly this sibling path:
# git clone <your-guide-repo-URL> jupyterhub-user-guide

# 2. Into the RAG code repo
cd jupyterhub-user-guide-rag

# 3. (Safety) install deps if anything is missing from the base env
pip install -r requirements.txt

# 4. M1 — INGEST: build chunks.jsonl from the guide (offline, no token)
python -m ingest --config config/ingest.yaml
#    → "Wrote N chunks from M source pages." ; writes data/chunks/chunks.jsonl

# 5. Put your token in the environment
export RAG_API_KEY=<your-NRP-token>
# or if stored in .env
export $(grep -v '^#' .env | xargs)

# 6. M2 — BUILD INDEX: embed the chunks into LanceDB (calls the gateway)
python -m rag --build
#    → "Indexed N chunks into .../data/index/chunks"

# 7. Ask a question (calls the gateway; length cap + anonymous logging applied)
python -m service "How do I request a GPU?"

Shortcut: no guide checkout handy

If you can't check out the guide repo but already have a chunks.jsonl, skip ingest: upload it to data/chunks/chunks.jsonl, then run steps 5–7. (chunks.jsonl is the only thing --build needs.)

Local development

Locally the repo uses a Python 3.14 virtualenv:

python3 -m venv .venv          # only if .venv is missing/broken
source .venv/bin/activate
pip install -r requirements.txt

Then the same steps 4–7 apply. Note a venv is not relocatable — if you move the project directory, recreate .venv (its activate script and config hardcode the original absolute path).

Notes / gotchas

• Order matters: --build (step 6) needs chunks.jsonl to exist, so run ingest (step 4) first.
• Token scope: steps 6 and 7 fail without RAG_API_KEY; step 4 does not need it. An export only lasts for the current terminal — re-export in a new shell.
• Guide repo path is load-bearing: it must sit at ~/rag-project/jupyterhub-user-guide. If it's elsewhere, move it or edit source.repo_path in config/ingest.yaml.
• Idempotent: both steps can be re-run freely — --build overwrites the existing table (mode="overwrite").

Tests

pytest