tai-aitutor — Migrating from LlamaIndex

The Towards AI course toolkit — plain-Python building blocks for the RAG AI Tutor built across the Full Stack AI Engineer course: provider-neutral LLM calls, embeddings, chunking, retrieval, and evaluation.

Coming from the LlamaIndex version of the course? What you learned carries over; the surface is what changes. Where the course used to set Settings, build an Index, hand a question to a QueryEngine, and score it with an Evaluator, it now calls configure(), opens a Chroma collection, runs search() and then build_rag_prompt() + generate(), and scores with hit_rate / reciprocal_rank / the judge functions. Each step you knew as a class is a function here, and the migration map below pairs them symbol by symbol — read it as a translation table, not a rebuild.

Coming from Sections 1–8? You have already written this code. chunk, embed, search, build_rag_prompt, hit_rate — each one is the function you built inline in a lesson, with the same name, signature, and return. Importing it from here is picking your own work back up, which is why the package starts at Section 9 and not before. Every module docstring names the lesson that builds its contents, so you can always trace a function back to the cell you wrote it in.

Small flat modules, plain functions, readable source. If you can't read a module in one sitting, that's a bug.

Install

Requires Python 3.12+ (3.12 is Colab's current runtime, so the floor stays there as long as Colab does).

# Course profile A (every notebook): the three providers
pip install "tai-aitutor[gemini,openai,anthropic]"

# Course profile B (retrieval lessons): + the vector store
pip install "tai-aitutor[gemini,openai,anthropic,rag]"

cohere (reranking, Cohere embeddings) and sentence-transformers (local embeddings) are lesson-specific: the package never imports them unless you call rerank / embed_cohere / embed_local, so install them only in the lessons that use them — via the extras ([rerank], [local]) or directly (pip install cohere sentence-transformers); any recent version works.

Quickstart

from tai_aitutor import (
    build_rag_prompt, chunk_document, configure, embed, generate,
    get_collection, load_csv, search, setup_notebook, show_answer,
)

IN_COLAB = setup_notebook(required_keys=("GOOGLE_API_KEY",))   # Colab Secrets or .env
PROVIDER = "gemini"   # @param ["gemini", "openai", "anthropic"]
configure(provider=PROVIDER)                                    # replaces Settings

docs = load_csv("mini-llama-articles.csv", text_col="content",
                meta_cols=("title", "url", "source"), id_col="title")
col = get_collection("kb", path="./db")                         # the collection IS the index

chunks = [c for d in docs for c in chunk_document(d)]           # chunk → embed → upsert:
col.add(                                                        # three steps, all visible
    ids=[c.id for c in chunks],
    documents=[c.text for c in chunks],
    embeddings=embed([c.text for c in chunks]),
    metadatas=[c.metadata for c in chunks],
)

question = "What is RAG?"
hits = search(question, col, top_k=5)                           # retrieve
show_answer(generate(build_rag_prompt(question, hits)), hits)   # prompt → generate

Notice what the package does not do for you: the indexing loop and the retrieve → prompt → generate sequence stay in your code, because those are the steps the lessons teach. The package carries the parts inside each step.

Data stays in your notebooks — by design

This package ships no dataset URLs and no downloaders. Datasets are downloaded in the notebook (wget / urllib / hf_hub_download), so the course decides where data is hosted and can move it without a package release; the package only loads whatever file you hand it:

# !wget https://.../mini-llama-articles.csv
docs = load_csv("mini-llama-articles.csv", text_col="content",
                meta_cols=("title", "url", "source"), id_col="title")

Other formats are loaded in the lesson that needs them — a JSONL reader is four lines and the reading of it is part of the point.

QADataset.load(path) opens the course's existing rag_eval_dataset*.json files byte-for-byte (the legacy EmbeddingQAFinetuneDataset JSON shape).

Migration map: LlamaIndex → tai_aitutor

Also delete on sight: every nest_asyncio.apply() cell (only existed for LlamaIndex's async), every llama-index-* pip pin, and LLAMA_CLOUD_API_KEY setup.

Config

LLMs and embeddings

Documents, chunking, ingestion, storage

Retrieval and answering

Evaluation

Agents, chat, tools, routing

Fine-tuning

What's in the package

config (configure, setup_notebook, require_keys, in_colab) · llm (generate, extract) · embeddings (embed, embed_cohere, embed_local, EMBED_DIM) · tokens (n_tokens) · documents (Document, load_csv) · chunking (Chunk, chunk, chunk_document, chunk_sentences, heading_aware_markdown_chunks, sentence_window_chunks) · vectorstore (get_collection, reset_collection, get_all_chunks, build_where_filter) · retrieval (search, ScoredChunk, expand_window, code_tokenize, BM25Index, rrf_fuse, rerank, judge_rerank, hyde_search, decompose_question) · synthesis (build_rag_prompt) · evals (QADataset, hit_rate, reciprocal_rank, evaluate_retrieval, sweep_top_k, the three judges) · tools (Tool, tool, search_web) · router (route, RouteDecision) · display · errors (all subclass ValueError)

Every one of these has an inline twin in a Section 1–8 lesson. Composites that would hide several taught steps behind one call are deliberately absent — you assemble the pipeline, the package supplies the parts.

Runnable end-to-end examples (with explicit data downloads) in examples/. History: CHANGELOG.md; build tracker: BUILD_STATUS.md; the original design + full LlamaIndex inventory: docs/PACKAGE_PLAN.md.

Design rules

1. Teach-then-import. A concept's first appearance is written out in its lesson notebook; later notebooks import it from here. Each module docstring names the lesson that builds it.
2. Functions over object graphs. State lives in one configure() call and in the objects you pass explicitly — the collection, the index, the dataset. Where LlamaIndex handed you an object that carried the pipeline inside it, you hold the pieces.
3. Data downloads live in notebooks, never in the package.
4. Mirror production. Where the production AI Tutor has an equivalent, constants and logic match.
5. Loud failures. Typo'd kwargs raise TypeError, data parsing never uses eval(), missing keys and missing extras say exactly what to install or set.

Development

pip install -e ".[gemini,openai,anthropic,rag,rerank,parse]" pytest ruff
pytest              # fully offline — provider SDKs are faked
ruff check src tests

Releases: bump __version__ + CHANGELOG.md, then git tag vX.Y.Z && git push --tags — .github/workflows/release.yml publishes to PyPI via Trusted Publishing.

License

MIT © Towards AI