Skip to main content

fastrag

fastrag

Build RAG chatbots over sources that keep changing.
Ingest, index, retrieve, generate, serve, and benchmark, in plain typed async Python.

PyPI version Python versions Documentation License: Apache 2.0

Documentation · Quickstart · API reference · Source code


fastrag is a Python SDK for the whole life of a retrieval-augmented generation (RAG) system. You compose typed building blocks into one lazy pipeline:

fetch → parse → chunk → embed → index → retrieve → generate → serve

It is built for knowledge bases that don't sit still, such as documentation sites, websites, and document folders. Point it at a source and run it on a schedule. New pages are added, changed pages are re-embedded, deleted pages are pruned, and everything unchanged comes straight from the cache.

Why fastrag?

  • Re-index without paying twice. Every step is cached by its input and its configuration. A re-run only processes what changed, and sync() removes the chunks of pages that no longer exist.
  • A pipeline you own, not a framework you configure. Steps are plain Python objects that you call on each other, evaluated lazily as an async stream. There is no YAML, no string registry, and no hidden global state.
  • Swap any piece. Fetchers, parsers, chunkers, embedders, vector stores, retrievers, rerankers, and chat models share small typed interfaces, so changing provider means changing one constructor.
  • Serving included. create_app() turns a query engine into a FastAPI app with persisted chats, cited answers streamed over Server-Sent Events, sessions, rate limiting, and Prometheus metrics.
  • Measure instead of guessing. Benchmark retrieval and answers against a labelled QA dataset and compare configurations side by side.
  • Install only what you use. The core is small and every integration is an optional extra. A missing extra fails with the exact pip install command to fix it.

Install

pip install "fastrag-ai[html,chunkers,model2vec,chroma,serving,openai]"

Requires Python 3.13 or newer. These extras cover both examples below, and the full list is under Optional extras further down.

Quick look

1. Ingest a website

Crawl a sitemap, convert each page to Markdown, split it into chunks, embed them, and keep a vector collection in sync. This runs entirely on your machine: SQLite holds the step cache, an embedded Chroma database holds the vectors, and a small Model2Vec model computes embeddings on the CPU. You don't need a server or an API key.

import asyncio

from fastrag.ingestion import Pipeline
from fastrag.ingestion.cache import SQLiteCache
from fastrag.ingestion.logger import ConsoleLogger
from fastrag.ingestion.steps.chunking import RecursiveCharacterTextSplitter
from fastrag.ingestion.steps.embedding import Model2VecEmbedder
from fastrag.ingestion.steps.fetching import SitemapCrawler
from fastrag.ingestion.steps.parsing import HtmlParser
from fastrag.vector_store import ChromaVectorStore


async def main():
    async with (
        SQLiteCache(path=".fastrag/cache.sqlite") as cache,
        ChromaVectorStore(path=".fastrag/vectors") as vector_store,
    ):
        crawler = SitemapCrawler(url="https://docs.example.com/sitemap.xml")
        parser = HtmlParser()
        chunker = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
        embedder = Model2VecEmbedder()

        # Each step wraps the one before it. Nothing runs until the pipeline does.
        indexed = vector_store.sync(embedder(chunker(parser(crawler))))
        await ConsoleLogger(Pipeline(indexed, cache=cache)).run()


asyncio.run(main())

Run it again and unchanged pages come from the cache instead of being parsed and embedded again:

+----------------+----------+---------+----------+-----------+----------+
| Source         | Fetching | Parsing | Chunking | Embedding | Indexing |
+----------------+----------+---------+----------+-----------+----------+
| SitemapCrawler | 0/5/0    | 0/5/0   | 0/30/0   | 0/30/0    | 0/30/0   |
+----------------+----------+---------+----------+-----------+----------+
Legend: new/cached/errors

Every piece is replaceable without touching the others. For example, you can use MilvusVectorStore or OpenSearchVectorStore instead of Chroma, PostgresCache instead of SQLite, or OpenAIEmbedder instead of Model2Vec.

2. Serve it over HTTP

Wrap the same collection in a QueryEngine and hand it to create_app. You get a FastAPI app that stores conversations in PostgreSQL and streams grounded answers over Server-Sent Events. Generation works with any OpenAI-compatible endpoint: OpenAI, Ollama, vLLM, or llama.cpp.

import asyncio

import uvicorn

from fastrag.ingestion.steps.embedding import Model2VecEmbedder
from fastrag.query import QueryEngine
from fastrag.query.chat import PostgresChatStore
from fastrag.query.llm import OpenAIChatModel
from fastrag.retrieval import VectorRetriever
from fastrag.serve import ServeConfig, create_app
from fastrag.vector_store import ChromaVectorStore


async def main():
    async with (
        ChromaVectorStore(path=".fastrag/vectors") as vector_store,
        PostgresChatStore(
            database_url="postgresql://postgres:postgres@localhost:5432/fastrag"
        ) as chat_store,
    ):
        retriever = VectorRetriever(
            vector_store=vector_store,
            embedder=Model2VecEmbedder(),  # the same model the collection was built with
        )
        engine = QueryEngine(
            retriever=retriever,
            llm=OpenAIChatModel(
                model="llama3.2:3b",
                api_key="ollama",
                base_url="http://localhost:11434/v1",  # omit to use OpenAI itself
            ),
        )
        app = create_app(
            engine=engine,
            chat_store=chat_store,
            config=ServeConfig(allowed_origins=["http://localhost:3000"]),  # your frontend
        )
        await uvicorn.Server(uvicorn.Config(app, port=8000)).serve()


asyncio.run(main())

Ask it a question, and the sources arrive before the first token:

curl -N http://localhost:8000/v1/chats \
  -H 'content-type: application/json' \
  -d '{"content": "How do I get started?"}'
event: message_start
data: {"chat_id": "01M6…", "user_message_id": "01M6…", "message_id": "01M6…"}

event: sources
data: {"message_id": "01M6…", "sources": [{"url": "https://docs.example.com/getting-started", "title": "Getting started", …}]}

event: delta
data: {"message_id": "01M6…", "delta": {"type": "text", "text": "Install"}}

…

event: message_end
data: {"message_id": "01M6…", "finish_reason": "stop"}

The app also serves chat history, message editing and regeneration, /healthz, and Prometheus /metrics. See the HTTP API and streaming references for the full contract.

If you don't need HTTP, the same engine answers directly from Python:

answer = await engine.ask("How do I get started?")
print(answer.text)     # the answer, ending with the documents it used, e.g. [1][3]
print(answer.sources)  # the retrieved chunks, with their metadata

What's in the box

Stage Built-in components
Fetch SitemapCrawler, WebCrawler, LocalFileFetcher, YouTubeFetcher; JavaScript-rendered pages through PlaywrightRenderer
Parse HtmlParser, DocusaurusParser, TrafilaturaParser, DoclingParser, PDF parsers (MineruParser, DoclingServeParser), ParserRouter
Chunk RecursiveCharacterTextSplitter, HeadingChunker, StructureAwareChunker, SemanticChunker, DoclingChunker
Embed Model2VecEmbedder (local), OpenAIEmbedder (any OpenAI-compatible API)
Store MilvusVectorStore and OpenSearchVectorStore (dense and BM25), ChromaVectorStore
Cache SQLiteCache, PostgresCache
Retrieve VectorRetriever, BM25Retriever, HybridRetriever (rank fusion), RerankingRetriever with CohereReranker, LanguageAwareRetriever
Generate QueryEngine with OpenAIChatModel, using ask() for a whole answer or stream() for tokens
Serve create_app (FastAPI, SSE, PostgreSQL-backed chats), CLI terminal client
Benchmark Benchmark, QADataset, Comparison, with retrieval, generation, and operational metrics

If none of these fit, each kind of component has a small base class to implement. See Customizing.

Optional extras

Extra Enables
html HtmlParser, HTML-to-Markdown conversion
trafilatura TrafilaturaParser, main-content extraction for arbitrary web pages
docling DoclingParser and DoclingChunker for PDFs, Office documents, spreadsheets
pdf pypdf, used by the PDF parsers
dynamic PlaywrightRenderer, for pages rendered by JavaScript
video YouTubeFetcher
chunkers RecursiveCharacterTextSplitter and the other LangChain-based chunkers
normalization Token-aware structural chunking and Markdown/RST normalization
model2vec Model2VecEmbedder, local embeddings
openai OpenAIEmbedder, OpenAIChatModel
milvus MilvusVectorStore
chroma ChromaVectorStore
language LinguaLanguageDetector, for LanguageAwareRetriever
serving create_app, CLI
geoip Country resolution for serving sessions
datasets Reserved for YAML, Parquet, and Hugging Face QADataset loading (planned)
logging Rich console output for the ingestion loggers

Documentation

The full documentation is at fastrag.agrospai.udl.cat.

Project status

fastrag is in alpha. The API may change between minor releases, and each release is recorded in the changelog. Bug reports and questions are welcome on the issue tracker.

Contributing

fastrag is developed in the open at github.com/AgrospAI/fastrag. The contributor guide covers the development setup, tests, conventions, and architecture. Releases are described in the repository README.

License

Licensed under the Apache License, Version 2.0. Built by AgrospAI at the Universitat de Lleida.

Metadata

Release files for fastrag-ai 0.2.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for fastrag-ai 0.2.0
File Size Uploaded Provenance Actions
fastrag_ai-0.2.0.tar.gz 307.7 kB Without provenance Details for fastrag_ai-0.2.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for fastrag-ai 0.2.0
File Interpreter ABI Platform Actions
fastrag_ai-0.2.0-py3-none-any.whl Python 3 none any Details for fastrag_ai-0.2.0-py3-none-any.whl

Total release size: 715.2 kB

Release files / fastrag_ai-0.2.0.tar.gz

Download URL fastrag_ai-0.2.0.tar.gz
Size 307.7 kB
Tags Source
SHA-256 checksum
How to use checksums
8b318d74656a23103634709712b58638b87c58ec996e9acf9ad408682ed73997
BLAKE2b-256 checksum
How to use checksums
4378a9e6dc6db3eec2e344c3816b0ca5bf40907fb8fde6ace3ec11f3bcbca2c3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / fastrag_ai-0.2.0-py3-none-any.whl

Download URL fastrag_ai-0.2.0-py3-none-any.whl
Size 407.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
ad4d783ff24f02b2d6aea3232e4b600e6916ce7ef3d8e55213ebb3ff4845a563
BLAKE2b-256 checksum
How to use checksums
f35333b7a44e469a46f9af128cb10548debcd3838617c823e5086b8dcca3436f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.11.28 {"installer":{"name":"uv","version":"0.11.28","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Security for fastrag-ai 0.2.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.2.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page