ragbits 0.20.0

Building blocks for rapid development of GenAI applications

🐰 Ragbits

Building blocks for rapid development of GenAI applications

Homepage | Documentation | Contact

---

Features

🔨 Build Reliable & Scalable GenAI Apps

• Swap LLMs anytime – Switch between 100+ LLMs via LiteLLM or run local models.
• Type-safe LLM calls – Use Python generics to enforce strict type safety in model interactions.
• Bring your own vector store – Connect to Qdrant, PgVector, and more with built-in support.
• Developer tools included – Manage vector stores, query pipelines, and test prompts from your terminal.
• Modular installation – Install only what you need, reducing dependencies and improving performance.

📚 Fast & Flexible RAG Processing

• Ingest 20+ formats – Process PDFs, HTML, spreadsheets, presentations, and more. Process data using Docling, Unstructured or create a custom parser.
• Handle complex data – Extract tables, images, and structured content with built-in VLMs support.
• Connect to any data source – Use prebuilt connectors for S3, GCS, Azure, or implement your own.
• Scale ingestion – Process large datasets quickly with Ray-based parallel processing.

🚀 Deploy & Monitor with Confidence

• Real-time observability – Track performance with OpenTelemetry and CLI insights.
• Built-in testing – Validate prompts with promptfoo before deployment.
• Auto-optimization – Continuously evaluate and refine model performance.
• Visual testing UI (Coming Soon) – Test and optimize applications with a visual interface.

Installation

To get started quickly, you can install with:

pip install ragbits

This is a starter bundle of packages, containing:

• ragbits-core - fundamental tools for working with prompts, LLMs and vector databases.
• ragbits-agents - abstractions for building agentic systems.
• ragbits-document-search - retrieval and ingestion piplines for knowledge bases.
• ragbits-evaluate - unified evaluation framework for Ragbits components.
• ragbits-chat - full-stack infrastructure for building conversational AI applications.
• ragbits-cli - ragbits shell command for interacting with Ragbits components.

Alternatively, you can use individual components of the stack by installing their respective packages.

Quickstart

Basics

To define a prompt and run LLM:

import asyncio
from pydantic import BaseModel
from ragbits.core.llms import LiteLLM
from ragbits.core.prompt import Prompt

class QuestionAnswerPromptInput(BaseModel):
    question: str

class QuestionAnswerPromptOutput(BaseModel):
    answer: str

class QuestionAnswerPrompt(Prompt[QuestionAnswerPromptInput, QuestionAnswerPromptOutput]):
    system_prompt = """
    You are a question answering agent. Answer the question to the best of your ability.
    """
    user_prompt = """
    Question: {{ question }}
    """

llm = LiteLLM(model_name="gpt-4.1-nano", use_structured_output=True)

async def main() -> None:
    prompt = QuestionAnswerPrompt(QuestionAnswerPromptInput(question="What are high memory and low memory on linux?"))
    response = await llm.generate(prompt)
    print(response.answer)

if __name__ == "__main__":
    asyncio.run(main())

Document Search

To build and query a simple vector store index:

import asyncio
from ragbits.core.embeddings import LiteLLMEmbedder
from ragbits.core.vector_stores import InMemoryVectorStore
from ragbits.document_search import DocumentSearch

embedder = LiteLLMEmbedder(model_name="text-embedding-3-small")
vector_store = InMemoryVectorStore(embedder=embedder)
document_search = DocumentSearch(vector_store=vector_store)

async def run() -> None:
    await document_search.ingest("web://https://arxiv.org/pdf/1706.03762")
    result = await document_search.search("What are the key findings presented in this paper?")
    print(result)

if __name__ == "__main__":
    asyncio.run(run())

Retrieval-Augmented Generation

To build a simple RAG pipeline:

import asyncio
from pydantic import BaseModel
from ragbits.core.embeddings import LiteLLMEmbedder
from ragbits.core.llms import LiteLLM
from ragbits.core.prompt import Prompt
from ragbits.core.vector_stores import InMemoryVectorStore
from ragbits.document_search import DocumentSearch

class QuestionAnswerPromptInput(BaseModel):
    question: str
    context: list[str]

class QuestionAnswerPromptOutput(BaseModel):
    answer: str

class QuestionAnswerPrompt(Prompt[QuestionAnswerPromptInput, QuestionAnswerPromptOutput]):
    system_prompt = """
    You are a question answering agent. Answer the question that will be provided using context.
    If in the given context there is not enough information refuse to answer.
    """
    user_prompt = """
    Question: {{ question }}
    Context: {% for item in context %}
        {{ item }}
    {%- endfor %}
    """

embedder = LiteLLMEmbedder(model_name="text-embedding-3-small")
vector_store = InMemoryVectorStore(embedder=embedder)
document_search = DocumentSearch(vector_store=vector_store)
llm = LiteLLM(model_name="gpt-4.1-nano", use_structured_output=True)

async def run() -> None:
    question = "What are the key findings presented in this paper?"

    await document_search.ingest("web://https://arxiv.org/pdf/1706.03762")
    result = await document_search.search(question)

    prompt = QuestionAnswerPrompt(QuestionAnswerPromptInput(
        question=question,
        context=[element.text_representation for element in result],
    ))
    response = await llm.generate(prompt)
    print(response.answer)

if __name__ == "__main__":
    asyncio.run(run())

Rapid development

Create Ragbits projects from templates:

uvx create-ragbits-app

Explore create-ragbits-app repo here. If you have a new idea for a template, feel free to contribute!

Documentation

• Quickstart - Get started with Ragbits in a few minutes
• How-to - Learn how to use Ragbits in your projects
• CLI - Learn how to run Ragbits in your terminal
• API reference - Explore the underlying Ragbits API

Contributing

We welcome contributions! Please read CONTRIBUTING.md for more information.

License

Ragbits is licensed under the MIT License.