Quira

Lightning-Fast, Context-Dense RAG Framework for Python

Stop waiting. Start predicting.

Quickstart  ·  How It Works  ·  Cost Savings  ·  API  ·  Contributing

🔥 The Problem with Traditional RAG

Traditional Retrieval-Augmented Generation (RAG) is slow and expensive:

1. High Latency: User types query → Hits Enter → WAIT → Vector search → WAIT → Stuff 10 large chunks into LLM → WAIT → Response.
2. "Lost in the Middle" Syndrome: You stuff massive chunks of text into the context window, most of which is useless filler. The LLM loses track of the actual facts.
3. Expensive Redundancy: On every turn of the conversation, you re-fetch and re-process the exact same context over and over again.

✨ The Quira Solution (v3.0 Edge & Enterprise)

Quira solves this by predicting what users need before they finish typing, dynamically compressing context to maximize density, and statefully tracking the conversation.

⏱️ Real Latency Reduction | 🧠 3-Tier Context Compression | 💰 Proven Token Savings

🚀 New in v3.0

• Quira Edge (Zero-Server Mode): Run Quira entirely locally using embedded vector databases like DuckDB or SQLite (sqlite-vec). No Redis or Qdrant servers required. Perfect for client-side apps, edge devices, and testing.
• GraphRAG Capabilities: Solves the multi-hop reasoning problem. Quira automatically extracts Entity-Relationship Triplets during ingestion and traverses this Knowledge Graph in parallel with semantic search to provide hyper-accurate context.
• Agentic Routing: Zero-latency heuristics intercept conversational queries (e.g., "Hi", "Thanks"). Bypasses the entire RAG pipeline to return an instant canned response, saving 100% of vector database latency and LLM token costs on chitchat.

🛡️ Enterprise-Ready Core

• Lexical Intent Debouncing: Saves up to 80% on vector database costs by only firing speculative fetches when human intent changes, not on every keystroke.
• Semantic Fuzzy Caching: Matches predicted queries using Cosine Similarity, ensuring cache hits even with typos or phrasing differences.
• 3-Tier Context Compression: Extractive TextRank, Entity-anchored Extraction, and optional LLM Abstractive Summarization packed into the context window.
• Differential Retrieval State: Tracks multi-turn conversations and reuses context securely, reducing database reads and latency safely with proper Garbage Collection.
• Provider Abstraction Layer: Massive database support including Qdrant, Pinecone, Chroma, Weaviate, Supabase, Milvus, pgvector, MongoDB Atlas, Elasticsearch, FAISS, and Neo4j. LLM support for Groq, OpenAI, Gemini, Anthropic, or local Ollama instances.

🏗️ Architecture

graph TD
    User([User Typing]) -->|WebSocket Stream| Speculative[1. Speculative Retriever]
    Speculative -->|Predictive Search| Cache[(Redis Cache)]
    UserSubmit([User Hits Enter]) --> Diff[3. Differential Retriever]
    Diff -->|Cosine Similarity > 0.6?| DeltaFetch{Fetch Delta Chunks Only}
    Cache --> DeltaFetch
    DeltaFetch --> Tetris[2. Context Tetris]
    Tetris -->|Relevance, Recency, Density| Groq[Groq LLM Compression]
    Groq -->|U-Shape Order| FinalContext[Packed Context]
    FinalContext --> MainLLM{Your Main LLM}

📦 Quickstart & Environment Setup

1. Installation

Quira offers a modular installation depending on which providers you want to use.

# Install everything (includes OpenAI, Anthropic, Qdrant, Pinecone, Redis, etc.)
pip install "quira[all]"

# OR install a lightweight minimal setup just for local LLMs and Qdrant
pip install "quira[ollama,qdrant]"

2. Environment Variables

Quira does not hardcode API keys. Make sure your environment is configured for the providers you use:

OPENAI_API_KEY=sk-proj-...
ANTHROPIC_API_KEY=sk-ant-...
GROQ_API_KEY=gsk_...
QDRANT_URL=http://localhost:6333
REDIS_URL=redis://localhost:6379

3. End-to-End Working Example

Here is a complete, runnable script from ingestion to streaming response using the Provider Abstraction Layer.

import asyncio
from quira import quiraPipeline, UserSession

async def main():
    # 1. Initialize Quira Pipeline using simple string configuration
    pipeline = quiraPipeline(
        vector_store="qdrant",
        cache="redis",
        llm="openai/gpt-4o"
    )

    # 2. Create a session for a specific user
    session = UserSession(user_id="user_123")

    # 3. Ingest documents (Auto-detects format: pdf, html, csv, md, docx)
    print("Ingesting document...")
    await pipeline.ingest_file("sample_doc.md", user_id="user_123")

    # 4. 🏎️ Speculative fetch (Requires real-time UI/WebSocket feeding keystrokes)
    # This prepares the context in Redis while the user is typing
    await pipeline.handle_typing_event(session, "What is the ")

    # 5. 🎯 Submit & Stream Response
    print("\nAnswer: ", end="", flush=True)
    async for chunk in pipeline.process_submission_stream(session, "What is the main topic?"):
        print(chunk, end="", flush=True)
    print()

if __name__ == "__main__":
    asyncio.run(main())

⚙️ How It Works: The 4 Core Modules

Quira is built on 4 beautifully orchestrated modules:

🏎️ Module 1: Speculative Retrieval

Instead of waiting for the user to hit "Enter", Quira listens to keystrokes. Using adaptive debouncing, it fires searches in the background. By the time the user hits Enter, the vector search is already cached in Redis.

Note: Speculative Retrieval requires a frontend WebSocket connection feeding typing events to handle_typing_event. Without it, Quira gracefully falls back to standard retrieval on submit.

🧩 Module 2: Context Tetris

Not all retrieved context is equal. Quira scores every chunk on 4 dimensions:

1. Relevance (Cosine similarity)
2. Recency (Half-life decay for older chunks)
3. Uniqueness (Penalizes duplicate information)
4. Density (Entity-to-token ratio)

It then uses a fast LLM to compress filler text out of the chunks, and orders them in a U-shape (best chunks at the very start and end) to prevent the LLM from "losing" facts in the middle of the prompt.

🔄 Module 3: Differential Retrieval

In a normal RAG chat, asking a follow-up question triggers a completely new vector search. Quira maintains a Context Pool. It measures the cosine similarity between the current and previous query. If the topic hasn't changed drastically, Quira only fetches Delta Chunks (new information) and merges it, saving massive amounts of redundant processing.

📄 Module 4: Document Ingestion

Built-in multi-format parsing (PDF, DOCX, HTML, CSV, Markdown) with overlapping text chunking (default 1000 chars / 200 overlap) to prevent sentence fragmentation. Automatically generates embeddings and upserts them directly into your Vector Store.

🛡️ Resilience & Debugging

Quira is built for production reliability. It features a robust Exception Hierarchy (QuiraError) and transparent Retry & Fallback Logic.

Provider Fallbacks

You can provide a secondary fallback_llm or fallback_vector_store. If your primary provider goes down, Quira will seamlessly failover to the backup provider without dropping the user's request.

pipeline = quiraPipeline(
    llm="anthropic/claude-3-opus",
    fallback_llm="openai/gpt-4o", # Used if Anthropic goes down!
    vector_store="pinecone",
    fallback_vector_store="qdrant"
)

Telemetry & Tracing

Want visual graphs of your context compression and speculative fetches? Quira natively instruments itself. If you have langsmith or opentelemetry-api installed in your environment, Quira automatically detects them and wraps the entire pipeline in nested, beautiful traces. No configuration required.

Error Handling & Debugging

If you encounter issues, Quira uses standard Python logging. Enable debug logs to see exact scoring metrics, fallback triggers, and compression ratios:

import logging
logging.getLogger("quira").setLevel(logging.DEBUG)

You can catch specific Quira exceptions such as VectorStoreUnavailableError or LLMProviderError from quira.exceptions for graceful UI degradation.

💰 Why Quira Saves You Money

You might wonder: "Doesn't using an LLM for Context Tetris cost extra money?"

No, it actually saves you up to 40-80% on your bill. Here's why:

1. Compression is Cheap: The models used to compress context cost fractions of a penny.
2. Your Main LLM is Expensive: You are likely sending your final prompt to a heavy model like GPT-4o or Claude 3.5 Sonnet. By using cheap tokens to compress the context, you send significantly fewer tokens to the expensive main LLM.
3. Differential Caching: You stop re-fetching and re-sending identical chunks of text on every single conversational turn.
4. Native Prompt Caching: Quira is fully compatible with Anthropic's Ephemeral Caching, meaning your long-running context pools cost virtually nothing on subsequent turns!

📊 Benchmarks

To verify these metrics yourself, run the test harness in the benchmarks/ directory.

📚 API Reference

quiraPipeline(vector_store, cache, llm, ...)

The main pipeline class. Accepts your own client instances or string identifiers for the Provider Abstraction Layer.

v3.0 Configuration:

• edge_mode (bool): Enable zero-server local execution.
• edge_store (str): "sqlite-vec" or "duckdb".
• enable_graph_rag (bool): Enable Knowledge Graph multi-hop reasoning.
• enable_agentic_routing (bool): Enable zero-latency conversational heuristics.

UserSession(user_id)

Tracks per-user conversation state, context pools, and turn history. Keeps different users' data strictly isolated.

📊 Academic Benchmarks

Quira was evaluated on a comprehensive suite of datasets to measure exact match accuracy against latency reductions across different advanced RAG use-cases.

Run the benchmarks yourself:

python -m benchmarks.run_triviaqa
python -m benchmarks.run_popqa
python -m benchmarks.run_hotpotqa
python -m benchmarks.run_coqa

Metrics recorded on a simulated 80-WPM typing speed using Groq llama-3.1-8b-instant. On average, Quira achieves a 91% reduction in perceived latency and 80% fewer database calls via Lexical Intent Debouncing.

🤝 Contributing

We welcome contributions! Please see our Contributing Guidelines for details on how to submit pull requests, report issues, and request features.po git clone https://github.com/DevDarsh26/Quira.git cd Quira

Create a virtual environment

python -m venv .venv source .venv/bin/activate # Windows: .venv\Scripts\activate

Install in editable mode with dev dependencies

pip install -e ".[dev]"

Run tests

pytest tests/

---

<div align="center">
  <br/>
  <p>Built with ❤️ by <strong><a href="https://darshmodii.in">darshmodii.in</a></strong></p>
  <p>
    <a href="https://github.com/DevDarsh26">
      <img src="https://img.shields.io/badge/GitHub-DevDarsh26-181717?style=flat-square&logo=github" alt="GitHub" />
    </a>
    &nbsp;
    <a href="https://darshmodii.in">
      <img src="https://img.shields.io/badge/Website-darshmodii.in-0969da?style=flat-square&logo=googlechrome&logoColor=white" alt="Website" />
    </a>
  </p>
  <sub>If you like Quira, drop a ⭐ on GitHub — it means the world!</sub>
</div>