ragkitpy 0.1.1

A lightweight RAG pipeline toolkit powered by HuggingFace

ragkitpy 🤗

A lightweight, beginner-friendly RAG (Retrieval-Augmented Generation) pipeline toolkit powered by HuggingFace 🤗 — no OpenAI key required.

---

🚀 Why ragkitpy?

Setting up a RAG pipeline from scratch is painful. LangChain and LlamaIndex are powerful but heavy and complex for beginners. ragkitpy gives you a working RAG pipeline in 5 lines of code using only open-source models.

• ✅ No OpenAI API key needed — runs fully local with HuggingFace
• ✅ Supports PDF and TXT files
• ✅ Built on FAISS for fast vector search
• ✅ Beginner-friendly API
• ✅ Lightweight — minimal dependencies

---

📦 Installation

pip install ragkitpy

---

⚡ Quick Start

from ragkitpy import RAGPipeline

# Initialize
rag = RAGPipeline()

# Load your document (PDF or TXT)
rag.load_document("my_document.pdf")

# Query it
results = rag.query("What is this document about?")

for i, chunk in enumerate(results, 1):
    print(f"Result {i}: {chunk}")

That's it. 5 lines to a working RAG pipeline. 🎯

---

🧠 How It Works

Your Document (PDF/TXT)
        ↓
   [loader.py]  →  Extract raw text
        ↓
  [chunker.py]  →  Split into overlapping chunks
        ↓
  [embedder.py] →  HuggingFace embeddings (all-MiniLM-L6-v2)
        ↓
[vectorstore.py]→  Store in FAISS index
        ↓
   Your Query   →  Embed query → Search FAISS → Return top-k chunks

---

📖 Full API Reference

RAGPipeline

from ragkitpy import RAGPipeline

rag = RAGPipeline(
    model_name="all-MiniLM-L6-v2",  # Any SentenceTransformer model
    chunk_size=500,                   # Characters per chunk
    overlap=50                        # Overlap between chunks
)

Method	Description
rag.load_document(path)	Load a PDF or TXT file
rag.query(question, top_k=3)	Get top-k relevant chunks
rag.query_with_scores(question, top_k=3)	Get chunks with similarity scores
rag.is_ready	Check if document is loaded
rag.source	Path of loaded document

---

Use a different HuggingFace model

# Larger model, better quality
rag = RAGPipeline(model_name="all-mpnet-base-v2")

Get similarity scores

results = rag.query_with_scores("What is machine learning?", top_k=3)
for chunk, score in results:
    print(f"Score: {score:.4f} | {chunk[:100]}")

Use individual modules

from ragkitpy import load_file, chunk_text, HFEmbedder, VectorStore

# Use just the loader
text = load_file("document.pdf")

# Use just the chunker
chunks = chunk_text(text, chunk_size=300, overlap=30)

# Use sentence chunker instead
from ragkitpy.chunker import chunk_by_sentences
chunks = chunk_by_sentences(text, sentences_per_chunk=3)

---

🗂️ Project Structure

ragkitpy/
├── ragkitpy/
│   ├── __init__.py       # Public API
│   ├── loader.py         # PDF & TXT file loading
│   ├── chunker.py        # Text chunking strategies
│   ├── embedder.py       # HuggingFace embeddings wrapper
│   ├── vectorstore.py    # FAISS vector store
│   └── pipeline.py       # End-to-end RAG pipeline
├── tests/                # 24 unit tests
├── examples/             # Working examples
└── pyproject.toml

---

🔧 Dependencies

Package	Purpose
sentence-transformers	HuggingFace text embeddings
faiss-cpu	Vector similarity search
pypdf	PDF text extraction
numpy	Array operations

---

🤝 Contributing

Contributions are welcome! Please read CONTRIBUTING.md first.

git clone https://github.com/SRINATH-GITHU/ragkitpy.git
cd ragkitpy
pip install -e ".[dev]"
pytest tests/ -v

---

📄 License

MIT License — see LICENSE for details.

---

👨‍💻 Author

Built by Srinath Dhumnor as an open-source contribution to the GenAI community.

⭐ Star this repo if it helped you!

---

Built by Srinath Dhumnoor as an open-source contribution to the GenAI community.

If this project helped you build something cool, ⭐ star the repo — it really helps others discover it!

Questions, ideas or feedback? Feel free to open an issue. 🚀