Skip to main content

A modular RAG library with embeddings and vector store support

Project description

๐Ÿง  MyRAGProject

A lightweight, developer-friendly Retrieval-Augmented Generation (RAG) engine in Python.

This project allows you to ingest .txt files, generate embeddings, store them in Qdrant (a vector database), and later query them to build intelligent AI applications (like chatbots, document Q&A, etc.).


๐Ÿ“ฆ Features

  • ๐Ÿ” Smart chunking of input documents
  • ๐Ÿค– Embedding generation via HuggingFace
  • ๐Ÿง  Vector storage using Qdrant
  • ๐Ÿ’ฌ Ready for question-answering and search
  • ๐Ÿš€ Modular and easy to extend

๐Ÿ—‚๏ธ Folder Structure

myragproject/ โ”œโ”€โ”€ main.py โ”œโ”€โ”€ data/ โ”‚ โ””โ”€โ”€ sample.txt โ”œโ”€โ”€ app/ โ”‚ โ”œโ”€โ”€ config.py โ”‚ โ””โ”€โ”€ ingest.py โ”œโ”€โ”€ raglib/ โ”‚ โ”œโ”€โ”€ chunking/ โ”‚ โ”‚ โ””โ”€โ”€ smart_chunker.py โ”‚ โ”œโ”€โ”€ embeddings/ โ”‚ โ”‚ โ””โ”€โ”€ huggingface.py โ”‚ โ””โ”€โ”€ vector_stores/ โ”‚ โ””โ”€โ”€ qdrant_store.py

yaml Copy Edit


๐Ÿ”ง Setup Instructions

1. Clone the repository

git clone https://github.com/your-username/myragproject.git
cd myragproject
2. Install dependencies
bash
Copy
Edit
pip install -r requirements.txt
Or manually install:

bash
Copy
Edit
pip install sentence-transformers qdrant-client requests
3. Start Qdrant (vector DB)
bash
Copy
Edit
docker run -p 6333:6333 -v qdrant_data:/qdrant/storage qdrant/qdrant
This ensures persistent vector storage.

4. Add your document
Place a .txt file inside the data/ folder, e.g.:

bash
Copy
Edit
data/sample.txt
๐Ÿš€ Run the RAG Ingest Pipeline
bash
Copy
Edit
python main.py
It will:

Read the file

Split it into chunks

Generate embeddings

Store in Qdrant with payloads

๐Ÿ“– Example Output
bash
Copy
Edit
๐Ÿ“„ Starting ingestion for: data/sample.txt
๐Ÿงฉ Created 4 chunks
๐Ÿ”ข Generated embeddings
โœ… Stored 4 points in Qdrant.
โœ… Embeddings stored successfully in Qdrant.
๐Ÿ” Coming Soon
โ“ Ask questions and retrieve top relevant chunks

๐Ÿ“š Support for PDFs and multiple file types

๐Ÿงช FastAPI-based chatbot API

๐Ÿง  Tech Stack
Python 3.10+

HuggingFace (sentence-transformers)

Qdrant vector store

Docker (for Qdrant server)

๐Ÿงฉ Credits
Built with โค๏ธ by [Abhishek]

Inspired by modern RAG systems like LangChain, LlamaIndex

๐Ÿ“„ License
MIT License

yaml
Copy
Edit

---

## โœ… To Use It Now

Just create a `README.md` file in your root folder and paste the above content.

---

Would you like me to generate:

- `requirements.txt` for you?
- Add FastAPI interface?
- Publish this to GitHub with a badge?

Letโ€™s make this production-ready ๐Ÿš€

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

raggenius-0.2.1.tar.gz (6.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

raggenius-0.2.1-py3-none-any.whl (7.6 kB view details)

Uploaded Python 3

File details

Details for the file raggenius-0.2.1.tar.gz.

File metadata

  • Download URL: raggenius-0.2.1.tar.gz
  • Upload date:
  • Size: 6.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.0

File hashes

Hashes for raggenius-0.2.1.tar.gz
Algorithm Hash digest
SHA256 c6224aba3881b7eeaefc3856861ece436ccaf535c4ba3c26a2868f5ccceb0d99
MD5 c57f75886e74abf3cb0d6ec34977b332
BLAKE2b-256 240cb34ece04f8b491ae07f4cda2e42b9bdeeb22e6844bc4f0f74d7f74947116

See more details on using hashes here.

File details

Details for the file raggenius-0.2.1-py3-none-any.whl.

File metadata

  • Download URL: raggenius-0.2.1-py3-none-any.whl
  • Upload date:
  • Size: 7.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.0

File hashes

Hashes for raggenius-0.2.1-py3-none-any.whl
Algorithm Hash digest
SHA256 10792ebd43e2659a6b98d1755b61d2b2a8b352b1796933bea9ce5aa21ab5ad48
MD5 fb4e61460c9a23e2050d3e70984d2c7c
BLAKE2b-256 012ee3ef4314bf2bcbd9d410b9073e3e0c6cafba996418c0fb42d94bd227e9d8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page