Skip to main content

A modular RAG library with embeddings and vector store support

Project description

๐Ÿง  MyRAGProject

Now with interactive setup!

  • On first use, you'll be prompted to choose your embedding model (HuggingFace or Gemini) and vector database (Qdrant or FAISS).
  • The setup wizard will create a .env file for your configuration and install only the dependencies you need.
  • Supports Gemini API key and Qdrant port via .env.

๐Ÿ“ฆ Features

  • ๐Ÿ” Smart chunking of input documents
  • ๐Ÿค– Embedding generation via HuggingFace
  • ๐Ÿง  Vector storage using Qdrant
  • ๐Ÿ’ฌ Ready for question-answering and search
  • ๐Ÿš€ Modular and easy to extend

๐Ÿ—‚๏ธ Folder Structure

myragproject/ โ”œโ”€โ”€ main.py โ”œโ”€โ”€ data/ โ”‚ โ””โ”€โ”€ sample.txt โ”œโ”€โ”€ app/ โ”‚ โ”œโ”€โ”€ config.py โ”‚ โ””โ”€โ”€ ingest.py โ”œโ”€โ”€ raglib/ โ”‚ โ”œโ”€โ”€ chunking/ โ”‚ โ”‚ โ””โ”€โ”€ smart_chunker.py โ”‚ โ”œโ”€โ”€ embeddings/ โ”‚ โ”‚ โ””โ”€โ”€ huggingface.py โ”‚ โ””โ”€โ”€ vector_stores/ โ”‚ โ””โ”€โ”€ qdrant_store.py

yaml Copy Edit


๐Ÿ”ง Setup Instructions

1. Clone the repository

git clone https://github.com/your-username/myragproject.git
cd myragproject
2. Install dependencies
bash
Copy
Edit
pip install -r requirements.txt
Or manually install:

bash
Copy
Edit
pip install sentence-transformers qdrant-client requests
3. Start Qdrant (vector DB)
bash
Copy
Edit
docker run -p 6333:6333 -v qdrant_data:/qdrant/storage qdrant/qdrant
This ensures persistent vector storage.

4. Add your document
Place a .txt file inside the data/ folder, e.g.:

bash
Copy
Edit
data/sample.txt
๐Ÿš€ Run the RAG Ingest Pipeline
bash
Copy
Edit
python main.py
It will:

Read the file

Split it into chunks

Generate embeddings

Store in Qdrant with payloads

๐Ÿ“– Example Output
bash
Copy
Edit
๐Ÿ“„ Starting ingestion for: data/sample.txt
๐Ÿงฉ Created 4 chunks
๐Ÿ”ข Generated embeddings
โœ… Stored 4 points in Qdrant.
โœ… Embeddings stored successfully in Qdrant.
๐Ÿ” Coming Soon
โ“ Ask questions and retrieve top relevant chunks

๐Ÿ“š Support for PDFs and multiple file types

๐Ÿงช FastAPI-based chatbot API

๐Ÿง  Tech Stack
Python 3.10+

HuggingFace (sentence-transformers)

Qdrant vector store

Docker (for Qdrant server)

๐Ÿงฉ Credits
Built with โค๏ธ by [Abhishek]

Inspired by modern RAG systems like LangChain, LlamaIndex

๐Ÿ“„ License
MIT License

yaml
Copy
Edit

---

## โœ… To Use It Now

Just create a `README.md` file in your root folder and paste the above content.

---

Would you like me to generate:

- `requirements.txt` for you?
- Add FastAPI interface?
- Publish this to GitHub with a badge?

Letโ€™s make this production-ready ๐Ÿš€

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

raglab_easygenius-0.1.0.tar.gz (5.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

raglab_easygenius-0.1.0-py3-none-any.whl (6.9 kB view details)

Uploaded Python 3

File details

Details for the file raglab_easygenius-0.1.0.tar.gz.

File metadata

  • Download URL: raglab_easygenius-0.1.0.tar.gz
  • Upload date:
  • Size: 5.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.10.0

File hashes

Hashes for raglab_easygenius-0.1.0.tar.gz
Algorithm Hash digest
SHA256 3ed21af27366636909d4c284ca2b798834aca26897ed22d12758d8200ffc304b
MD5 e55840fb16ecee702b0ea581cea2725c
BLAKE2b-256 329e6a6efb2b4f6304a63b2d50c5d36600b7514a36fb97d0eb3712be5d1bf95c

See more details on using hashes here.

File details

Details for the file raglab_easygenius-0.1.0-py3-none-any.whl.

File metadata

File hashes

Hashes for raglab_easygenius-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 d69647d5cdb974537cc234cd09fa6c88dea3b339107a674073b5db6a5b4813f3
MD5 554ff8944eb39d47c6adafc892d3aeac
BLAKE2b-256 7e44945991b8991739610f304acf8228d6225f30e4d4b1c933de742e72e0e1eb

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page