🤖 ROSRAG — Robotics Retrieval-Augmented Generation Assistant

An intelligent robotics assistant that answers ROS2, drone, SLAM, and computer vision questions using a production-quality Retrieval-Augmented Generation (RAG) pipeline.

---

📋 Table of Contents

• Overview
• Architecture
• Project Structure
• Setup & Installation
• Running ROSRAG
• Example Queries & Output
• Configuration
• Extending the Knowledge Base
• Deployment
• Future Improvements

---

🎯 Overview

ROSRAG is a domain-specific AI assistant built on a Retrieval-Augmented Generation (RAG) architecture. Instead of relying solely on an LLM's parametric knowledge, ROSRAG:

1. Retrieves the most relevant passages from a curated robotics knowledge base (ROS2 docs, drone manuals, SLAM research notes)
2. Augments the LLM prompt with retrieved context
3. Generates accurate, grounded, technically precise answers via Google Gemini

This approach drastically reduces hallucinations and keeps answers anchored to verified robotics documentation.

---

🏗️ Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        ROSRAG Pipeline                          │
│                                                                 │
│  ┌──────────────┐    ┌──────────────┐    ┌──────────────────┐  │
│  │   Knowledge  │    │  Embedding   │    │   FAISS Vector   │  │
│  │   Base JSON  │───▶│  Layer       │───▶│   Store (Index)  │  │
│  │  (data/)     │    │ (MiniLM-L6)  │    │                  │  │
│  └──────────────┘    └──────────────┘    └──────────────────┘  │
│                                                   │             │
│  ┌──────────────┐                                 │ Top-K       │
│  │  User Query  │                          ┌──────▼──────┐     │
│  │  (CLI / UI)  │─────────────────────────▶│  Retriever  │     │
│  └──────────────┘    Query Embedding        └──────┬──────┘     │
│                                                   │             │
│                             ┌─────────────────────▼───────┐    │
│                             │      Prompt Builder         │    │
│                             │  System + Context + Query   │    │
│                             └─────────────────┬───────────┘    │
│                                               │                 │
│                             ┌─────────────────▼───────────┐    │
│                             │   Google Gemini 1.5 Flash   │    │
│                             │        (LLM Layer)          │    │
│                             └─────────────────┬───────────┘    │
│                                               │                 │
│                             ┌─────────────────▼───────────┐    │
│                             │  Response + Source Citations │    │
│                             └─────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────┘

Component Breakdown

Component	File	Responsibility
Ingestion	src/ingestion.py	Load JSON docs, clean text, chunk into passages
Embedding	src/embedding.py	Convert text chunks to L2-normalized float32 vectors
Vector Store	src/vector_store.py	FAISS IndexFlatIP build, save/load, similarity search
Retriever	src/retriever.py	Embed query, search store, return ranked results
LLM	src/llm.py	Prompt construction, Gemini API calls
Pipeline	src/rag_pipeline.py	Orchestrates full flow, chat history, response struct
Utils	src/utils.py	Logging, config loading, formatting helpers
CLI	app.py	Interactive terminal interface with special commands
Web UI	streamlit_app.py	Streamlit chat interface with source panel

---

📁 Project Structure

rosrag/
│
├── data/
│   ├── knowledge.json           # Primary robotics knowledge base
│   ├── rosrag_index.faiss       # (generated) FAISS vector index
│   └── rosrag_metadata.pkl      # (generated) Document metadata
│
├── src/
│   ├── __init__.py
│   ├── ingestion.py             # Data loading, cleaning, chunking
│   ├── embedding.py             # SentenceTransformer embedding
│   ├── vector_store.py          # FAISS index management
│   ├── retriever.py             # Query → context retrieval
│   ├── llm.py                   # Gemini API + prompt engineering
│   ├── rag_pipeline.py          # Full pipeline orchestrator
│   └── utils.py                 # Logging, config, formatting
│
├── logs/
│   └── rosrag.log               # (generated) Application logs
│
├── app.py                       # CLI entry point
├── streamlit_app.py             # Streamlit web UI
├── requirements.txt             # Python dependencies
├── config.json                  # Configuration file
└── README.md

---

⚙️ Setup & Installation

Prerequisites

• Python 3.10+
• Ubuntu 20.04 / 22.04 (or any Linux/macOS terminal)
• Google Gemini API key (get one here)

Step 1 — Clone / Download the Project

git clone <your-repo-url> rosrag
cd rosrag

Step 2 — Create a Virtual Environment

python3 -m venv venv
source venv/bin/activate

Step 3 — Install Dependencies

pip install -r requirements.txt

⚠️ First install may take a few minutes to download the embedding model (~90MB) and PyTorch.

Step 4 — Set Your Gemini API Key

export GEMINI_API_KEY="your-gemini-api-key-here"

Add to ~/.bashrc for persistence:

echo 'export GEMINI_API_KEY="your-key-here"' >> ~/.bashrc
source ~/.bashrc

---

🚀 Running ROSRAG

CLI Mode (Recommended)

python app.py

Options:

python app.py --rebuild      # Force rebuild FAISS index from knowledge base
python app.py --debug        # Enable verbose debug logging
python app.py --config path/to/config.json
python app.py --data path/to/custom_knowledge.json

CLI Commands during chat:

/help      — Show available commands
/history   — View conversation history
/clear     — Clear chat history
/sources   — Show sources from last answer
/quit      — Exit

Web UI Mode (Streamlit)

streamlit run streamlit_app.py

Then open http://localhost:8501 in your browser.

---

💬 Example Queries & Output

Query 1: ROS2 Basics

You ❯ How do I create a publisher node in ROS2 Python?

🤖 ROSRAG:

To create a publisher node in ROS2 using Python (rclpy), follow these steps:

1. Import the required modules:
   import rclpy
   from rclpy.node import Node
   from std_msgs.msg import String

2. Create a class inheriting from Node:
   class MyPublisher(Node):
       def __init__(self):
           super().__init__('my_publisher_node')
           self.publisher_ = self.create_publisher(String, 'topic_name', 10)
           timer_period = 0.5  # seconds
           self.timer = self.create_timer(timer_period, self.timer_callback)

       def timer_callback(self):
           msg = String()
           msg.data = 'Hello ROS2!'
           self.publisher_.publish(msg)

── Sources Retrieved ─────────────────────
  [1] ROS2 Documentation — ROS2 Topics  (score: 0.842)
  [2] ROS2 Documentation — ROS2 Nodes   (score: 0.791)
── Latency: 1243ms ───────────────────────

Query 2: SLAM

You ❯ What is SLAM and which algorithms are used with LiDAR?

🤖 ROSRAG:

SLAM (Simultaneous Localization and Mapping) is the process of building a map
of an unknown environment while simultaneously tracking the robot's location
within it...

[LiDAR SLAM algorithms include Cartographer, LOAM, LeGO-LOAM, LIO-SAM, KISS-ICP...]

Query 3: Drone Systems

You ❯ How does PX4 Offboard mode work with ROS2?

🤖 ROSRAG:

PX4 Offboard mode allows an external computer to control the drone by sending
setpoints at a minimum rate of 2Hz. In ROS2, you connect via the px4_msgs
package and micro-XRCE-DDS bridge...

---

🔧 Configuration

Edit config.json to customize behavior:

{
  "data_path": "data/knowledge.json",
  "index_path": "data/rosrag_index.faiss",
  "meta_path": "data/rosrag_metadata.pkl",
  "embed_model": "all-MiniLM-L6-v2",
  "gemini_model": "gemini-1.5-flash",
  "top_k": 5,
  "min_score": 0.2,
  "log_level": "INFO",
  "log_file": "rosrag.log"
}

Parameter	Default	Description
top_k	5	Number of chunks to retrieve per query
min_score	0.2	Minimum cosine similarity threshold (0–1)
gemini_model	gemini-1.5-flash	Gemini model variant
embed_model	all-MiniLM-L6-v2	SentenceTransformer model
log_level	INFO	Logging verbosity

---

📖 Extending the Knowledge Base

Add new knowledge entries to data/knowledge.json:

{
  "id": "custom_001",
  "source": "My Custom Notes",
  "topic": "Custom Topic",
  "content": "Your knowledge text here. Can be multiple sentences or paragraphs. The ingestion pipeline will chunk it automatically."
}

Then rebuild the index:

python app.py --rebuild

---

☁️ Deployment

Local (Ubuntu)

source venv/bin/activate
export GEMINI_API_KEY="your-key"
python app.py                          # CLI
streamlit run streamlit_app.py         # Web UI

Streamlit Cloud

1. Push your project to a public GitHub repo
2. Go to share.streamlit.io
3. Connect your repo and set streamlit_app.py as the entry point
4. Add GEMINI_API_KEY as a secret in the Streamlit Cloud dashboard

AWS EC2 (Ubuntu 22.04)

# Launch t3.medium instance, SSH in, then:
sudo apt update && sudo apt install python3-pip python3-venv -y
git clone <your-repo-url> rosrag && cd rosrag
python3 -m venv venv && source venv/bin/activate
pip install -r requirements.txt

# Set API key
export GEMINI_API_KEY="your-key"

# Run CLI in background with tmux
tmux new-session -d -s rosrag 'python app.py'

# Or run Streamlit with nohup
nohup streamlit run streamlit_app.py --server.port 8501 --server.address 0.0.0.0 &
# Open EC2 security group port 8501

---

🔮 Future Improvements

Feature	Priority	Description
Hybrid Search	High	Combine BM25 lexical search with dense retrieval
Re-ranking	High	Add cross-encoder re-ranking for improved precision
AWS S3 Integration	Medium	Load/save knowledge base and index from S3
Multi-modal RAG	Medium	Support ingesting robot documentation PDFs
Streaming Responses	Medium	Stream Gemini output token-by-token in Streamlit
Docker Container	Medium	Containerize for one-command deployment
Query Expansion	Low	Use LLM to rephrase query before retrieval
Evaluation Suite	Low	Automated RAG evaluation with RAGAS metrics
Knowledge Graph	Low	Add entity relationship graph over robotics concepts

---

📜 License

MIT License — free to use, modify, and distribute.

---

🙏 Credits

• Google Generative AI — Gemini LLM
• Sentence Transformers — Embedding model
• FAISS — Facebook AI Similarity Search
• Streamlit — Web UI framework
• ROS2 documentation, PX4 documentation, and robotics research community

---

Built as a production-grade engineering portfolio project demonstrating end-to-end RAG system design.