multimodal-agent 0.2.8

A Python multimodal agent for interacting with Gemini models via text, images, and CLI.

Multimodal-Agent

A lightweight, production-ready multimodal wrapper for Google Gemini — with RAG memory, session-based chat, embeddings, retry logic, and a clean CLI.

---

Features (v0.2.7)

Core

• 🔹 RAG Memory (SQLite vector store, embedding retrieval)

• 🔹 Session-based chat (agent chat --session <id>)

• 🔹 Cross-session RAG retrieval

• 🔹 History management CLI

  (show / delete / clear / summary)

• 🔹 Text + Image multimodal generation

Engine

• 🔹 Token-safe chunking (sentence-aware, fallback for long text)
• 🔹 Exponential backoff retry logic
• 🔹 Custom exception hierarchy
• 🔹 Production logging
• 🔹 Extensible & test-covered architecture

Tooling

• 🔹 Minimal CLI: agent
• 🔹 94% unit test coverage

---

Installation

From PyPI (recommended)

pip install multimodal-agent

From source

git clone https://github.com/yourname/multimodal-agent.git
cd multimodal-agent
pip install -e .

---

Requirements

• Python 3.9+
• GOOGLE_API_KEY set in .env file:

GOOGLE_API_KEY=your_key_here

Dependencies (google-genai, google-adk) are installed automatically.

---

CLI Usage

Ask a question

agent ask "Explain quantum tunneling"

Ask about an image

agent image cat.jpg "Describe this"

Interactive chat (stateful)

agent chat

Chat with a custom session

agent chat --session project-x

Each session stores its own memory and embeddings.

---

RAG Memory (0.2.6+)

Multimodal-Agent now includes a Retrieval-Augmented Generation (RAG) engine powered by an internal SQLite vector store.

What RAG does:

• Stores all user and assistant messages in a database
• Splits large messages into normalized chunks before embedding (0.2.7+)
• Generates embeddings for each chunk
• Retrieves the most relevant past chunks during answers
• Uses both current session and cross-session memory
• Improves contextual accuracy

Disable RAG:

agent chat --no-rag
agent ask "hello" --no-rag

---

Chunk Tokenization (0.2.7+)

Multimodal-Agent now includes a robust token-safe chunking engine to improve embedding quality and RAG retrieval.

What this adds:

• Sentence-aware splitting (split_into_chunks)
• Paragraph + sentence windowing (chunk_text)
• Safe handling of long unbroken strings
• Ensures embeddings stay within expected token limits
• More consistent similarity search results

Tokenization happens automatically whenever text is added to the RAG store.

---

History Commands (RAG-backed)

Show recent stored chunks

agent history show --limit 20

Show history for a specific session

agent history show--session project-x

Delete a specific chunk

agent history delete12

Clear the entire database

agent historyclear

Summarize all history using the LLM

agent history summary

---

Python API

Text

from multimodal_agent import MultiModalAgent

agent = MultiModalAgent()
print(agent.ask("What is recursion?"))

Image + text

from multimodal_agent import MultiModalAgent
from multimodal_agent.utils import load_image_as_part

agent = MultiModalAgent()
img = load_image_as_part("car.jpg")
response = agent.ask_with_image("What model is this?", img)
print(response)

Chunking Utilities (0.2.7+)

from multimodal_agent.tokenizer import split_into_chunks
from multimodal_agent.chunking import chunk_text

print(split_into_chunks("very long text...", max_tokens=200))
print(chunk_text("paragraphs and sentences...", max_chars=800))

---

Project Structure

multimodal-agent/
│
├── src/multimodal_agent/
│   ├── agent_core.py        # Core agent logic (RAG, chat, ask)
│   ├── rag_store.py         # SQLite vector store (chunks + embeddings)
│   ├── embedding.py         # Embedding client wrapper
│   ├── tokenizer.py         # Token-safe chunk splitting (v0.2.7)
│   ├── chunking.py          # Paragraph/sentence chunking (v0.2.7)
│   ├── cli.py               # CLI entrypoint
│   ├── utils.py             # Helpers (images, history)
│   ├── logger.py            # Logging setup
│   ├── errors.py            # Custom exceptions
│   └── VERSION
│
├── tests/                   # 90%+ coverage
└── README.md

---

Tests

pytest --cov

Coverage is enforced in CI.

---

Roadmap

• RAG Memory (0.2.6)
• Token-safe chunking (0.2.7)
• Token usage logging
• Async agent (AsyncMultiModalAgent)
• Plugin system (tools, external modules)
• Flutter extension (planned)
• IDE extensions (later)
• Streaming support