Flexible Voice RAG Agent

A reusable, domain-neutral LangChain agent core extracted into a separate project. It supports custom system/user prompts, multiple RAG documents, custom tools, custom models, embeddings, and vector stores. The original application outside this directory is not imported or modified.

Install

Core only:

pip install flexible-voice-rag-agent

With the default Gemini embeddings and Chroma implementation:

pip install "flexible-voice-rag-agent[google-rag]"

During local development:

python -m venv .venv
. .venv/bin/activate
python -m pip install -e ".[google-rag,dev]"

Quick start

import asyncio

from flexible_voice_agent import FlexibleAgent, VoiceAgentConfig

config = VoiceAgentConfig.from_file("examples/customer_support/config.toml")
agent = FlexibleAgent(config)

async def main():
    answer = await agent.ainvoke(
        "How do I return an item?",
        thread_id="conversation-123",
    )
    print(answer)

asyncio.run(main())

For a TTS pipeline, stream text chunks:

async for text in agent.astream_text(user_text, thread_id=session_id):
    await tts.send_text(text)

Customize a domain

Each deployment has a TOML or JSON profile. Paths are resolved relative to the profile file, so a profile and its documents can move together.

domain = "customer_support"

[agent]
model = "google_genai:gemini-3.1-flash-lite"
language_instruction = "Reply in the user's language. Keep responses concise."

[prompt]
system_path = "prompts/system.txt"
user_path = "prompts/user.txt"

[prompt.variables]
company_name = "Example Shop"
tone = "warm and direct"

[rag]
enabled = true
document_paths = ["documents", "../shared/legal.md"]
persist_directory = ".chroma/customer_support"
collection_name = "customer_support_docs"
embedding_model = "gemini-embedding-001"
top_k = 4
relevance_threshold = 0.55
chunk_size = 900
chunk_overlap = 120

Prompt files use Python-style placeholders. The package supplies {user_input}, {rag_context}, {domain}, and {language_instruction}. Any value in [prompt.variables] is also available:

You are the {company_name} assistant for the {domain} domain.
Tone: {tone}.
{language_instruction}
Only use approved context for company-specific facts.

<approved_context>
{rag_context}
</approved_context>

<user_message>
{user_input}
</user_message>

document_paths accepts individual UTF-8 .md/.txt files or directories. Directories are scanned recursively. Document content and paths are hashed, so the persistent collection is rebuilt when the configured knowledge base changes.

Custom tools and RAG backends

Pass any LangChain-compatible tools without changing the package:

agent = FlexibleAgent(config, tools=[create_ticket, lookup_order])

The defaults use Gemini embeddings and Chroma. Other providers can be injected:

retriever = RAGRetriever(
    config.rag,
    embeddings=my_embeddings,
    vector_store_factory=my_vector_store_factory,
)
agent = FlexibleAgent(config, retriever=retriever, model=my_chat_model)

The retriever is also available as a tool when agent-controlled retrieval is preferred:

rag_tool = retriever.as_tool(
    name="search_product_docs",
    description="Search approved product documentation.",
)

Validate, test, and build

flexible-voice-agent validate examples/customer_support/config.toml
pytest
python -m build
python -m twine check dist/*

Publish to PyPI

Before publishing, replace the placeholder author in pyproject.toml, add your real project URLs if available, confirm the distribution name is available, and increment the version for every release.

TestPyPI first:

python -m twine upload --repository testpypi dist/*

Then publish the same checked artifacts to PyPI:

python -m twine upload dist/*

Use a PyPI API token when Twine prompts for credentials. Do not store the token in this repository.