vtai 0.7.5

VT.ai - Minimal multimodal AI chat app with dynamic conversation routing

VT

Multimodal AI Chat App with Dynamic Routing

VT.ai

VT.ai is a multimodal AI chat application designed to simplify interaction with different AI models through a unified interface. It employs vector-based semantic routing to direct queries to the most suitable model, eliminating the need to switch between multiple applications and interfaces.

Documentation

Key Features

• Multi-Provider Integration: Unified access to models from OpenAI (o1/o3/4o), Anthropic (Claude), Google (Gemini), DeepSeek, Llama, Cohere, and local models via Ollama
• Semantic Routing System: Vector-based classification automatically routes queries to appropriate models using FastEmbed embeddings, removing the need for manual model selection
• Multimodal Capabilities: Comprehensive support for text, image, and audio inputs with advanced vision analysis
• Image Generation: GPT-Image-1 integration with support for transparent backgrounds, multiple formats, and customizable quality parameters
• Web Search Integration: Real-time information retrieval with source attribution via Tavily API
• Voice Processing: Advanced speech-to-text and text-to-speech functionality with configurable voice options and silence detection
• Reasoning Visualization: Step-by-step model reasoning visualization with the <think> tag for transparent AI decision processes

Installation & Setup

Multiple installation methods are available depending on requirements:

# Standard PyPI installation
uv pip install vtai

# Zero-installation experience with uvx
export OPENAI_API_KEY='your-key-here'
uvx vtai

# Development installation
git clone https://github.com/vinhnx/VT.ai.git
cd VT.ai
uv venv
source .venv/bin/activate  # Linux/Mac
uv pip install -e ".[dev]"  # Install with development dependencies

API Key Configuration

Configure API keys to enable specific model capabilities:

# Command-line configuration
vtai --api-key openai=sk-your-key-here

# Environment variable configuration
export OPENAI_API_KEY='sk-your-key-here'  # For OpenAI models
export ANTHROPIC_API_KEY='sk-ant-your-key-here'  # For Claude models
export GEMINI_API_KEY='your-key-here'  # For Gemini models

API keys are securely stored in ~/.config/vtai/.env for future use.

Usage Guide

Interface Usage

The application provides a clean, intuitive interface with the following capabilities:

1. Dynamic Conversations: The semantic router automatically selects the most appropriate model for each query
2. Image Generation: Create images using prompts like "generate an image of..." or "draw a..."
3. Visual Analysis: Upload or provide URLs to analyze visual content
4. Reasoning Visualization: Add <think> to prompts to observe step-by-step reasoning
5. Voice Interaction: Use the microphone feature for speech input and text-to-speech output

Detailed usage instructions are available in the Getting Started Guide.

Documentation

The documentation is organized into sections designed for different user needs:

• User Guide: Installation, configuration, and feature documentation
• Developer Guide: Architecture details, extension points, and implementation information
• API Reference: Comprehensive API documentation for programmatic usage

Implementation Options

VT.ai offers two distinct implementations:

• Python Implementation: Full-featured reference implementation with complete support for all capabilities
• Rust Implementation: High-performance alternative with optimized memory usage and native compiled speed

The implementation documentation provides a detailed comparison of both options.

Supported Models

Category	Models
Chat	GPT-o1, GPT-o3 Mini, GPT-4o, Claude 3.5/3.7, Gemini 2.0/2.5
Vision	GPT-4o, Gemini 1.5 Pro/Flash, Claude 3, Llama3.2 Vision
Image Gen	GPT-Image-1 with custom parameters
TTS	GPT-4o mini TTS, TTS-1, TTS-1-HD
Local	Llama3, Mistral, DeepSeek R1 (1.5B to 70B via Ollama)

The Models Documentation provides detailed information about model-specific capabilities and configuration options.

Technical Architecture

VT.ai leverages several open-source projects to deliver its functionality:

• Chainlit: Modern chat interface framework
• LiteLLM: Unified model abstraction layer
• SemanticRouter: Intent classification system
• FastEmbed: Efficient embedding generation
• Tavily: Web search capabilities

The application architecture follows a clean, modular design:

• Entry Point: vtai/app.py - Main application logic
• Routing Layer: vtai/router/ - Semantic classification system
• Assistants: vtai/assistants/ - Specialized handlers for different query types
• Tools: vtai/tools/ - Web search, file operations, and other integrations

Contributing

Contributions to VT.ai are welcome. The project accepts various types of contributions:

• Bug Reports: Submit detailed GitHub issues for any bugs encountered
• Feature Requests: Propose new functionality through GitHub issues
• Pull Requests: Submit code improvements and bug fixes
• Documentation: Enhance documentation or add examples
• Feedback: Share user experiences to help improve the project

Development Setup

Follow these steps to set up a local development environment:

Prerequisites

• Python 3.11 or higher
• uv package manager (recommended) or pip
• Git

Step-by-step Setup

1. Clone the repository

   git clone https://github.com/vinhnx/VT.ai.git
   cd VT.ai
   

2. Set up Python virtual environment

   # Using uv (recommended)
   uv venv
   
   # Or using standard Python venv
   python -m venv .venv
   

3. Activate the virtual environment

   # On macOS/Linux
   source .venv/bin/activate
   
   # On Windows
   .venv\Scripts\activate
   

4. Install development dependencies

   # Install the package in editable mode with development dependencies
   uv pip install -e ".[dev]"
   
   # Or if using pip
   pip install -e ".[dev]"
   

5. Set up environment variables Copy the .env.example file to create your own .env file:

   cp .env.example .env
   

   Then edit .env to add your API keys:

   # Edit the .env file with your preferred editor
   nano .env
   # or
   code .env  # if using VS Code
   

6. Run the application in development mode

   # Using chainlit (recommended for development)
   chainlit run vtai/app
   
   # The application will be available at http://localhost:8000
   

7. Run tests to verify your setup

   # Run all tests
   pytest
   
   # Run tests with coverage
   pytest --cov=vtai
   
   # Run specific test categories
   pytest tests/unit/
   pytest tests/integration/
   

All-in-One Development Run

For convenience, you can use the run_app.py script which provides an all-in-one solution for running the application:

# Direct Python execution (will initialize but not start server)
python run_app.py

# With chainlit for full interactive development
chainlit run run_app.py -w

# Or simply use the main app module
chainlit run vtai/app

The run_app.py script serves as a wrapper that:

• Sets up the proper Python path
• Initializes the application with all necessary components
• Provides a single entry point for development
• Handles environment setup automatically

Alternative Development Setup (using pip)

If you prefer to use pip instead of uv:

# Clone and navigate to the project
git clone https://github.com/vinhnx/VT.ai.git
cd VT.ai

# Create and activate virtual environment
python -m venv .venv
source .venv/bin/activate  # On macOS/Linux
# .venv\Scripts\activate  # On Windows

# Install in development mode
pip install -e ".[dev]"

# Run the application
chainlit run vtai/app

Development Commands

Common commands you'll use during development:

• chainlit run vtai/app - Run the development server
• pytest - Run all tests
• pytest -x - Run tests and stop on first failure
• pytest --cov=vtai - Run tests with coverage report
• ruff check . - Check code for linting issues
• ruff format . - Format code according to project standards
• uv pip install -e ".[dev]" - Reinstall after dependency changes

Troubleshooting Common Issues

Issue: ModuleNotFoundError when running the application

• Solution: Make sure you've activated your virtual environment and installed the package in editable mode:

  source .venv/bin/activate
  pip install -e .
  

Issue: Permission denied when creating virtual environment

• Solution: Make sure you have write permissions in the project directory:

  chmod 755 .
  uv venv
  

Issue: Chainlit not found

• Solution: Install chainlit separately or make sure you've installed all dependencies:

  pip install chainlit
  # or
  pip install -e ".[dev]"
  

Issue: API keys not being recognized

• Solution: Verify your .env file is in the correct location and contains properly formatted keys:

  # Check if .env file exists in project root
  ls -la .env
  
  # Verify content format
  cat .env
  # Should contain: OPENAI_API_KEY=your_actual_key_here
  

Issue: Application fails to start with port binding errors

• Solution: Check if another process is using the default port:

  # Find processes using port 8000
  lsof -ti:8000
  # Kill the process if needed
  kill $(lsof -ti:8000)
  

Issue: Slow startup times

• Solution: You can enable fast startup mode by setting an environment variable:

  export VT_FAST_START=1
  chainlit run vtai/app
  

Issue: Dependency conflicts

• Solution: Create a fresh virtual environment and reinstall dependencies:

  rm -rf .venv
  python -m venv .venv
  source .venv/bin/activate
  pip install -e ".[dev]"
  

Testing and Quality

Quality is maintained through comprehensive testing:

# Run the test suite
pytest

# Run with coverage reporting
pytest --cov=vtai

# Run specific test categories
pytest tests/unit/
pytest tests/integration/

License

VT.ai is available under the MIT License - See LICENSE for details.