testteller 0.1.1

TestTeller : A versatile RAG AI agent for generating test cases from project documentation (PRDs, Contracts, Design Docs, etc.) and project code, leveraging LLMs.

TestTeller RAG Agent

TestTeller RAG Agent TestTeller RAG Agent is a versatile CLI-based RAG (Retrieval Augmented Generation) agent designed to generate software test cases. It leverages Google's Gemini LLM and ChromaDB as a vector store. The agent can process various input sources, including PRD documentation, API contracts, technical design documents (HLD/LLD), and code from GitHub repositories or local folders.

🚀 Quick Start

1. Install the Package

# Install from PyPI
pip install testteller

# Or clone and install locally
git clone https://github.com/yourusername/testteller-rag-agent.git
cd testteller-rag-agent
pip install -e .

2. Configure the Agent

# Run the configuration wizard
testteller configure

# Or set environment variables manually
export GOOGLE_API_KEY="your_gemini_api_key"
export GITHUB_TOKEN="your_github_token"  # Optional, for private repos

3. Start ChromaDB

# Using Docker (Optional)
docker run -d -p 8000:8000 chromadb/chroma:0.4.15

4. Generate Test Cases

# Ingest code or documentation
testteller ingest-code https://github.com/owner/repo.git

# Generate tests
testteller generate "Create API integration tests for user authentication"

✨ Features

🔄 Intelligent Test Generation

• Multiple Test Types
  • Technical test cases for components, APIs, and system architecture
  • User journey test cases for end-to-end flows
  • Integration test scenarios
  • Edge case and error handling tests
  • Performance test considerations

📚 Document Processing

• Multi-Format Support
  • PDF documents (.pdf)
  • Word documents (.docx)
  • Excel spreadsheets (.xlsx)
  • Markdown files (.md)
  • Text files (.txt)
  • Source code files (multiple languages)

💻 Code Analysis

• Repository Integration
  • GitHub repository cloning (public and private)
  • Local codebase analysis
  • Multiple programming language support

🧠 Advanced RAG Pipeline

• State-of-the-Art LLM Integration
  • Google Gemini 2.0 Flash for fast generation
  • Optimized embeddings using text-embedding-004
  • Context-aware prompt engineering
  • Streaming response support

📊 Output Management

• Flexible Output Formats
  • Markdown documentation
  • Structured test cases

📋 Prerequisites

• Python 3.11 or higher (Required)
• Docker and Docker Compose (for containerized deployment)
• Google Gemini API key (Get it here)
• (Optional) GitHub Personal Access Token for private repos

🛠️ Installation

Option 1: Install from PyPI

# Install the package
pip install testteller

# Set environment variables
export GOOGLE_API_KEY="your_gemini_api_key"
export GITHUB_TOKEN="your_github_token"  # Optional, for private repos

# Start ChromaDB (in a separate terminal) (Optional)
docker run -d -p 8000:8000 chromadb/chroma:0.4.15

Option 2: Docker Installation

1. Clone the repository:

git clone https://github.com/yourusername/testteller-rag-agent.git
cd testteller-rag-agent

2. Create environment file:

cat > .env << EOL
GOOGLE_API_KEY=your_gemini_api_key
# Only set GITHUB_TOKEN if you need to access private repos
# GITHUB_TOKEN=your_github_token
LOG_LEVEL=INFO
LOG_FORMAT=json
DEFAULT_COLLECTION_NAME=my_test_collection
EOL

3. Start services:

docker-compose up -d

📖 Available Commands

Using pip installation (recommended)

Configuration

# Run interactive configuration wizard
testteller configure

# Show all available commands
testteller --help

# Show help for specific command
testteller generate --help

Ingest Documentation & Code

# Ingest a single document or directory
testteller ingest-docs path/to/document.pdf --collection-name my_collection

# Ingest a directory of documents
testteller ingest-docs path/to/docs/directory --collection-name my_collection

# Ingest code from GitHub repository or local folder
testteller ingest-code https://github.com/owner/repo.git --collection-name my_collection

# Ingest code with custom collection name
testteller ingest-code ./local/code/folder --collection-name my_collection

Generate Test Cases

# Generate tests with default settings
testteller generate "Create API integration tests for user authentication"

# Generate tests with custom output file
testteller generate "Create technical tests for login flow" --output-file tests.md

# Generate tests with specific collection and number of retrieved docs
testteller generate "Create end-to-end tests" --collection-name my_collection --num-retrieved 10 --output-file ./tests.md

Manage Data

# Check collection status
testteller status --collection-name my_collection

# Clear collection data
testteller clear-data --collection-name my_collection --force

Using Docker or Local Development

When using Docker or running from source, use the module format:

# Format for Docker:
docker-compose exec app python -m testteller.main [command]

# Format for local development:
python -m testteller.main [command]

First, ensure your environment variables are set in the .env file:

# Create .env file with required variables
cat > .env << EOL
GOOGLE_API_KEY=your_gemini_api_key
# Only set GITHUB_TOKEN if you need to access private repos
# GITHUB_TOKEN=your_github_token
LOG_LEVEL=INFO
LOG_FORMAT=json
DEFAULT_COLLECTION_NAME=my_test_collection
EOL

Note: Only set GITHUB_TOKEN if you actually need to access private repositories. If you don't need it, comment it out or remove it entirely. Setting it to an empty value will cause validation errors.

Then use the commands:

# Get help
docker-compose exec app python -m testteller.main --help

# Get command-specific help
docker-compose exec app python -m testteller.main generate --help

# Run configuration wizard
docker-compose exec app python -m testteller.main configure

# Ingest documentation
docker-compose exec app python -m testteller.main ingest-docs /path/to/doc.pdf --collection-name my_collection

# Generate tests
docker-compose exec app python -m testteller.main generate "Create API tests" --collection-name my_collection --output-file tests.md

# Check status
docker-compose exec app python -m testteller.main status --collection-name my_collection

# Clear data
docker-compose exec app python -m testteller.main clear-data --collection-name my_collection --force

Note: Make sure both the app and ChromaDB containers are healthy before running commands:

# Check container status
docker-compose ps

# Check container logs if needed
docker-compose logs -f

⚙️ Configuration

Environment Variables

Variable	Description	Default	Required	Notes
GOOGLE_API_KEY	Google Gemini API key	-	Yes	Must be valid API key
GITHUB_TOKEN	GitHub Personal Access Token	-	No	Don't set if not using private repos
LOG_LEVEL	Logging level	INFO	No	DEBUG, INFO, WARNING, ERROR
LOG_FORMAT	Logging format	text	No	text or json
GEMINI_GENERATION_MODEL	Gemini model for generation	gemini-2.0-flash	No
GEMINI_EMBEDDING_MODEL	Model for embeddings	text-embedding-004	No
CHUNK_SIZE	Document chunk size	1000	No
CHUNK_OVERLAP	Chunk overlap size	200	No
API_RETRY_ATTEMPTS	Number of API retry attempts	3	No
API_RETRY_WAIT_SECONDS	Wait time between retries	2	No	Seconds

🔧 Troubleshooting

Common Issues

1. Container Health Check Failures

# Check container logs
docker-compose logs -f

# Restart services
docker-compose restart

2. ChromaDB Connection Issues

# Verify ChromaDB is running
curl http://localhost:8000/api/v1/heartbeat

# Check ChromaDB logs
docker-compose logs chromadb

3. Permission Issues

# Fix volume permissions
sudo chown -R 1000:1000 ./chroma_data
sudo chmod -R 777 ./temp_cloned_repos

📝 License

This project is licensed under the MIT License - see the LICENSE file for details.