testbench-ai-service 1.0.0

REST API service supporting multiple AI agents for imbus TestBench

TestBench AI Service

An asynchronous REST API service that connects imbus TestBench to external LLM providers to support AI-driven Agents during test design and execution.

Features

• Multiple Agents: test case set reviews, description generation, and defect explanations, each configurable independently
• Pluggable LLM providers: ships with OpenAI, Azure OpenAI, and Anthropic support; implement a custom LLMClient to bring your own
• Automatic provider routing: model names starting with gpt-* and o-series (o1, o3, o4-mini, …) are routed to OpenAI, claude-* to Anthropic — regardless of the global provider setting
• Configurable prompts: YAML-based templates with Jinja2 variables and per-project overrides
• Per-project configuration: language, LLM provider, prompt variant, and enabled agents can be overridden per TestBench project
• Localization: built-in English and German translations for AI-generated output
• JWT authentication: every request is validated against the TestBench REST API using a JWT token; no separate credential management
• Async processing: Agents run as background tasks so the API responds immediately
• Swagger UI: interactive API docs at /docs
• SSL/TLS & reverse proxy support: optional HTTPS with mTLS and trusted-proxy headers

Installation

With pip (Python 3.10–3.14 required):

pip install testbench-ai-service

Standalone executable (no Python required): download the pre-built binary from the GitHub releases page.

Quick Start

1. Set your LLM API key

Create a .env file in the installation directory with the key for your chosen provider:

# .env
# OpenAI
OPENAI_API_KEY=your_openai_api_key

# Azure OpenAI
# AZURE_OPENAI_API_KEY=your_azure_openai_api_key

# Anthropic
# ANTHROPIC_API_KEY=your_anthropic_api_key

2. Initialize configuration

testbench-ai-service init

Open config.toml and point tb_server_url at your TestBench REST API:

[testbench-ai-service]
tb_server_url = "https://localhost:9443/api/"

[testbench-ai-service.llm_config]
# openai (default), azure_openai, anthropic, or custom
provider = "openai"

3. Start the service

testbench-ai-service start

The service runs at http://127.0.0.1:8010 by default. Open /docs for the interactive Swagger UI.

Documentation

Full documentation is available in the docs/ folder of the repository:

• Introduction
• Installation
• Quickstart
• Configuration
• Agents
• Prompts
• TestBench Integration
• CLI Reference

Contributing

Contributions are welcome. See CONTRIBUTING.md for setup instructions and guidelines.

Changelog

See CHANGELOG.md for release history.

License

Apache 2.0. See LICENSE for details.