uuv-assistant-ai

Assistant AI - Image-based test generation

AI-powered assistant that helps testers and developers generate Cucumber BDD test scenarios from GUI screenshots and HTML content.

What is uuv-assistant-ai?

uuv-assistant-ai is an AI-powered service that extends the UUV ecosystem by enabling AI assisted tests. It uses Vision Language Models (VLMs) and Large Language Models (LLMs) to:

Classify images - Determine if image elements are decorative or informative (in that case it generate suitable image description)

This service integrates with the main @uuv/assistant or @uuv/assistant-desktop to provide a complete solution for E2E test generation.

Getting started

Prerequisites

Python >=3.10, <3.15

Environment Variables

Create a .env file with the following variables:

Required (for API access)

LLM_API_URL=https://localhost:11434
LLM_API_KEY=your-api-key
LLM_MODEL=ministral-3:8b

VLM_API_URL=https://localhost:11434
VLM_API_KEY=your-api-key
VLM_MODEL=ministral-3:8b

Setup with `pip`

pip install uuv-assistant-ai

# After installing from PyPI
uuv-assistant-ai

The API will be available at http://localhost:8000

Setup with `uv`

uvx add uuv-assistant-ai[mlflow]

The API will be available at http://localhost:8000

Programmatic Usage

from uuv_assistant_ai.image_classifier import (
    UUVMultipleImageDescriberAgent,
    UUVImageClassifierAgent
)
from PIL import Image

# Describe images
describer = UUVMultipleImageDescriberAgent(
    vlm_api_url="https://api.openai.com/v1",
    vlm_api_key="your-key",
    vlm_model="gpt-4-vision-preview"
)

image = Image.open("screenshot.png")
descriptions = describer(image)

# Classify images
classifier = UUVImageClassifierAgent(
    llm_api_url="https://api.openai.com/v1",
    llm_api_key="your-key",
    llm_model="gpt-4"
)

result = classifier(
    html_content="<html>...</html>",
    css_selector=".element",
    image_description="A button labeled Submit"
)

API Endpoints

1. Classify Image (Unified)

Stream image analysis results including description and classification.

curl -X POST "http://localhost:8000/api/v1/image/classify-unified" \
  -F "html_content=<html>" \
  -F "css_selector=.element-selector" \
  -F "target_img_file=@screenshot.png"

Response (Server-Sent Events):

{"image_description": "A button labeled 'Submit' with blue background"}
{"is_decorative": false, "confidence": 0.95, "analysis_details": "This is a functional button..."}

2. Multiple Image Description

Describe multiple images in a single request.

curl -X POST "http://localhost:8000/api/v1/image/multiple-describe" \
  -F "target_img_file=@screenshot.png"

Response:

{
    "descriptions": [
        { "element": "button", "description": "Submit button" },
        { "element": "input", "description": "Text input field" }
    ]
}

3. Classify Image (Standard)

Classify an image using a pre-computed description, html_content and css_selector.

curl -X POST "http://localhost:8000/api/v1/image/classify" \
  -F "html_content=<html>" \
  -F "css_selector=.element-selector" \
  -F "image_description=A button labeled Submit"

Response:

{
    "is_decorative": false,
    "confidence": 0.95,
    "analysis_details": "This is a functional button..."
}

When to use which endpoint:

Quick Start (Recommended)
Use classify-unified for a simple, one-call approach that returns both image description and classification.

Advanced / Step-by-step
For more control, use the two-step approach:

multiple-describe - Get descriptions for all UI elements in one screenshot
classify - Classify each element using the pre-computed description

This step by approach is useful when you need to:

Reuse descriptions for multiple operations
Analyze multiple elements separately
Build custom workflows with intermediate processing

Integration with UUV Ecosystem

The uuv-assistant-ai service integrates with:

@uuv/assistant - Web interface for test generation (integrates with this service)
@uuv/cypress - Cypress execution engine
@uuv/playwright - Playwright execution engine

Documentation

Full documentation: https://e2e-test-quest.github.io/uuv/

License

This project is licensed under the terms of the MIT license.

Authors

@luifr10 - Louis Fredice NJAKO MOLOM
@stanlee974 - Stanley SERVICAL

Support UUV

If you want to help UUV grow, you can fund the project directly via Open Collective. Every contribution helps us dedicate more time and energy to improving this open-source tool.

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

uuv_assistant_ai-1.12.0.tar.gz (384.6 kB view details)

Uploaded Jul 12, 2026 Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

The dropdown lists show the available interpreters, ABIs, and platforms. Enable javascript to be able to filter the list of wheel files.

uuv_assistant_ai-1.12.0-py3-none-any.whl (13.2 kB view details)

Uploaded Jul 12, 2026 Python 3

File details

Details for the file uuv_assistant_ai-1.12.0.tar.gz.

File metadata

Download URL: uuv_assistant_ai-1.12.0.tar.gz
Upload date: Jul 12, 2026
Size: 384.6 kB
Tags: Source
Uploaded using Trusted Publishing? Yes
Uploaded via: twine/6.1.0 CPython/3.13.12

File hashes

Hashes for uuv_assistant_ai-1.12.0.tar.gz
Algorithm	Hash digest
SHA256	`d13b0ca6372f7c2c41a4f3317318fe2a6ad941647df051db352dd0dd2e04d958`
MD5	`81a33e09c2dbd31d1df2b80876ccb677`
BLAKE2b-256	`a653ce9cf6761569a8ca02c7b6dc2501bc344d4c5473a2f5bca114521b74cff3`

See more details on using hashes here.

Provenance

The following attestation bundles were made for uuv_assistant_ai-1.12.0.tar.gz:

Publisher: uuv-github-ci.yml on e2e-test-quest/uuv

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Statement:
- Statement type: https://in-toto.io/Statement/v1
- Predicate type: https://docs.pypi.org/attestations/publish/v1
- Subject name: uuv_assistant_ai-1.12.0.tar.gz
- Subject digest: d13b0ca6372f7c2c41a4f3317318fe2a6ad941647df051db352dd0dd2e04d958
- Sigstore transparency entry: 2155583148
- Sigstore integration time: Jul 12, 2026
Source repository:
- Permalink: e2e-test-quest/uuv@a40755756666b2e94c0f9ad82d9f641fb3e20a62
- Branch / Tag: refs/heads/main
- Owner: https://github.com/e2e-test-quest
- Access: public
Publication detail:
- Token Issuer: https://token.actions.githubusercontent.com
- Runner Environment: github-hosted
- Publication workflow: uuv-github-ci.yml@a40755756666b2e94c0f9ad82d9f641fb3e20a62
- Trigger Event: push