UUV Assistant AI based, used to improve the life of testers and developers by generating cucumber phrases from the GUI.
Verified details
These details have been verified by PyPIProject links
GitHub Statistics
Maintainers
Project description
uuv-assistant-ai
Assistant AI - Image-based test generation
AI-powered assistant that helps testers and developers generate Cucumber BDD test scenarios from GUI screenshots and HTML content.
What is uuv-assistant-ai?
uuv-assistant-ai is an AI-powered service that extends the UUV ecosystem by enabling AI assisted tests. It uses Vision Language Models (VLMs) and Large Language Models (LLMs) to:
- Classify images - Determine if image elements are decorative or informative (in that case it generate suitable image description)
This service integrates with the main @uuv/assistant or @uuv/assistant-desktop to provide a complete solution for E2E test generation.
Getting started
Prerequisites
- Python >=3.10, <3.15
Environment Variables
Create a .env file with the following variables:
Required (for API access)
LLM_API_URL=https://localhost:11434
LLM_API_KEY=your-api-key
LLM_MODEL=ministral-3:8b
VLM_API_URL=https://localhost:11434
VLM_API_KEY=your-api-key
VLM_MODEL=ministral-3:8b
Setup with pip
pip install uuv-assistant-ai
# After installing from PyPI
uuv-assistant-ai
The API will be available at http://localhost:8000
Setup with uv
uvx add uuv-assistant-ai[mlflow]
The API will be available at http://localhost:8000
Programmatic Usage
from uuv_assistant_ai.image_classifier import (
UUVMultipleImageDescriberAgent,
UUVImageClassifierAgent
)
from PIL import Image
# Describe images
describer = UUVMultipleImageDescriberAgent(
vlm_api_url="https://api.openai.com/v1",
vlm_api_key="your-key",
vlm_model="gpt-4-vision-preview"
)
image = Image.open("screenshot.png")
descriptions = describer(image)
# Classify images
classifier = UUVImageClassifierAgent(
llm_api_url="https://api.openai.com/v1",
llm_api_key="your-key",
llm_model="gpt-4"
)
result = classifier(
html_content="<html>...</html>",
css_selector=".element",
image_description="A button labeled Submit"
)
API Endpoints
1. Classify Image (Unified)
Stream image analysis results including description and classification.
curl -X POST "http://localhost:8000/api/v1/image/classify-unified" \
-F "html_content=<html>" \
-F "css_selector=.element-selector" \
-F "target_img_file=@screenshot.png"
Response (Server-Sent Events):
{"image_description": "A button labeled 'Submit' with blue background"}
{"is_decorative": false, "confidence": 0.95, "analysis_details": "This is a functional button..."}
2. Multiple Image Description
Describe multiple images in a single request.
curl -X POST "http://localhost:8000/api/v1/image/multiple-describe" \
-F "target_img_file=@screenshot.png"
Response:
{
"descriptions": [
{ "element": "button", "description": "Submit button" },
{ "element": "input", "description": "Text input field" }
]
}
3. Classify Image (Standard)
Classify an image using a pre-computed description, html_content and css_selector.
curl -X POST "http://localhost:8000/api/v1/image/classify" \
-F "html_content=<html>" \
-F "css_selector=.element-selector" \
-F "image_description=A button labeled Submit"
Response:
{
"is_decorative": false,
"confidence": 0.95,
"analysis_details": "This is a functional button..."
}
When to use which endpoint:
Quick Start (Recommended)
Use classify-unified for a simple, one-call approach that returns both image description and classification.
Advanced / Step-by-step
For more control, use the two-step approach:
multiple-describe- Get descriptions for all UI elements in one screenshotclassify- Classify each element using the pre-computed description
This step by approach is useful when you need to:
- Reuse descriptions for multiple operations
- Analyze multiple elements separately
- Build custom workflows with intermediate processing
Integration with UUV Ecosystem
The uuv-assistant-ai service integrates with:
@uuv/assistant- Web interface for test generation (integrates with this service)@uuv/cypress- Cypress execution engine@uuv/playwright- Playwright execution engine
Documentation
Full documentation: https://e2e-test-quest.github.io/uuv/
License
This project is licensed under the terms of the MIT license.
Authors
@luifr10- Louis Fredice NJAKO MOLOM@stanlee974- Stanley SERVICAL
Support UUV
If you want to help UUV grow, you can fund the project directly via Open Collective. Every contribution helps us dedicate more time and energy to improving this open-source tool.
Project details
Verified details
These details have been verified by PyPIProject links
GitHub Statistics
Maintainers
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file uuv_assistant_ai-1.9.1.tar.gz.
File metadata
- Download URL: uuv_assistant_ai-1.9.1.tar.gz
- Upload date:
- Size: 379.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6f17255af82384ccd733ffb0f5c41cb5b6f08e92e0c9cc59eaf42c23848f75e2
|
|
| MD5 |
2f9d9586c6d24b63d5c12c9433d11b8b
|
|
| BLAKE2b-256 |
29149774a1b07161409d00e76f18d66feec4cd92605ddc24842d98bd05b7d245
|
Provenance
The following attestation bundles were made for uuv_assistant_ai-1.9.1.tar.gz:
Publisher:
uuv-github-ci.yml on e2e-test-quest/uuv
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
uuv_assistant_ai-1.9.1.tar.gz -
Subject digest:
6f17255af82384ccd733ffb0f5c41cb5b6f08e92e0c9cc59eaf42c23848f75e2 - Sigstore transparency entry: 1640647847
- Sigstore integration time:
-
Permalink:
e2e-test-quest/uuv@72b30e94c50767b9780db1b53a49ddb32427f0da -
Branch / Tag:
refs/heads/main - Owner: https://github.com/e2e-test-quest
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
uuv-github-ci.yml@72b30e94c50767b9780db1b53a49ddb32427f0da -
Trigger Event:
push
-
Statement type:
File details
Details for the file uuv_assistant_ai-1.9.1-py3-none-any.whl.
File metadata
- Download URL: uuv_assistant_ai-1.9.1-py3-none-any.whl
- Upload date:
- Size: 13.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.13.12
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2b9de933d6f94fb9877b40500d67163102d492ea1de7b17de873ab46e35543bc
|
|
| MD5 |
01ab267fd66e5009c1ca9afdb7211bba
|
|
| BLAKE2b-256 |
4f6ccf8aaab6a5b6507e4787a1d165ba13d2532bdc3046f75a94d83bf66123a1
|
Provenance
The following attestation bundles were made for uuv_assistant_ai-1.9.1-py3-none-any.whl:
Publisher:
uuv-github-ci.yml on e2e-test-quest/uuv
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
uuv_assistant_ai-1.9.1-py3-none-any.whl -
Subject digest:
2b9de933d6f94fb9877b40500d67163102d492ea1de7b17de873ab46e35543bc - Sigstore transparency entry: 1640647978
- Sigstore integration time:
-
Permalink:
e2e-test-quest/uuv@72b30e94c50767b9780db1b53a49ddb32427f0da -
Branch / Tag:
refs/heads/main - Owner: https://github.com/e2e-test-quest
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
uuv-github-ci.yml@72b30e94c50767b9780db1b53a49ddb32427f0da -
Trigger Event:
push
-
Statement type:
