robotframework-aivision 1.3.0

AI Vision library for Robot Framework

Robot Framework AI Vision Library

AI VIsion Library for Robot Framework that verifies UI/screenshots (including template “look & feel”) by sending instructions plus one or more images to an OpenAI-compatible API (Ollama, OpenAI, Perplexity, Gemini, etc.).

The main keyword (Verify That) expects the model to return a strict RESULT: / EXPLANATION: format and will fail the test if the result is not pass.

Features

• Visual assertions on one or more screenshots using natural-language instructions.
• Optional non-image attachments (txt, pdf, source, log) included as context for assertions.
• Template comparison keyword to validate “actual vs expected” look & feel (optionally creates a side-by-side image).
• Image utilities built on Pillow: open/convert, watermark, combine images, auto-generate names, save into Robot Framework output directory.
• Configurable AI system prompt for enforcing response format and verification behavior.
• Works with multiple providers via the openai Python client and OpenAI-compatible endpoints (base_url).

Keywords Documentation

Keywords documentation can be found here.

Installation

Install from PyPI (once published):

pip install -U robotframework-aivision

Runtime dependencies include Robot Framework, Pillow, and the openai Python client.

Configuration

Import the library in Robot Framework and choose a provider using platform plus optional overrides (base_url, api_key, model, image_detail, image_dpi).

Robot Framework import examples

Default (Ollama-like local setup):

*** Settings ***
Library  AIVision

OpenAI (API key required):

*** Settings ***
Library  AIVision
... platform=OpenAI
... api_key=%{OPENAI_API_KEY}
... model=gpt-5.2

Perplexity:

*** Settings ***
Library  AIVision
... platform=Perplexity
... api_key=%{PPLX_API_KEY}
... model=sonar-pro

Gemini (OpenAI-compatible endpoint):

*** Settings ***
Library  AIVision
... platform=Gemini
... api_key=%{GEMINI_API_KEY}
... model=gemini-2.5-flash

Advanced configuration

You can override the default AI system prompt that enforces the strict RESULT: / EXPLANATION: response format.

*** Settings ***
Library  AIVision
... platform=OpenAI
... api_key=%{OPENAI_API_KEY}
... system_prompt=You are a test automation assistant. Return exactly: RESULT: pass|fail and EXPLANATION: ...

Supported platforms (defaults)

The library defines these platform presets (model and base_url) which you can override via import arguments.

Platform	Default base_url	Default model	API key
Ollama	http://localhost:11434/v1	kimi-k2.5:cloud	Not required
DockerModel	http://localhost:12434/engines/v1	ai/qwen3-vl:8B-Q8_K_XL	Not required.
OpenAI	https://api.openai.com/v1	gpt-5.2	Required.
Perplexity	https://api.perplexity.ai	sonar-pro	Required.
Gemini	https://generativelanguage.googleapis.com/v1beta/openai/	gemini-2.5-flash	Required.
Manual	None	None	Required.

Keywords

All keywords below are implemented in AIVision and are available after importing the library.

Keyword	Purpose
Verify That	Send one or more screenshots and/or file attachments with instructions to the model, parse the RESULT and raise AssertionError on failure.
Verify Screenshot Matches Look And Feel Template	Compare a screenshot against a reference template with built-in instructions, optional override_instructions, optional additional_instructions, and optional combined image creation.
Open Image	Open an image (and optionally convert mode, default RGB).
Save Image	Save a PIL image to a path (defaults to RF output directory) with optional watermark.
Generate Image Name	Create a unique timestamp-based filename with prefix/extension.
Combine Images On Paths Side By Side	Combine two image files side-by-side (optionally watermark) and optionally save.
Combine Images Side By Side	Combine two in-memory PIL images side-by-side (optionally watermark).
Add Watermark To Image	Add watermark text using the included font file.

Usage examples

Simple visual assertion

*** Settings ***
Library  AIVision  platform=Ollama

*** Test Cases ***
Login button is correct
   Verify That  ${CURDIR}/screens/login.png  Login button is visible and labeled as 'Sign In'

Visual assertion with attachments

*** Settings ***
Library  AIVision

*** Test Cases ***
UI matches log output
   @{files}  =  Create List  ${CURDIR}/screens/home.png  ${CURDIR}/logs/ui.log
   Verify That  ${files}  The log mentions the same banner text shown on the page.

Note: For PDF attachments, the library will extract text using PyMuPDF (fitz). If no text is found and a vision model is in use, it will try to render PDF pages as images using PyMuPDF (fitz); otherwise the PDF is included as base64 text. All other non-image attachments are read as text by default (including unknown extensions).

Compare screenshot to design template

*** Settings ***
Library  AIVision

*** Test Cases ***
Home page matches template
   Verify Screenshot Matches Look And Feel Template
   ...  ${CURDIR}/screens/home_actual.png
   ...  ${CURDIR}/templates/home_expected.png

Override template instructions

*** Settings ***
Library  AIVision

*** Test Cases ***
Home page matches template - custom rules
   Verify Screenshot Matches Look And Feel Template
   ...  ${CURDIR}/screens/home_actual.png
   ...  ${CURDIR}/templates/home_expected.png
   ...  override_instructions=Verify layout, spacing, typography, and brand colors match the template exactly.

Add additional template instructions

*** Settings ***
Library  AIVision

*** Test Cases ***
Home page matches template - ignore dynamic parts
   Verify Screenshot Matches Look And Feel Template
   ...  ${CURDIR}/screens/home_actual.png
   ...  ${CURDIR}/templates/home_expected.png
   ...  additional_instructions=Ignore the live clock and rotating promotional banner.

additional_instructions is appended to the default template checks (or to override_instructions if you provide both).

Version history

• 1.3.0, 2026-05-25 -- Look and Feel layout comparison assertion strategy improved and made more robust
• 1.2.0, 2026-02-24 -- Image size optimizations, additional_instructions can now be passed to look and feel check keyword
• 1.1.2, 2026-02-09 -- Improved debug logging
• 1.1.1, 2026-02-09 -- Pass/Fail AI response optimized
• 1.1.0, 2026-02-08 -- Added support for (text) files attachments
• 1.0.0, 2026-02-02 -- Release v1.0
• 0.2.0a1, 2026-02-02 -- Alpha1 version
• 0.2.0, 2026-01-29 -- AI System Prompt is configurable
• 0.1.0, 2025-12-19 -- Additional GenAI Providers added
• 0.0.1, 2024-05-11 -- Initial version