Mobile UI Comparator

Compare two mobile screenshots and get a structured report of UI regressions — layout shifts, spacing changes, missing elements, and text differences — with numbered issues, severity ratings, and fix hints.

---

Contents

• Installation
• Quick start
• CLI usage
• HTTP API usage
• Single-image audit
• Docker
• Output files
• CLI reference
• Integrating in your application
  • 1 — Install from PyPI
  • 2 — Use as a Python library
  • 3 — Run the server and call via HTTP
  • 4 — CI/CD integration (GitHub Actions)
  • 5 — Deploy with Docker (production)
  • 6 — Minimum requirements

---

Installation

git clone <repo>
cd mobile-ui-comparator
pip install -e ".[server]"

---

Quick start

CLI — compare two screenshots

compare-ui screen_a.png screen_b.png \
  --device-a "Build 123" \
  --device-b "Baseline" \
  --screen "Home Screen"

API — start the server, then compare

uvicorn mobile_ui_comparator.server:app --host 0.0.0.0 --port 8765

curl -s -X POST http://localhost:8765/compare \
  -H "Content-Type: application/json" \
  -d '{"image_a": "https://…/a.png", "image_b": "https://…/b.png"}'

---

CLI usage

Local files

compare-ui screen_a.png screen_b.png

HTTP image URLs

compare-ui \
  "https://your-cdn.com/screenshots/build-123/home.png" \
  "https://your-cdn.com/screenshots/baseline/home.png" \
  --device-a "Build 123" \
  --device-b "Baseline" \
  --screen "Home"

Exclude a region

Use this when part of the screen always changes (e.g. a banner ad, a live clock).
Coordinates are x,y,width,height — repeatable.

compare-ui a.png b.png --ignore-region 0,200,1080,150

Compare the same screen in different states

When the two images are the same screen but in different states (logged-in vs guest, light vs dark mode), skip the same-screen gate:

compare-ui a.png b.png --skip-phash

Example terminal output

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  MOBILE UI COMPARATOR v0.1.0
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
  STATUS: FAIL  |  Critical: 3  Major: 0  Minor: 0  |  Canvas diff: 39.8%

  #1 — Bottom Action Bar — element repositioned
       Fix: Check bottom bar position, container height, and padding

  #2 — Profile / Hero Section — spacing/padding changed
       Fix: Check profile card spacing, margins, padding

  📸  ui-diff-reports/20261001_091843_d4a7e2ce/diff_annotated.png
  🌡️   ui-diff-reports/20261001_091843_d4a7e2ce/heatmap.png
  📄  ui-diff-reports/20261001_091843_d4a7e2ce/report.json
  🌐  ui-diff-reports/20261001_091843_d4a7e2ce/report.html
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

---

HTTP API usage

Start the server

uvicorn mobile_ui_comparator.server:app --host 0.0.0.0 --port 8765

Open http://localhost:8765/docs for the interactive Swagger UI.

---

How the API works

POST /compare returns a job_id in under 50 ms — the comparison runs in a background worker.
Poll GET /jobs/{job_id} every 2–3 seconds until status is "done" or "failed".

POST /compare  →  { job_id, status: "queued", poll_url }   ← returns in ~23 ms
GET  /jobs/{id}  →  { status: "processing" }               ← check again
GET  /jobs/{id}  →  { status: "done", result: { … } }      ← result ready in ~5 s

---

POST /compare — enqueue a comparison

curl -s -X POST http://localhost:8765/compare \
  -H "Content-Type: application/json" \
  -d '{
    "image_a": "https://your-cdn.com/screenshots/build-123/home.png",
    "image_b": "https://your-cdn.com/screenshots/baseline/home.png",
    "device_a": "Build 123",
    "device_b": "Baseline",
    "screen": "Home Screen"
  }'

Request fields

Field	Type	Default	Description
image_a	string	required	HTTP URL or local path
image_b	string	required	HTTP URL or local path
device_a	string	"Device A"	Label for the reference image
device_b	string	"Device B"	Label for the candidate image
screen	string	"Unknown"	Screen name shown in the report
threshold	float	0.85	SSIM pass threshold (0–1)
no_mask_status_bar	bool	false	Include the OS status bar in analysis
ignore_regions	[[x,y,w,h], …]	[]	Rectangles to exclude from analysis
include_html	bool	false	Embed the HTML report as base64 in the result
skip_phash	bool	false	Skip the same-screen gate

Response

{
  "job_id":      "d171918ff0d04e87b10a7517fe2a7721",
  "status":      "queued",
  "queue_depth": 0,
  "workers":     2,
  "poll_url":    "/jobs/d171918ff0d04e87b10a7517fe2a7721"
}

---

GET /jobs/{job_id} — poll for result

curl http://localhost:8765/jobs/d171918ff0d04e87b10a7517fe2a7721 | python3 -m json.tool

While processing

{
  "job_id": "d171918ff0d04e87b10a7517fe2a7721",
  "status": "processing",
  "queued_at": 1790759923.5,
  "started_at": 1790759923.6
}

When done

{
  "job_id":    "d171918ff0d04e87b10a7517fe2a7721",
  "status":    "done",
  "wait_s":    0.01,
  "process_s": 5.44,
  "result": {
    "meta": {
      "transid":   "20261001_091843_d4a7e2ce",
      "device_a":  "Build 123",
      "device_b":  "Baseline",
      "screen":    "Home Screen",
      "timestamp": "2026-10-01T09:18:48+00:00"
    },
    "scores": {
      "overall_ssim":    0.6632,
      "canvas_diff_pct": 39.82,
      "status":          "FAIL"
    },
    "summary": {
      "ui_issues": 3,
      "critical":  3,
      "major":     0,
      "minor":     0
    },
    "issues": [
      {
        "id":          3,
        "priority":    "P1",
        "severity":    "CRITICAL",
        "category":    "POSITION",
        "zone":        "Bottom Action Bar",
        "description": "#3 — Bottom Action Bar — element repositioned (1080×437px region at x=0, y=1672)",
        "fix_hint":    "Check bottom bar position, container height, and padding"
      }
    ],
    "ocr_analysis": {
      "status": "FAIL",
      "issues": [ "…17 text issues…" ]
    },
    "zone_summary": {
      "Bottom Action Bar": {
        "priority": "P1",
        "fix_hint": "Check bottom bar position, container height, and padding",
        "issues":   [3]
      }
    },
    "report_url":      "/reports/20261001_091843_d4a7e2ce",
    "report_html_url": "/reports/20261001_091843_d4a7e2ce/html"
  }
}

---

GET /jobs — queue stats

curl http://localhost:8765/jobs

{
  "queued": 0, "processing": 1, "done": 42, "failed": 0,
  "total": 43, "queue_depth": 0, "workers": 2
}

---

GET /reports/{transid}/html — open a report in the browser

open http://localhost:8765/reports/20261001_091843_d4a7e2ce/html

---

GET /reports — list past runs

curl http://localhost:8765/reports | python3 -m json.tool

---

GET /health — liveness check

curl http://localhost:8765/health
# {"status":"ok","version":"0.1.0"}

---

Save the HTML report from a job result

JOB_ID=$(curl -s -X POST http://localhost:8765/compare \
  -H "Content-Type: application/json" \
  -d '{"image_a":"https://…/a.png","image_b":"https://…/b.png","include_html":true}' \
  | python3 -c "import sys,json; print(json.load(sys.stdin)['job_id'])")

# Wait for it to finish, then save the report
curl -s http://localhost:8765/jobs/$JOB_ID \
  | python3 -c "
import sys, json, base64
r = json.load(sys.stdin)
b64 = r.get('result', {}).get('report_html_b64', '')
if b64:
    open('report.html', 'wb').write(base64.b64decode(b64))
    print('Saved report.html')
"

---

Docker

Build and run

docker build -t mobile-ui-comparator .
docker run -p 8765:8765 \
  -v $(pwd)/ui-diff-reports:/app/ui-diff-reports \
  mobile-ui-comparator

Docker Compose (recommended)

docker compose up --build

Reports are written to ./ui-diff-reports/ on your machine.

Build profiles

Profile	Command	Notes
standard (default)	docker build -t mobile-ui-comparator .	Recommended starting point
lite	docker build --build-arg INCLUDE_OCR=false -t mui-lite .	Smaller image, no text diff
full	docker build --build-arg INCLUDE_CLIP=true -t mui-full .	Largest image, all analysis stages

Verify after startup

curl http://localhost:8765/health

curl -s -X POST http://localhost:8765/compare \
  -H "Content-Type: application/json" \
  -d '{
    "image_a": "https://your-cdn.com/screenshot_a.png",
    "image_b": "https://your-cdn.com/screenshot_b.png",
    "device_a": "Build X",
    "device_b": "Baseline",
    "screen": "Home"
  }'

---

Output files

Each comparison creates a folder ui-diff-reports/<transid>/:

File	What it contains
report.html	Visual report — open in any browser, no internet needed
report.json	Full machine-readable report with all scores and issues
diff_annotated.png	Side-by-side view with numbered bounding boxes
heatmap.png	Pixel-diff heat map — red means changed, green means matching

---

CLI reference

Usage: compare-ui [OPTIONS] IMAGE_A IMAGE_B

  Compare two mobile screenshots for UI regressions.
  IMAGE_A and IMAGE_B can be local file paths or https:// URLs.

Options:
  --device-a TEXT          Label for the reference image  [default: Device A]
  --device-b TEXT          Label for the candidate image  [default: Device B]
  --screen TEXT            Screen name shown in the report  [default: Unknown]
  --output PATH            Output directory  [default: ./ui-diff-reports]
  --threshold FLOAT        SSIM pass threshold 0.0–1.0  [default: 0.85]
  --skip-phash             Skip the same-screen gate (use for different states)
  --no-mask-status-bar     Include the OS status bar in analysis
  --ignore-region X,Y,W,H  Exclude a rectangle from analysis (repeatable)
  --no-heatmap             Skip heatmap generation
  --json-only              Output JSON only, no images
  --no-verify-ssl          Skip SSL certificate verification for URL downloads
  --debug                  Enable verbose debug logging
  --log-file PATH          Write a timestamped log to a file
  --version                Show version and exit
  --help                   Show this message and exit

---

Single-image audit

Audit one screenshot for UI quality issues without a baseline — checks text size, contrast, touch targets, and visual anomalies.

CLI

analyze-ui screenshot.png --device "Pixel 8 Pro"
analyze-ui https://your-cdn.com/screen.png --device "Build 123" --open

API

# Enqueue
curl -s -X POST http://localhost:8765/analyze \
  -H "Content-Type: application/json" \
  -d '{"image": "https://your-cdn.com/screen.png", "device": "Build 123"}'
# → { "job_id": "…", "poll_url": "/analyze-jobs/<id>" }

# Poll
curl http://localhost:8765/analyze-jobs/<job_id>

Result fields (inside result when status is "done")

Field	Description
status	PASS / WARNING / FAIL
critical / major / minor	Issue counts by severity
issues	Array of numbered issues with type, zone, description, fix hint
annotated_path	Server-side path to the numbered bounding-box overlay image
html_path	Server-side path to the full HTML report
json_path	Server-side path to the machine-readable JSON report

Single-image output files

Each run creates ui-diff-reports/analyze_<transid>/:

File	What it contains
analyze_annotated.png	Screenshot with numbered bounding boxes, severity colours, and legend
analyze_report.html	Visual report showing the annotated image and issue table
analyze_report.json	Machine-readable issue list

---

Integrating in your application

1 — Install from PyPI

Once published, install with pip:

pip install mobile-ui-comparator          # CLI only
pip install "mobile-ui-comparator[server]" # CLI + REST API server

After install, compare-ui and analyze-ui are available as terminal commands.

---

2 — One-call Python functions (simplest)

Sync vs async: The function API is synchronous — analyze() and compare() block your thread until the analysis is complete and return the result directly. No polling, no job IDs. The HTTP server (section 4) uses an async job queue and requires polling.

The package exposes two top-level functions that run the full pipeline with one call and write all reports automatically.

---

Analyze a local screenshot

from pathlib import Path
from mobile_ui_comparator import analyze

# Pass a local file path (string or pathlib.Path)
result = analyze(
    Path("screenshots/home_screen.png"),   # or just "screenshots/home_screen.png"
    device="Pixel 8 Pro",
    output_dir="reports",                  # reports are written here; created if missing
)

# ── Status ────────────────────────────────────────────────────────────
print(result.status)       # "PASS" | "WARNING" | "FAIL"
print(result.critical)     # number of critical-severity issues
print(result.major)        # number of major-severity issues
print(result.minor)        # number of minor-severity issues
print(result.total_issues) # critical + major + minor

# ── Output files ──────────────────────────────────────────────────────
print(result.annotated)    # Path  →  reports/analyze_20241005_143201_a3f9/annotated.png
print(result.report_json)  # Path  →  reports/analyze_.../report.json
print(result.report_html)  # Path  →  reports/analyze_.../report.html

# ── Issues list ───────────────────────────────────────────────────────
for issue in result.issues:
    print(
        issue["id"],           # 1, 2, 3 …  (matches number drawn on annotated image)
        issue["severity"],     # "CRITICAL" | "MAJOR" | "MINOR"
        issue["type"],         # "CONTRAST" | "TOUCH_TARGET" | "OCR_OVERLAP" | "VISUAL" …
        issue["description"],  # human-readable description
        issue["fix_hint"],     # suggested fix
        issue["region"],       # {"x": 42, "y": 120, "w": 300, "h": 56}
    )

# ── Use as a boolean gate ─────────────────────────────────────────────
# result == True  when status is "PASS"
# result == False when status is "WARNING" or "FAIL"
if not result:
    raise SystemExit(f"UI audit failed — {result.critical} critical, {result.major} major")

---

Compare two local screenshots

from pathlib import Path
from mobile_ui_comparator import compare

result = compare(
    Path("screenshots/baseline/home.png"),   # reference  (image A)
    Path("screenshots/build-123/home.png"),  # candidate  (image B)
    device_a="Baseline",
    device_b="Build 123",
    screen="Home Screen",
    output_dir="reports",
)

# ── Status ────────────────────────────────────────────────────────────
print(result.status)           # "PASS" | "WARNING" | "FAIL"
print(result.critical)
print(result.major)
print(result.minor)

# ── Similarity scores ─────────────────────────────────────────────────
print(result.overall_ssim)       # 0.0 – 1.0   (structural similarity)
print(result.dino_similarity)    # 0.0 – 1.0   (semantic/perceptual)
print(result.canvas_diff_pct)    # 0.0 – 100.0 (% pixels changed)

# ── Output files ──────────────────────────────────────────────────────
print(result.annotated)    # side-by-side diff with numbered boxes
print(result.heatmap)      # heatmap highlighting changed regions
print(result.report_json)
print(result.report_html)

# ── Issues list (same structure as analyze) ───────────────────────────
for issue in result.issues:
    print(issue["severity"], issue["description"])

# ── CI gate ───────────────────────────────────────────────────────────
if not result:
    raise SystemExit(f"Regression: {result.critical} critical issues found")

---

Using URLs instead of local paths

Both functions also accept http:// or https:// URLs — the image is downloaded to a temporary file automatically and cleaned up when the call returns.

from mobile_ui_comparator import analyze, compare

# analyze a remote screenshot
result = analyze(
    "https://storage.example.com/builds/123/home.png",
    device="Build 123",
)

# compare a remote baseline against a remote candidate
result = compare(
    "https://cdn.example.com/baseline/home.png",
    "https://cdn.example.com/build-456/home.png",
    device_a="Baseline", device_b="Build 456",
)

---

AnalyzeResult fields

Field	Type	Description
status	str	PASS / WARNING / FAIL
critical / major / minor	int	Issue counts by severity
total_issues	int	Sum of all severities
issues	list[dict]	Each issue: id, severity, type, description, fix_hint, region
ocr_blocks	int	Number of text blocks detected
checks_run	list[str]	Names of checks that executed
annotated	Path	Numbered bounding-box overlay image
report_json	Path	Machine-readable JSON
report_html	Path	Human-readable HTML report
run_dir	Path	Directory containing all output files

CompareResult fields (all AnalyzeResult fields plus)

Field	Type	Description
overall_ssim	float	Full-canvas SSIM score (0–1)
canvas_diff_pct	float	% pixels that changed
dino_similarity	float	DINOv2 semantic similarity (0–1)
heatmap	Path	Heatmap PNG highlighting changed regions

---

3 — Use as a Python library (low-level)

from pathlib import Path
from mobile_ui_comparator import single_analyzer, reporter, normalizer
from mobile_ui_comparator import phash_gate, dino_similarity, ocr_diff, ssim_regions

# ── Single-image audit ────────────────────────────────────────────────
result = single_analyzer.run("screenshot.png", device_label="Pixel 8 Pro")

print(result.status)          # PASS | WARNING | FAIL
print(result.critical)        # number of critical issues
for issue in result.issues:
    print(issue["id"], issue["severity"], issue["description"])

run_dir = Path("my-reports/run-001")
run_dir.mkdir(parents=True, exist_ok=True)

annotated = single_analyzer.generate_annotated(result, Path("screenshot.png"), run_dir)
json_file  = single_analyzer.save_json(result, Path("screenshot.png"), run_dir)
html_file  = single_analyzer.save_html(result, Path("screenshot.png"), run_dir,
                                        annotated_path=annotated)

# ── Two-image comparison ──────────────────────────────────────────────
from mobile_ui_comparator import (
    phash_gate, dino_similarity, ocr_diff,
    ssim_regions, categorizer, annotator, reporter, normalizer
)
from pathlib import Path

path_a, path_b = Path("baseline.png"), Path("build.png")
run_dir = Path("my-reports/compare-001")
run_dir.mkdir(parents=True, exist_ok=True)

norm        = normalizer.run(path_a, path_b)
dino_result = dino_similarity.run(norm.img_a, norm.img_b,
                                  canvas_size=(norm.canvas_w, norm.canvas_h))
ocr_result  = ocr_diff.run(path_a, path_b)
ssim_result = ssim_regions.run(path_a, path_b,
                                img_a_array=norm.img_a, img_b_array=norm.img_b)

regions = dino_result.regions or ssim_result.regions
categorizer.classify(regions, img_a_path=path_a, img_b_path=path_b,
                     canvas_size=(norm.canvas_w, norm.canvas_h))

annotator.generate(image_a=path_a, image_b=path_b,
                   img_a_array=norm.img_a, img_b_array=norm.img_b,
                   regions=regions, diff_map=dino_result.diff_map,
                   output_dir=run_dir, device_a="Baseline", device_b="Build 123")

---

4 — Run the server and call via HTTP (async job queue)

This API is asynchronous. Every POST /compare and POST /analyze call returns a job_id immediately. You must poll GET /jobs/{job_id} (or GET /analyze-jobs/{job_id}) until status is "done" or "failed", then read result from the response. Use the function API (section 2) if you want a simpler synchronous call.

# Start (keep running; models load once at startup)
uvicorn mobile_ui_comparator.server:app --host 0.0.0.0 --port 8765

import time, requests

BASE = "http://localhost:8765"

# ── Option A: remote URLs (server downloads them) ─────────────────────
r = requests.post(f"{BASE}/compare", json={
    "image_a": "https://cdn.example.com/baseline/home.png",
    "image_b": "https://cdn.example.com/build-123/home.png",
    "device_a": "Baseline", "device_b": "Build 123", "screen": "Home"
})

# ── Option B: local files on the same machine as the server ──────────
# Pass the absolute path as a string; the server reads it from disk.
r = requests.post(f"{BASE}/compare", json={
    "image_a": "/absolute/path/to/baseline.png",
    "image_b": "/absolute/path/to/build.png",
    "device_a": "Baseline", "device_b": "Build 123", "screen": "Home"
})

job_id = r.json()["job_id"]

# ── Poll until done (the API is async — always poll) ─────────────────
while True:
    poll = requests.get(f"{BASE}/jobs/{job_id}").json()
    if poll["status"] in ("done", "failed"):
        break
    time.sleep(2)

if poll["status"] == "failed":
    raise RuntimeError(poll["error"])

result = poll["result"]

# ── Read the response ─────────────────────────────────────────────────
print(result["scores"]["status"])      # "PASS" | "WARNING" | "FAIL"
print(result["summary"]["critical"])   # critical issue count
print(result["summary"]["major"])
print(result["run_dir"])               # server-side directory with all report files
print(result["report_html"])           # absolute path to HTML report on the server

for issue in result["issues"]:
    print(issue["id"], issue["severity"], issue["description"])

import time, requests

BASE = "http://localhost:8765"

# ── Option A: remote URL ──────────────────────────────────────────────
r = requests.post(f"{BASE}/analyze", json={
    "image": "https://cdn.example.com/build-123/home.png",
    "device": "Build 123"
})

# ── Option B: local file on the same machine as the server ───────────
r = requests.post(f"{BASE}/analyze", json={
    "image": "/absolute/path/to/screenshot.png",
    "device": "Pixel 8 Pro"
})

job_id = r.json()["job_id"]

# ── Poll until done ───────────────────────────────────────────────────
while True:
    poll = requests.get(f"{BASE}/analyze-jobs/{job_id}").json()
    if poll["status"] in ("done", "failed"):
        break
    time.sleep(2)

if poll["status"] == "failed":
    raise RuntimeError(poll["error"])

result = poll["result"]

# ── Read the response ─────────────────────────────────────────────────
print(result["status"])           # "PASS" | "WARNING" | "FAIL"
print(result["critical"])         # 0
print(result["major"])            # 2
print(result["total_issues"])     # 2
print(result["report_html"])      # absolute path on the server

for issue in result["issues"]:
    print(
        issue["id"],              # matches number on annotated image
        issue["severity"],        # "CRITICAL" | "MAJOR" | "MINOR"
        issue["description"],
        issue["fix_hint"],
    )

---

5 — CI/CD integration (GitHub Actions)

Add visual regression checks to every pull request:

# .github/workflows/ui-check.yml
name: UI regression check

on: [pull_request]

jobs:
  ui-compare:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Mobile UI Comparator
        run: pip install "mobile-ui-comparator[server]"

      # Start the server in the background
      - name: Start server
        run: |
          uvicorn mobile_ui_comparator.server:app \
            --host 0.0.0.0 --port 8765 &
          sleep 10   # wait for model warm-up

      # Compare PR screenshot against baseline
      - name: Compare screenshots
        run: |
          compare-ui \
            screenshots/baseline/home.png \
            screenshots/pr/home.png \
            --device-a "Baseline" \
            --device-b "PR ${{ github.event.pull_request.number }}" \
            --screen "Home" \
            --output ui-reports

      # Audit the PR screenshot for standalone issues
      - name: Single-image audit
        run: |
          analyze-ui screenshots/pr/home.png \
            --device "PR ${{ github.event.pull_request.number }}" \
            --output ui-reports

      # Upload all reports as CI artifacts
      - name: Upload reports
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: ui-reports
          path: ui-reports/

---

6 — Deploy with Docker (production)

# Build
docker build -t mobile-ui-comparator .

# Run — mount a volume so reports survive container restarts
docker run -d \
  --name ui-comparator \
  -p 8765:8765 \
  -v /data/ui-reports:/app/ui-diff-reports \
  --restart unless-stopped \
  mobile-ui-comparator

# docker-compose.yml (recommended)
services:
  ui-comparator:
    build: .
    ports:
      - "8765:8765"
    volumes:
      - ./ui-reports:/app/ui-diff-reports
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8765/health"]
      interval: 30s
      timeout: 5s
      retries: 3

docker compose up -d
curl http://localhost:8765/health   # {"status":"ok","version":"0.1.0"}

---

7 — Minimum requirements

Resource	Minimum	Recommended
CPU	2 cores	4 cores
RAM	4 GB	8 GB
Disk	2 GB (models + reports)	10 GB
Python	3.9+	3.11+

Model weights download once on first startup (~400 MB) and are cached locally.
Warm inference time per request: 5–8 s on 4 CPU cores.

---

License

MIT