Shopify Image Audit

A Lighthouse-based image audit tool for Shopify stores. Produces per-image scores, role assignments, optimisation recommendations, and a before/after comparison workflow that proves image-optimisation ROI to paying customers.

Designed for the 99–199 € audit-on-demand business model: run the audit, deliver a customer-ready HTML report, optionally compare live metrics after the customer implements the recommendations.

Quickstart

End-user install (PyPI)

# Recommended: pipx creates an isolated env
pipx install shopify-image-audit

# Alternative: pip into a venv
python -m venv .venv && source .venv/bin/activate
pip install shopify-image-audit

The PDF renderer (audit report --pdf) requires native libraries (libpango, libcairo, libgdk-pixbuf). On Linux:

sudo apt-get install -y libpango-1.0-0 libpangoft2-1.0-0 libcairo2 libgdk-pixbuf-2.0-0

The lighthouse Node CLI is required for audit run <url>. Install with npm i -g lighthouse (or pass --lhr <file> to use a pre-existing report). The binary can be overridden via --lighthouse-bin PATH or $LIGHTHOUSE_BIN; see docs/integrations/LIGHTHOUSE.md for the full install guide and CI recipes.

Developer install (from source)

git clone https://github.com/xopsio/shopify-image-audit.git
cd shopify-image-audit
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -e ".[dev]"

# Verify the install
audit version
pytest -q                                              # 830 tests

Shopify stores (OAuth, preferred)

Log in once per store — the token is stored encrypted in tokens.json (system keyring), so no manual --access-token is needed afterwards:

audit shopify login mystore.myshopify.com
audit shopify inventory mystore.myshopify.com -o inventory.json

# Many stores in one go (v0.16.2+): log in via the same stores.json
# that batch consumes, then audit them all
audit shopify login --stores-file stores.json
audit shopify batch --stores-file stores.json -o batch.json

CLI commands

The tool ships with a single Typer app. Run audit --help for the full list; below are the high-value entry points.

audit run <url> — full Lighthouse + audit pipeline

audit run https://kauppa.myshopify.com --device mobile --runs 3 \
    --out-dir artifacts
# -> artifacts/lhr_run1.json, audit_result.json (schema-compliant)

audit baseline <lhr.json> --save baseline.json — capture baseline

audit baseline fixtures/before_after/before_lcp.json \
    --save baseline.json \
    --url https://demo.myshopify.com
# -> Baseline saved to baseline.json

audit compare <baseline> <current> — before/after (file or URL)

# File vs file (offline)
audit compare baseline.json fixtures/before_after/after_lcp.json \
    -o comparison.html --json comparison.json

# File vs live URL (fetches via PageSpeed Insights API)
audit compare baseline.json https://demo.myshopify.com \
    --strategy mobile --api-key YOUR_KEY \
    -o comparison.html

# HTML report includes a per-image delta table (bytes, score, status per image).
# PDF export via --pdf flag.

Exit codes: 0 success, 2 invalid args, 10 backend failure.

audit report <audit_result.json> — HTML or PDF report

audit report baseline.json -o report.html          # HTML
audit report baseline.json -o report.pdf --pdf     # PDF (WeasyPrint)

audit measure <url> — live PageSpeed metrics only

audit measure https://demo.myshopify.com --strategy mobile
# -> JSON metrics to stdout (or --output metrics.json)

audit shopify <login|auth|inventory|batch> — Shopify Admin API

# Preferred (v0.15+): OAuth login, token persisted encrypted in
# tokens.json — no manual tokens needed
audit shopify login mystore.myshopify.com
audit shopify auth mystore.myshopify.com           # verify the stored token
audit shopify inventory mystore.myshopify.com -o inventory.json

# One-shot login for many stores, then batch-audit them (v0.16.2+)
audit shopify login --stores-file stores.json
audit shopify batch --stores-file stores.json -o batch.json

Read-only scopes required (read_products, read_themes, read_shop). Manual tokens still work via --access-token / $SHOPIFY_ACCESS_TOKEN. See docs/integrations/SHOPIFY_ADMIN.md for token acquisition and docs/integrations/SHOPIFY_OAUTH.md for the OAuth flow.

audit score <audit_input.json> --ranker {heuristic|ml}

audit score extracted.json                    # default: heuristic
audit score extracted.json --ranker ml        # weighted feature ensemble

Full reference: docs/spec/cli_v0_1.md.

Scoring algorithms

The pipeline assigns each image a role, a score (0–100), and a recommendation. Two rankers ship, switchable via --ranker ml:

Both produce the same output contract (role + score + recommendation). The ML ranker is a hand-coded feature ensemble, not a statistical model — see src/audit/ranker_ml.py for the design rationale (no model deps, deterministic, fully explainable via ml_features()).

Configuration

Repeated options can be set once in ~/.config/shopify-image-audit/config.toml (or $XDG_CONFIG_HOME/shopify-image-audit/config.toml):

[defaults]
device = "mobile"    # run / baseline / schedule add
strategy = "mobile"  # measure / compare
parallel = 4         # shopify batch / schedule run-all (0 = unlimited)

[pagespeed]
api_key = "AIza..."  # or --api-key / $PAGESPEED_API_KEY

[report]
brand_color = "#ff6b35"   # report / compare branding

Precedence: CLI flag > env var > config > default. A broken config warns and falls back to defaults — it never blocks a run. Full reference (13 keys / 5 sections) in CONTRIBUTING §5b.

Architecture

src/
├── audit/                    # scoring + reporting
│   ├── models.py             # Pydantic v2 schemas (AuditResult, ComparisonResult, ImageDelta)
│   ├── parser.py             # Lighthouse / fixture JSON parser
│   ├── ranker_heuristic.py   # default ranker (bpp-based)
│   ├── ranker_ml.py          # opt-in ML-style ranker (weighted ensemble)
│   └── report.py             # HTML/PDF report renderer (split into _render_* funcs)
├── core/                     # core algorithms
│   ├── image_extractor.py    # LHR image audit extraction
│   ├── image_signals.py      # shared displayed_area, assign_role, _safe_int
│   └── baseline_manager.py   # save/load baselines + compare() + per-image matching
├── engine/                   # orchestration + CLI
│   ├── cli.py                # Typer app (run, measure, baseline, compare, shopify, ...)
│   ├── cli_helpers/          # extracted CLI helpers (validators, dispatchers, table, errors)
│   └── audit_orchestrator.py # run_audit() pipeline
└── integrations/             # external APIs
    ├── pagespeed_api.py      # PageSpeed Insights (measure + fetch_lighthouse_json)
    └── shopify_admin.py      # Shopify Admin API (auth, products, theme_assets)

src/audit/schemas/audit_result.schema.json  # JSON Schema contract (validated by tests)
tests/                              # 830 tests, single-writer (ZCode)
docs/examples/                       # live demo report + comparison JSON
docs/integrations/                   # Shopify Admin API token guide

The codebase is governed by a single ZCode agent (see docs/governance.md v1.3).

Testing

pytest -q                                # 830 tests, single-writer discipline
pytest --cov=src --cov-report=term       # ~91% coverage
ruff check src/ tests/                   # 0 violations

The CI workflow runs pytest -q + ruff check on Python 3.11 and 3.12 for every PR. Branch protection on main requires both checks to pass before merge. See .github/workflows/ci.yml.

Customer deliverables (Phase 1)

• Customer report template — docs/CUSTOMER_REPORT_TEMPLATE.md
• Onboarding workflow — docs/CUSTOMER_ONBOARDING.md
• Example audit report (Nordic Lifestyle demo store, LCP 4200ms → 1800ms) — docs/examples/demo_audit_report.html
• Example comparison data — docs/examples/demo_comparison.json

Roadmap

• ✅ Sprint 1 — v0.1.0 baseline (parser, ranker, orchestrator, CLI, HTML report, 103 tests)
• ✅ Sprint 2 — before/after workflow, customer docs, ML ranker, live URL compare, CI, governance cleanup (276 tests)
• ✅ Sprint 3 — PDF export, per-image deltas, Shopify Admin API, v0.2.0 release prep (390 tests)
• ✅ Sprint 4 — Branded reports, ROI-ranked recs, audit history, v0.3.0 (489 tests)
  • Branded report templates (--brand-logo, --brand-color)
  • ROI-ranked recommendations (ComparisonRecommendation model)
  • Audit history + trend view (HistoryStore, audit history list/show)
• ✅ Sprint 5 — Snapshot tests, CLI coverage, error decorator wiring, history diff, v0.4.0 (546 tests)
  • Snapshot testing infrastructure (syrupy) for HTML renderers
  • CLI coverage for all 10 commands
  • Error decorator wiring + consistency pass (RuntimeError → exit 10)
  • audit history diff with stable entry-ids
  • CHANGELOG.md and --cov-fail-under=85 CI gate
• ✅ Sprint 6 — Coverage close-out, test isolation, multi-store batch, observability, v0.5.0 (606 tests)
  • tests/test_table_snapshots.py (Rich Console captures)
  • Zero CWD-relative writes in tests
  • audit shopify batch --stores-file for multi-store inventory
  • engine._logging with 6 structured log hooks
  • CONTRIBUTING.md, --cov-fail-under=90
• ✅ Sprint 7 — Scheduled re-audit, dependency hygiene, PageSpeed cache, v0.6.0 (642 tests)
  • audit schedule list/add/remove/run-all + crontab runbook
  • Dependabot + SLSA build-provenance attestation
  • PageSpeed response cache (PAGESPEED_CACHE_TTL)
  • Report footer version drift fixed
• ✅ Sprint 8 — UX polish, test architecture, shared run_parallel, v0.7.0 (665 tests)
  • "Did you mean: X?" suggestions on every typo site
  • tests/conftest.py + tests/__init__.py (single source of truth for REPO_ROOT + fixtures)
  • Shared run_parallel helper (batch + scheduler share it)
  • audit schedule run-all --parallel (deferred from Sprint 7)
  • [pdf] extra — WeasyPrint no longer required by default
  • docs/tutorial.md walkthrough
• ✅ Sprint 9 — Doc hygiene, wheel packaging, security polish, v0.7.1 (672 tests)
  • JSON schema ships inside the wheel (importlib.resources-readable)
  • PAGESPEED_API_KEY env var + API-key redaction in error messages
  • schedules.json written with 0600 permissions
  • Issue templates (bug report + feature request)
  • Doc hygiene: canonical env-var reference, drift-free test counts
• ✅ Sprint 10 — Type safety + SBOM, v0.8.0 (672 tests)
  • Mypy CI-gate, zero type: ignore comments
  • ImageDict TypedDict contract across the pipeline
  • CycloneDX SBOM in release artifacts (complements SLSA provenance)
• ✅ Sprint 11 — User-side TOML config, v0.9.0 (695 tests)
  • ~/.config/shopify-image-audit/config.toml (13 keys / 5 sections)
  • Precedence: CLI flag > env var > config > default
  • Broken config warns + falls back — never blocks a run
• ✅ Sprint 12 — Repo hygiene + format gate, v0.9.1 (695 tests)
  • ruff format applied across the whole repo (53 files)
  • ruff format --check added to CI (no future drift)
  • 4 Dependabot PRs merged (rich constraint widened, GitHub Actions bumps)
• ✅ Sprint 13 — Strict mypy, v0.10.0 (695 tests)
  • mypy --strict clean across 30 source files (zero ignore comments)
  • ParamSpec + TypeVar preserves decorator signatures
  • 26 mechanical annotation fixes (dict[str, Any], list[Any], return-type annotations)
• ✅ Sprint 14 — ImageDelta schema, v0.10.1 (698 tests)
  • ImageDelta.before/.after are ImageItem (was dict[str, Any])
  • Closes the Sprint 13 strict-mypy follow-up — no more cast() in baseline_manager.py
  • extra="forbid" on ImageItem now protects against future drift
• ✅ Sprint 15 — TypedDict hierarchy, v0.11.0 (705 tests)
  • LighthouseJson + AuditEntry + Categories + PerformanceCategory
  • CachedPageSpeedResponse (envelope) + CachedEntry (cache)
  • run_audit() parses LighthouseJson directly
• ✅ Sprint 16 — Lighthouse install validation, v0.12.0 (715 tests)
  • --lighthouse-bin PATH flag + $LIGHTHOUSE_BIN env override
  • 10-minute per-run timeout (no more infinite hangs)
  • docs/integrations/LIGHTHOUSE.md (install, CI recipes, troubleshooting)
  • 6/6 _run_lighthouse testing gap closed (+ 4 resolver tests)
• ✅ Sprint 17 — Rich Progress bar, v0.13.0 (721 tests)
  • Live progress bar for audit shopify batch and audit schedule run-all
  • on_done callback in run_parallel (per-item observer hook)
  • Transient bar erases on completion; per-store post-mortem lines remain the canonical output
• ✅ Sprint 18 — Shopify Admin API TypedDicts, v0.14.0 (726 tests)
  • Typed contracts for shop/products/themes/theme_assets
  • Slim client-side output shapes typed separately
  • All total=False — .get(...)-based reads and partial mocks work
• ✅ Sprint 19 — Shopify Admin OAuth, v0.15.0 (752 tests)
• ✅ Sprint 20 — Token encryption, v0.16.0 (780 tests)
  • tokens.json encrypted at rest with Fernet (AES-128-CBC + HMAC)
  • Key stored in system keyring (macOS Keychain / Windows Credential Manager / Linux Secret Service) — zero prompts on desktop
  • $SHOPIFY_AUDIT_TOKENS_DISABLED=1 opt-out for CI
  • audit shopify login <store> automates the manual custom-app flow
  • Embedded loopback HTTP server with CSRF state (constant-time check)
  • Tokens persisted to tokens.json (mode 0600)
  • docs/integrations/SHOPIFY_OAUTH.md covers setup + troubleshooting
  • Closes the long-deferred "OAuth flow" roadmap item
• ✅ Sprint 21 — Batch auto-token from TokensStore, v0.16.1 (785 tests)
  • audit shopify batch falls back to tokens.json when a stores-file entry has no access_token
  • Credential-free stores.json shareable across a team
  • Explicit access_token still wins (backwards compatible)
• ✅ Sprint 22 — Multi-store batch login, v0.16.2 (797 tests)
  • audit shopify login --stores-file stores.json authorises all stores in one run
  • Per-store failures reported in a summary; exit 2 if any failed
  • Closes the last OAuth follow-up deferred since Sprint 19
• ✅ Sprint 23 — Lighthouse mobile preset regression, v0.16.3 (800 tests)
  • v0.16.2 broke audit run --device mobile with --preset=mobile (Lighthouse 13 rejects it)
  • Mobile now uses the Lighthouse default config; desktop still passes --preset=desktop
  • TestRunLighthouseCmdShape pins the exact subprocess.run cmd list to prevent regressions
• ✅ Sprint 24 — Audit-truth & redirect detection, v0.16.4 (812 tests)
  • Empty-image audits no longer claim "All images look well optimised" — they surface "No images were extracted" instead
  • Redirects (e.g. Shopify storefront password -> /password) are caught via finalUrl and fail with exit 10 + a clear error
  • No audit_result.json is written for off-target audits
  • Docs: docs/integrations/LIGHTHOUSE.md Limitations section
• ✅ Sprint 25 — Authenticated Lighthouse, v0.17.0 (825 tests)
  • audit run --storefront-password <pwd> (or $SHOPIFY_STOREFRONT_PASSWORD) authenticates password- protected Shopify storefronts
  • POSTs the /password form once, threads the _shopify_essential cookie into Lighthouse via --extra-headers
  • Closes the long-standing "authenticated storefronts are not supported" limitation from v0.16.4
• ✅ Sprint 26 — Image extractor network-requests fallback, v0.17.1 (830 tests)
  • When image-elements and resource-summary are both empty, the extractor now falls back to network-requests filtered by resourceType == "Image"
  • Restores image discovery for older fixtures and certain dynamic pages that v0.17.0 still missed

Further reading

• docs/spec/cli_v0_1.md — full CLI specification
• docs/governance.md — ownership + workflow
• docs/runbook/measurement_protocol.md — how LCP/CLS/INP are measured deterministically
• docs/SPRINT_1_COMPLETE.md — what shipped in Sprint 1
• docs/SPRINT_3_PLAN.md — Sprint 3 breakdown (all done)
• docs/SPRINT_4_PLAN.md — Sprint 4 breakdown (all done)
• docs/SPRINT_5_PLAN.md — Sprint 5 breakdown (all done)
• QA_CHECKLIST.md — quality gates