TireKicker

TireKicker is a persona-driven evaluator for web apps. Give it a skeptical user, a target URL, and a real browser; it walks the product, records objective browser signals, files persona-voiced findings, and renders a report of what worked, what broke, and what would make that user bounce.

60-second quickstart

Install the published tool with either uv or pipx:

uv tool install tirekicker
# or: pipx install tirekicker

From a checkout, use uv tool install . instead. Then create a project configuration and point the evaluator at a backend:

kick init
export TIREKICKER_LLM_BASE_URL=https://api.openai.com/v1
export TIREKICKER_LLM_MODEL=your-model
export TIREKICKER_LLM_API_KEY=your-key
kick run organizer --brain openai_compat

Edit tirekicker.yaml and the persona before pointing a run at a real target; kick doctor checks the setup and explains missing pieces.

Backends

TireKicker talks to any OpenAI-compatible chat-completions endpoint through one httpx client, with no provider SDKs. It also ships a Codex CLI backend that shells out to a local codex exec.

The validated execution path is openai_compat, and it requires an API key. That is the backend the published walkthroughs were produced with, using a capable frontier model. The endpoint can be a hosted service, a local server, or a gateway. Keep keys in the environment, never in a persona or checked-in YAML, and run kick doctor to verify the configured 1-token health check. The request timeout defaults to 120 seconds; local reasoning models such as Ollama + Qwen may need brain.openai_compat.timeout_s: 300 or higher.

The Codex CLI backend (--brain codex) is not the validated path for this release. It needs no API key and reuses an existing codex login, which makes it attractive for offline experimentation, but in our own testing the model driving it repeatedly declined to interact with the page, ending runs without ever clicking. Model choice matters more than plumbing here: a backend that will not commit to a click produces an empty evaluation no matter how good the browser layer is. Prefer openai_compat for real work, and treat evidence-free runs as a backend signal rather than a finding about your product.

What a run leaves behind

Each run creates a timestamped directory under runs/:

• report.md is the readable seven-section evaluation with severity-grouped findings and relative screenshot links.
• findings.json is the structured record, including goal outcomes, the final verdict, and objective signals.
• transcript.jsonl records every observation, action, result, and snapshot digest.
• screenshots/ contains a full-page capture for each finding, linked from the report. A run that files no findings leaves this directory empty.
• console.log and network.har preserve browser console/page errors and network evidence.
• llm/ retains failed backend request material for diagnosis; add --debug-llm to kick run to retain every decide/reflect prompt and response, including successful calls.

Authoring a persona

Persona files are Markdown with a YAML frontmatter block. Start from kick init or copy personas/organizer.md. Give the persona an id, role, credential mode, one or more concrete goals, optional discovery hints, a disposition, limits, a test card, and a never list. Put the persona's voice and domain knowledge in the Markdown body; that body is passed verbatim to the evaluator. Use fresh_signup with a disposable tk-* email pattern, or provided with a credential environment prefix. Hints should help the evaluator judge the experience, not turn the run into a hard-coded script.

Safety and target setup

Run only against a target you are authorized to evaluate. Use test accounts, test payment cards, isolated mail and payment integrations, and an explicit cleanup plan. The live-MVP and staging guidance is in examples/target-setup.md.

TireKicker is MIT-licensed and intentionally keeps the browser, LLM, and report artifacts visible so a human can review what happened.