crowd-test 0.5.0

Hire a crowd of AI virtual users to test your web app in minutes

crowd-test 🔥

Let the mob loose on your app — before the internet does.

The internet is not a QA lab. It's a mob: impatient, confused, distracted, skeptical, and occasionally feral. crowd-test recreates that mob out of AI personas and unleashes it on your website in real browsers. They rage-quit your checkout, get lost in your navigation, tab through your forms, reject your cookie banners, hunt for hidden fees, and mash your buttons looking for race conditions.

Then they hand you the damage report — and a survival grade.

pip install crowd-test
crowd-test run https://your-app.com --mob 20

Does your app survive the mob? Most don't on the first run.

Why

• Unit tests tell you your functions work.
• E2E tests tell you your happy path works.
• crowd-test tells you what happens when a 72-year-old, an impatient shopper on 4G, a privacy hawk, a keyboard-only user, and a chaos monkey all hit your app at once — the thing you currently learn after launch, from one-star reviews.

The mob

Ten named ringleaders ship built-in. The rest of the mob is generated on demand.

Persona	Who they are	What they catch
🕐 Mai — The Impatient Shopper	28, mobile, zero patience	Long flows, slow feedback, needless form fields
👴 Harold — The Senior Citizen	72, barely uses computers	Unlabeled icons, tiny text, confusing conventions
⌨️ Kai — The Keyboard-Only User	35, navigates without a mouse	Focus traps, missing outlines, a11y blockers
👀 Amara — The First-Time Visitor	31, zero context about you	Unclear value prop, buried pricing, jargon
🐒 Rex — The Chaos Monkey	19, breaks things for fun	Edge cases, race conditions, raw error screens
🍼 Dana — The Distracted Parent	38, one-handed, interrupted	Lost state, wiped forms, unforgiving flows
🕵️ Viktor — The Privacy Hawk	44, gives you nothing	Dark patterns, forced accounts, data grabs
🌏 Yuki — The Non-Native Speaker	26, literal English	Jargon, idiom buttons, US-only form formats
💰 Penny — The Bargain Hunter	52, audits every cent	Hidden fees, price changes, silent coupon failures
⚡ Sam — The Speedrunner	23, counts every click	Wasted steps, broken Enter keys, forced waits

Mob mode

Ten not enough? Generate as many as you can afford:

crowd-test run https://your-app.com --mob 20              # ringleaders + 20 randoms
crowd-test run https://your-app.com --personas none --mob 50 --seed 1   # pure reproducible mob

Every mob member gets a random age, patience level, tech skill, device, goal, and two behavioral quirks. No two runs of your app face the same crowd — unless you pin a --seed for CI.

Roll your own ringleader

A persona is 20 lines of YAML:

name: grandma-linh
display_name: "Linh — Skeptical Grandmother"
age: 68
tech_savviness: very_low
patience: medium
goals:
  - "Find out whether this store ships to her city"
traits:
  - "Distrusts any site that asks for personal info too early"
quirks:
  - "Reads every word of small print before clicking anything"

crowd-test run https://your-app.com --persona-file grandma-linh.yaml

The survival grade

Every run ends with a verdict, computed from findings and mob satisfaction:

Grade	Meaning
S	the mob couldn't lay a finger on it
A	survived with a few scratches
B	walked away limping
C	took real damage
D	barely crawled out alive
F	☠️ the mob destroyed it

Add --fail-on-critical in CI to block the merge when the mob draws blood.

The tribunal ⚖️

A mob is dramatic — that's the point — but drama sometimes exaggerates. Worse, browser automation itself sometimes glitches, and then every agent on the same stack hallucinates the same bug. crowd-test answers with up to three independent layers of verification, each on a different architecture:

1. 🕵️ The detective (--verify) — a cold, skeptical QA agent with no persona and no stake starts from a clean page load and tries to reproduce every critical/major accusation.
2. 🔬 The cross-engine probe (--cross-verify) — an LLM translates the accusation into a tiny declarative repro script (plain data, never code) and a raw Playwright interpreter — a completely different browser stack — executes it. This is the layer that catches automation artifacts: bugs the mob and the detective both swear they saw, on a site that actually works.
3. ⚖️ The tribunal (--tribunal) — the last court of appeal. A Microsoft Webwright agent — a third harness that writes and debugs its own repro scripts — retries the bug independently and files its verdict.

crowd-test run https://your-app.com --mob 10 --verify        # detective only
crowd-test run https://your-app.com --mob 10 --cross-verify  # + Playwright probe
crowd-test run https://your-app.com --mob 10 --tribunal      # all three

• ✅ verified — reproduced independently. It's real. Go fix it.
• ⚠️ not reproduced — stays in the report for transparency, but stops counting toward your survival grade.
• 🔬 disputed — a different engine watched the same flow work fine; the accusation was almost certainly an automation artifact. Excluded from grade.

The mob accuses. The tribunal convicts. Your grade only bleeds for real bugs.

Extra installs for the deeper layers:

pip install crowd-test[probe]        # --cross-verify (Playwright)
playwright install chromium
pip install git+https://github.com/microsoft/Webwright   # --tribunal

Note that crowd-test runs the mob's browser with automation extensions disabled so personas see your site exactly as real users do — cookie banners and all.

Quick start

pip install crowd-test

export ANTHROPIC_API_KEY=sk-...      # or OPENAI_API_KEY

crowd-test run https://your-app.com                  # the ten ringleaders
crowd-test run https://your-app.com --mob 10         # plus ten randoms
crowd-test run https://your-app.com --personas rex-chaos-monkey
crowd-test run https://your-app.com --goal "Sign up and create a project"
crowd-test list-personas

You get:

• report.md — findings ranked by severity, ready to paste into an issue or PR (sample)
• report.html — a shareable dark-mode damage report with the survival grade, each persona's verdict, and their in-character complaints
• results.json — machine-readable results for your own tooling

Use as a Claude Code skill

Let your coding agent run the mob for you. Install the skill once:

git clone https://github.com/anhhuyn411-alt/crowd-test /tmp/crowd-test-skill
mkdir -p ~/.claude/skills
cp -r /tmp/crowd-test-skill/skills/crowd-test ~/.claude/skills/crowd-test

Then, inside Claude Code (or any agent that speaks the open skill standard):

/crowd-test send the mob at https://staging.your-app.com

The agent installs crowd-test if needed, picks sensible personas, runs the crew, and summarizes the damage report with fixes — instead of you reading raw findings. The same folder works in Codex, Cursor, and Gemini CLI.

How it works

flowchart LR
    A[Persona YAML / mob generator] --> B[Persona compiler]
    B --> C[AI agent + real browser]
    C --> D[Your website]
    C --> E[Per-persona verdict JSON]
    E --> F[Survival grade + damage report]

Each persona becomes a role-played AI agent (powered by browser-use) driving a real Chromium instance. Personas run in parallel with isolated browser profiles, so their cookies and sessions never mix.

Point it only at apps you own or have permission to test. The mob is for your own staging and production sites, not other people's.

What it is not

• Not a load-testing or stress tool — a mob of minds, not a flood of requests.
• Not a replacement for real user research — it's the cheap, brutal layer before it.
• Not a scripted E2E suite — runs are exploratory and slightly different every time. That's the point: humans are too.

Roadmap

• Detective agent: adversarial verification of every critical/major finding (--verify)
• Machine-readable results.json next to every report
• Cross-engine detective: verify accusations in a second, independent browser stack (--cross-verify)
• The tribunal: a third harness (Microsoft Webwright) as the last court of appeal (--tribunal)
• Claude Code skill: /crowd-test from inside your coding agent (skills/crowd-test/)
• GitHub Action: the mob tests your preview deploy and posts the survival grade on the PR
• Screenshots attached to every finding
• Survival badge for your README (survived the mob: A)
• Persona marketplace: community-contributed ringleaders in personas/
• Grade history: track your survival grade across releases

Contributing

The easiest and most fun contribution: add a ringleader. Send a PR with one YAML file to crowdtest/personas/ and a row in the table above. Bug reports and feature ideas are welcome in issues.

git clone https://github.com/anhhuyn411-alt/crowd-test
cd crowd-test
pip install -e .[dev]
pytest

License

MIT