Metadata-Version: 2.4
Name: crowd-test
Version: 0.5.0
Summary: Hire a crowd of AI virtual users to test your web app in minutes
Project-URL: Homepage, https://github.com/anhhuyn411-alt/crowd-test
Project-URL: Issues, https://github.com/anhhuyn411-alt/crowd-test/issues
Author: Huy Nguyen
License: MIT
License-File: LICENSE
Keywords: ai-agents,browser-automation,personas,qa,testing,ux
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: browser-use>=0.13
Requires-Dist: jinja2>=3.1
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Provides-Extra: probe
Requires-Dist: playwright>=1.40; extra == 'probe'
Description-Content-Type: text/markdown

# crowd-test 🔥

**Let the mob loose on your app — before the internet does.**

The internet is not a QA lab. It's a mob: impatient, confused, distracted,
skeptical, and occasionally feral. `crowd-test` recreates that mob out of AI
personas and unleashes it on your website in real browsers. They rage-quit your
checkout, get lost in your navigation, tab through your forms, reject your
cookie banners, hunt for hidden fees, and mash your buttons looking for race
conditions.

Then they hand you the damage report — and a **survival grade**.

```
pip install crowd-test
crowd-test run https://your-app.com --mob 20
```

Does your app survive the mob? Most don't on the first run.

![Sample crowd-test damage report](docs/sample-report.png)

## Why

- **Unit tests** tell you your functions work.
- **E2E tests** tell you your happy path works.
- **crowd-test** tells you what happens when a 72-year-old, an impatient
  shopper on 4G, a privacy hawk, a keyboard-only user, and a chaos monkey all
  hit your app at once — the thing you currently learn *after* launch, from
  one-star reviews.

## The mob

Ten named ringleaders ship built-in. The rest of the mob is generated on demand.

| Persona | Who they are | What they catch |
|---|---|---|
| 🕐 **Mai — The Impatient Shopper** | 28, mobile, zero patience | Long flows, slow feedback, needless form fields |
| 👴 **Harold — The Senior Citizen** | 72, barely uses computers | Unlabeled icons, tiny text, confusing conventions |
| ⌨️ **Kai — The Keyboard-Only User** | 35, navigates without a mouse | Focus traps, missing outlines, a11y blockers |
| 👀 **Amara — The First-Time Visitor** | 31, zero context about you | Unclear value prop, buried pricing, jargon |
| 🐒 **Rex — The Chaos Monkey** | 19, breaks things for fun | Edge cases, race conditions, raw error screens |
| 🍼 **Dana — The Distracted Parent** | 38, one-handed, interrupted | Lost state, wiped forms, unforgiving flows |
| 🕵️ **Viktor — The Privacy Hawk** | 44, gives you nothing | Dark patterns, forced accounts, data grabs |
| 🌏 **Yuki — The Non-Native Speaker** | 26, literal English | Jargon, idiom buttons, US-only form formats |
| 💰 **Penny — The Bargain Hunter** | 52, audits every cent | Hidden fees, price changes, silent coupon failures |
| ⚡ **Sam — The Speedrunner** | 23, counts every click | Wasted steps, broken Enter keys, forced waits |

### Mob mode

Ten not enough? Generate as many as you can afford:

```bash
crowd-test run https://your-app.com --mob 20              # ringleaders + 20 randoms
crowd-test run https://your-app.com --personas none --mob 50 --seed 1   # pure reproducible mob
```

Every mob member gets a random age, patience level, tech skill, device, goal,
and two behavioral quirks. No two runs of your app face the same crowd —
unless you pin a `--seed` for CI.

### Roll your own ringleader

A persona is 20 lines of YAML:

```yaml
name: grandma-linh
display_name: "Linh — Skeptical Grandmother"
age: 68
tech_savviness: very_low
patience: medium
goals:
  - "Find out whether this store ships to her city"
traits:
  - "Distrusts any site that asks for personal info too early"
quirks:
  - "Reads every word of small print before clicking anything"
```

```
crowd-test run https://your-app.com --persona-file grandma-linh.yaml
```

## The survival grade

Every run ends with a verdict, computed from findings and mob satisfaction:

| Grade | Meaning |
|---|---|
| **S** | the mob couldn't lay a finger on it |
| **A** | survived with a few scratches |
| **B** | walked away limping |
| **C** | took real damage |
| **D** | barely crawled out alive |
| **F** | ☠️ the mob destroyed it |

Add `--fail-on-critical` in CI to block the merge when the mob draws blood.

## The tribunal ⚖️

A mob is dramatic — that's the point — but drama sometimes exaggerates. Worse,
browser automation itself sometimes glitches, and then every agent on the same
stack hallucinates the same bug. crowd-test answers with up to **three
independent layers of verification**, each on a different architecture:

1. 🕵️ **The detective** (`--verify`) — a cold, skeptical QA agent with no
   persona and no stake starts from a clean page load and tries to reproduce
   every critical/major accusation.
2. 🔬 **The cross-engine probe** (`--cross-verify`) — an LLM translates the
   accusation into a tiny declarative repro script (plain data, never code) and
   a raw **Playwright** interpreter — a completely different browser stack —
   executes it. This is the layer that catches automation artifacts: bugs the
   mob *and* the detective both swear they saw, on a site that actually works.
3. ⚖️ **The tribunal** (`--tribunal`) — the last court of appeal. A
   [Microsoft Webwright](https://github.com/microsoft/Webwright) agent — a
   third harness that writes and debugs its own repro scripts — retries the
   bug independently and files its verdict.

```bash
crowd-test run https://your-app.com --mob 10 --verify        # detective only
crowd-test run https://your-app.com --mob 10 --cross-verify  # + Playwright probe
crowd-test run https://your-app.com --mob 10 --tribunal      # all three
```

- ✅ **verified** — reproduced independently. It's real. Go fix it.
- ⚠️ **not reproduced** — stays in the report for transparency, but stops
  counting toward your survival grade.
- 🔬 **disputed** — a different engine watched the same flow work fine; the
  accusation was almost certainly an automation artifact. Excluded from grade.

The mob accuses. The tribunal convicts. Your grade only bleeds for real bugs.

Extra installs for the deeper layers:

```bash
pip install crowd-test[probe]        # --cross-verify (Playwright)
playwright install chromium
pip install git+https://github.com/microsoft/Webwright   # --tribunal
```

Note that crowd-test runs the mob's browser with automation extensions
disabled so personas see your site exactly as real users do — cookie banners
and all.

## Quick start

```bash
pip install crowd-test

export ANTHROPIC_API_KEY=sk-...      # or OPENAI_API_KEY

crowd-test run https://your-app.com                  # the ten ringleaders
crowd-test run https://your-app.com --mob 10         # plus ten randoms
crowd-test run https://your-app.com --personas rex-chaos-monkey
crowd-test run https://your-app.com --goal "Sign up and create a project"
crowd-test list-personas
```

You get:

- **`report.md`** — findings ranked by severity, ready to paste into an issue or PR
  ([sample](examples/sample-report.md))
- **`report.html`** — a shareable dark-mode damage report with the survival
  grade, each persona's verdict, and their in-character complaints
- **`results.json`** — machine-readable results for your own tooling

## Use as a Claude Code skill

Let your coding agent run the mob for you. Install the skill once:

```bash
git clone https://github.com/anhhuyn411-alt/crowd-test /tmp/crowd-test-skill
mkdir -p ~/.claude/skills
cp -r /tmp/crowd-test-skill/skills/crowd-test ~/.claude/skills/crowd-test
```

Then, inside Claude Code (or any agent that speaks the
[open skill standard](https://github.com/anthropics/skills)):

```
/crowd-test send the mob at https://staging.your-app.com
```

The agent installs crowd-test if needed, picks sensible personas, runs the
crew, and summarizes the damage report with fixes — instead of you reading raw
findings. The same folder works in Codex, Cursor, and Gemini CLI.

## How it works

```mermaid
flowchart LR
    A[Persona YAML / mob generator] --> B[Persona compiler]
    B --> C[AI agent + real browser]
    C --> D[Your website]
    C --> E[Per-persona verdict JSON]
    E --> F[Survival grade + damage report]
```

Each persona becomes a role-played AI agent (powered by
[browser-use](https://github.com/browser-use/browser-use)) driving a real
Chromium instance. Personas run in parallel with isolated browser profiles, so
their cookies and sessions never mix.

**Point it only at apps you own or have permission to test.** The mob is for
your own staging and production sites, not other people's.

## What it is not

- Not a load-testing or stress tool — a mob of minds, not a flood of requests.
- Not a replacement for real user research — it's the cheap, brutal layer *before* it.
- Not a scripted E2E suite — runs are exploratory and slightly different every time.
  That's the point: humans are too.

## Roadmap

- [x] Detective agent: adversarial verification of every critical/major finding (`--verify`)
- [x] Machine-readable `results.json` next to every report
- [x] Cross-engine detective: verify accusations in a second, independent browser stack (`--cross-verify`)
- [x] The tribunal: a third harness (Microsoft Webwright) as the last court of appeal (`--tribunal`)
- [x] Claude Code skill: `/crowd-test` from inside your coding agent (`skills/crowd-test/`)
- [ ] GitHub Action: the mob tests your preview deploy and posts the survival grade on the PR
- [ ] Screenshots attached to every finding
- [ ] Survival badge for your README (`survived the mob: A`)
- [ ] Persona marketplace: community-contributed ringleaders in `personas/`
- [ ] Grade history: track your survival grade across releases

## Contributing

The easiest and most fun contribution: **add a ringleader.** Send a PR with one
YAML file to `crowdtest/personas/` and a row in the table above. Bug reports
and feature ideas are welcome in issues.

```bash
git clone https://github.com/anhhuyn411-alt/crowd-test
cd crowd-test
pip install -e .[dev]
pytest
```

## License

[MIT](LICENSE)
