browser-recon

The thin client.

A capture-and-poll harness. Launches Chrome via the DevTools Protocol, streams network/DOM/interaction events into a single blob, uploads to the server, then renders a 9-step progress spinner until the report URL is ready.

Subcommands

Capture pipeline

What gets captured

Live progress polling

After upload the CLI polls GET /scans/{id}/events?since=<cursor> on a 1.5s cadence. A rich.live.Live table renders nine steps with ✓ / ✗ / spinner glyphs. Polling stops at terminal status — or pauses at awaiting_confirmation, the post-flow-confirm gate where the user must confirm or abandon before phase 2 spends credits.

# 9-step pipeline rendered in the CLI
capture → detection → analysis → flow_confirm → [GATE]
                                                                → intent_filter → validation → scrub
                                                                → synthesis → render

Two phases, one gate.

The pipeline runs as a FastAPI BackgroundTask and splits at a user-confirmation boundary. Phase 1 produces a flow verdict and pauses. Phase 2 spends the bulk of the LLM and proxy budget — but only after the user confirms the captured flow actually matches their intent.

Database tables

Persistence model

Inventory and classify.

Transforms a raw capture into a structured endpoint inventory: site shape, framework hints, deduplicated endpoint groups with response shapes, dependency edges, and four evidence signals for the LLM bucket filter.

Bucket taxonomy

Anti-bot fingerprinting.

Eight rule modules run in sequence; each produces unified Findings with confidence-weighted evidence. Cookie domain scoping (RFC 6265 + eTLD+1) is the lever that prevents third-party CDN false positives.

Anti-bot vendor coverage

Confidence weights

Each piece of evidence contributes a weight; Finding.confidence = min(1.0, Σ weights).

Other detectors

Measure, don't guess.

Real HTTP requests through real proxies. The single feature that makes the tool's recommendation grounded in what worked rather than what an LLM expected to work. Worst case: ~75-100 requests per endpoint, with a mandatory 1.5s delay between every fire.

Cascade architecture

Per-validator detail

Strip the secrets before anything else reads the blob.

Runs after validation — validation needs real cookies and auth headers to test against the live site, but the persisted blob (and everything the LLM ever sees) must be sanitised. Deep-copies capture.raw; never mutates it.

Scrub scope by field

Token-matching rule

Both pattern and field name are tokenized on [_\-.\s]+, lowercased, and require a contiguous token sequence match.

# MATCHES
password in client_password   ✓  # contiguous
cc_number in customer_cc_number_field   ✓

# DOES NOT MATCH
password in passport   ✗  # 'pass' ≠ 'password' token
cc_number in cc_field_number   ✗  # tokens not contiguous

Provider-agnostic, observability-instrumented.

Prefix-based routing dispatches every prompt to the right provider. The dispatcher records cost, tokens, latency, and S3 prompt dumps per call. Combined T16 synthesis collapsed 3 overlapping 50-89K-token calls into 1.

Routing

Model selection cascade: env BROWSER_RECON_LLM_MODEL_<PROMPT_NAME> (per-prompt override) → BROWSER_RECON_LLM_MODEL (global) → DEFAULT_MODEL.

Prompts

Observability

Every LLMClient.run() writes an llm_calls row when scan_id is set: model, provider, input/output tokens, cache read/write tokens, USD cost, latency, error class, retry count, and S3 paths to dumped prompt + response. Silently no-ops in test entry points. One row per run(); retry_count captured on re-attempt (1-shot retry on LLMParseError).

Two auth models, one user.

CLI uses bearer API keys (stateless). Web uses HttpOnly session cookies with sliding expiry and deterministic CSRF. Magic-link is the only entry point — passwords never exist.

Auth flows

recon login --email ...
  ↓
POST /auth/login  # {email, client_id}
  ↓
[email] magic link
  ↓
GET /auth/callback?token=...
  ↓
server stashes API key
  (in-memory · 5min TTL)
  ↓
CLI polls POST /auth/poll
  ↓
key written to config.toml
  (chmod 0o600)

GET /dashboard
  ↓
302 /dashboard/login
  ↓
POST /dashboard/login
  ↓
[email] magic link
  ↓
GET /auth/callback?token=...
  ↓
session cookie set
  (HttpOnly · 7d sliding · 30d hard cap)
  ↓
302 /dashboard
  ↓
CSRF token = HMAC(session, secret)

RBAC matrix

Routes

Email

Provider abstraction (email/service.py). Configured via BROWSER_RECON_EMAIL_PROVIDER:

• LoggingEmailProvider — dev/test. Dumps URLs to stdout so magic links are visible without a real inbox.
• ResendEmailProvider — production. POST to Resend HTTP API via httpx; requires RESEND_API_KEY.

Templates live under templates/emails/ as Jinja2 with inline CSS.

One full scan, from recon scan to a report URL.

$ recon scan https://walmart.com --template products
   │
   ├─ credit pre-flight check (advisory; server is source of truth)
   ├─ ASCII banner · suppressed under --quiet
   ├─ intent clarifier loop  # LLM · 1 credit on first call
   │     # 3-round cap · skip refund if before first LLM call
   ├─ launch Chrome → CDP monitor  # ephemeral profile by default
   │     # user browses · clicks · navigates · Ctrl+C to stop
   ├─ fetch full cookies · gzip · upload  # POST /scans/{id}/capture
   ↓
SERVER · Phase 1  # BackgroundTask, ~5-15s
   ├─ detection      # anti-bot, auth, pagination, rate-limit signals
   ├─ analysis       # architecture, framework, inventory, buckets, deps
   └─ flow_confirm   # LLM · status = awaiting_confirmation

CLI  # renders verdict, awaits user confirm/abandon
   ↓
POST /scans/{id}/confirm
   ↓
SERVER · Phase 2  # BackgroundTask, ~60-180s
   ├─ intent_filter  # LLM · Bucket A/B/C + guardrails
   ├─ validation     # real HTTP · real proxies · ~75-100 req/endpoint
   │                  # library_compare → header_reduce → cookies → rate_limit
   ├─ scrub          # PII/secrets stripped, blob re-uploaded to S3
   ├─ synthesis      # LLM · combined T16 (rec + verdict + code) + notes + drivers
   ├─ replay         # best-effort live verification through recommended library
   └─ render         # Jinja → HTML cached
                       # status=complete · expires_at = retention_for(tier)
   ↓
CLI opens report URL  # /reports/{base32_id}

Eight choices that shape the system.