One trace, one explorer, one re-run.

It's not missing functionality. It's missing connective tissue.

Six tabs, a sandbox, an eval matrix, and an audit log all exist. They each work in isolation. But a Super Admin who wants to ask "what happened during this scan, step by step?" bounces between four routes and still can't see deterministic-step durations or per-validation-request details.

What exists today

What a Super Admin can't currently do

One trace. One explorer. One re-run.

Every redesign decision answers to these three primitives.

What the new overview looks like.

Replaces the existing overview tab. Renders by joining scan_events (timing), llm_calls (model + cost + S3), and validation_runs (per-endpoint data) on scan_id, ordered by created_at. No new table, no schema change.

The trace shape is computable today. Step durations come from pairing started / complete rows on scan_events. LLM rows nest under their parent step by matching llm_calls.created_at to the enclosing step window. Validation phases collapse into one parent validation row with per-endpoint detail surfaced inside the Explorer.

One inspector, four renderings.

Every Explore → opens the same URL shape (/debug/scans/{scan_id}/steps/{step_idx}) but the renderer branches on step type. The renderings differ; the chrome and Re-run panel are shared.

4·a · LLM step (synthesis, intent_filter, flow_confirm, notes, difficulty_drivers)

# Loaded from s3 lazily on first view
You are a senior scraping consultant…

The user has captured network traffic from a target site.
Produce a JSON object with three top-level keys:
  - recommendation
  - verdict
  - starter_code

[truncated · 4,124 tokens · expand ↧]

# 27,838 tokens · 90% cache hit
target_url: https://www.officedepot.com
intent: Get all categories. Then for each category, get
         all products and then for each product, get data
         and reviews

detection_findings:
  - vendor: akamai_bot_manager
    severity: presence
    confidence: 1.0
    evidence: [_abck, ak_bmsc, bm_sz, bm_sv]

[truncated · 27,838 tokens · expand ↧]

{
  "recommendation": {
    "primary_library": "curl_cffi",
    "confidence": 0.86,
    "proxy_type": "residential",
    "cost_band": "$0.40-$2.50 per 1K",
    "rationale": "Akamai with full sensor stack..."
  },
  "verdict": { ... },
  "starter_code": { ... }
}
[expand raw response ↧]

4·b · Validation step

METHOD  GET
URL     https://www.officedepot.com/api/v3/catalog/categories
LIB     curl_cffi ▾      // editable
PROXY   residential ▾    // editable
HEADERS
  user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7)…
  accept: application/json
  accept-language: en-US,en;q=0.9
  x-request-id: [scrubbed]
  x-csrf-token: [scrubbed]          // supply value to fire

EXPECT  status 200 · ~12 KB JSON · top-level keys: [categories, meta]

4·c · Deterministic step (detection, analysis, scrub, render)

4·d · Human gate (awaiting_confirmation)

4·e · Scrub step

The scrub step strips secrets from the captured blob before anything downstream reads it. The Explorer surfaces the manifest deltas so an admin can audit what got redacted (and crucially, what didn't).

4·f · Replay step (pipeline replay, step 12)

After synthesis the pipeline fires the recommended endpoints through the recommended library to confirm the advice still works against the live site. This is distinct from per-request replay inside the Validation Explorer (§4·b) — that's an admin-triggered diagnostic; this is an automatic verification baked into the pipeline.

4·g · Error · partial · cancelled · in-progress states

The Trace renders different visual states depending on the scan's outcome. The admin-scans list (per decision below) filters to terminal states only, but the Explorer route is direct-URL accessible — so we handle the edge case too.

Always an experiment. Never a mutation.

Every re-run lands in an experiment table (existing LlmExperiment for LLM steps; proposed validation_replays for validation requests). The original LlmCall / ValidationRun row is immutable. The Explorer reads both and renders them side by side.

Re-run mechanics per step type

Every test is stored and visible

• LLM re-runs: appear as rows in the "Comparisons" table under the step they were forked from (see §4·a mockup). Plus an admin-wide index at /admin/experiments listing every experiment across every scan.
• Validation replays: appear in the "Replay history" panel under each endpoint card (see §4·b mockup). Plus filterable in the cross-scan index.
• Deterministic re-runs: diff against the original output, shown inline below the step's data.

Delete policy

Experiments are cheap (gzipped JSON in S3 + a small DB row) and they're evidence. Default behaviour is keep. But admins occasionally want to clear noise — failed prompt edits, accidental fires. So:

Quick-compare shortcut

The LLM Explorer (§4·a) shows four buttons — + haiku-4-5, + opus-4-7, + gpt-5, + grok-3 — that fire a re-run with the original prompts unchanged. This replaces the entire eval-matrix UI for the common case: "how would this prompt look on a cheaper / smarter / different-provider model?"

Existing surfaces, redistributed.

Nothing is deleted. The infrastructure that backs sandbox and evals (LlmExperiment, LlmEval, eval_runner, eval_lock, daily cost cap) is reused as the engine behind the Step Explorer's Re-run panel.

What "evals" actually means after the redesign

Today the word eval means two things at once: "compare this scan's prompts across models" (a single-scan question) and "run a fixture-based regression test of our prompts" (a cross-scan question). The redesign splits them:

• Single-scan model comparison → built into the Step Explorer. No special label needed.
• Fixture-based regression eval → keeps the /admin/evals route, but the landing page explains: "Pick a prompt and a fixture set. Run all current models against it. See cost / quality / drift over time." Plain-language in-page explainer above the matrix.

The Step Trace is a view, not a new table.

Most of the data the Explorer needs is already in the DB — scan_events for timing, llm_calls for prompts and cost, validation_runs for per-endpoint data. The Trace and Explorer ship without a single schema change. Only the new admin-triggered features (replay, deterministic re-run) need new tables.

7·a · What's already stored

7·b · What we add

Four phases. Each ships standalone.

High-level shape here. The task-by-task TODO list lives in the companion document: html_docs/admin-ui-rebuild-plan.html.

Parallel to this, an independent track ships the LLM Models Registry — an admin page to manage every model (company, version, pricing, frontier/reasoning flags, usage stats). Documented in the companion build plan.

What we've already settled.

Every question raised in earlier revisions has been answered. Locked-in decisions below — each shapes one or more build-plan tasks.

Still genuinely open

None — every question raised has been resolved. New decisions get logged here as they're made.