# v2.0 real-corpus run runbook

This runbook is the maintainer-side recipe for the real-corpus
validation that gates v2.0 tag. The aggregator script is committed
and reusable; the corpus itself stays gitignored.

## Prerequisites

- A populated `validation/corpus-private/` directory with a
  consolidated corpus list (e.g.,
  `validation/corpus-private/consolidated.txt`).
- Network access for live DNS, CT, and identity-discovery queries.
- A clean checkout on the v2.0 release candidate commit
  (v1.9.11 plus any v1.9.11.x patches).

## Step 1 — Run the corpus scan

```
python validation/scan.py \
    --corpus validation/corpus-private/consolidated.txt
```

Output lands at
`validation/runs-private/<UTC-stamp>/results.json`. Per-domain
JSON is gitignored.

## Step 2 — Aggregate the firings

```
python validation/corpus_aggregator.py \
    validation/runs-private/<UTC-stamp>/results.json \
    --output validation/runs-private/<UTC-stamp>/aggregate.json
```

The aggregator emits only counts (no apex names). Shape is the
same as `validation/synthetic_corpus/aggregate.json`. Sanity-check
the output looks reasonable before proceeding.

## Step 3 — Populate the result shell

Open `validation/v2.0-corpus-run.md` (the result-shell shipped in
v1.9.11). Each section header has a placeholder line; replace each
with the corresponding number from the aggregator output. Sections:

- **Corpus size and Bayesian-node firing counts** — from the
  aggregator's `multi_cloud`, `ceiling`, `per_stratum` sections.
- **Per-stratum firing rates on real corpus** — direct copy from
  `per_stratum`.
- **Comparison to synthetic corpus shape** — for each stratum,
  whether the real-corpus firing rate is within ±15 percentage
  points of the synthetic rate. If a stratum diverges by more,
  document why (catalog gap, corpus skew, etc.).
- **`okta_idp` firing count** — from the per-node firings (the
  aggregator may need a `--include-bayesian-node-counts` flag if
  v1.9.11 adds it; otherwise compute from `results.json`).
- **Date and corpus snapshot timestamp.**

## Step 4 — Commit

```
git add validation/v2.0-corpus-run.md
git commit -m "v2.0 prep: real-corpus run results"
git push origin main
```

## Step 5 — Verify the v2.0 gate

The v2.0 release workflow runs a check that
`validation/v2.0-corpus-run.md` does not contain placeholder text
(`<TO BE POPULATED>`). Once populated and committed, that check
passes and v2.0 tagging proceeds.

## Honest framing

- **The aggregate counts are committed; the per-domain data is
  not.** This preserves the no-real-data discipline while giving
  v2.0 the truth-of-record numbers it claims to rest on.
- **If the real-corpus firing rates diverge significantly from
  the synthetic corpus,** document the divergence in the result
  shell. A 0% multi-cloud firing rate in the real Oracle stratum
  vs 22% on synthetic would suggest the synthetic fixtures
  over-modelled Oracle multi-cloud adoption, which is a fixture
  bug to flag rather than a production issue.
- **The runbook is reusable.** v2.1+ can re-run the same workflow
  with the same script. The result shell stays at
  `validation/v2.0-corpus-run.md`; per-version runs land in
  `validation/v2.N-corpus-run.md` files.
