§5 — Rotation 1 halted on reviewer-quorum outage (2026-07-18)
Evergreen: model-selection-and-global-refine §5.
Rotation 1 of the stratified docs campaign (f-docs-campaign-rotations) was launched and halted by the orchestrator after ~6 minutes. The stratified-rotation-with-checkpoint design exists precisely to catch storms early — this rotation caught one: a live upstream outage of two of the three RD-quorum reviewer models. Halting bounded degraded-quorum accepts to ~27 (vs the ~200 a full rotation would have written) and preserved a valid healthy-path checkpoint baseline for the re-run. This is not a campaign defect — the engine behaved correctly; the blocker is external LLM-provider availability.
Quick status
| Item | Status | Detail |
|---|---|---|
| Rotation 1 command | launched | sn run --source dd --campaign "prose,audit:latex,audit:spelling,audit:length" --campaign-pilot 200 --include-accepted --campaign-batch-cost-cap 50 --campaign-cost-ceiling 120 |
| Selection at launch | 1,437 docs-fixable → stratified pilot 200 | 2 batches of 100, distributed across 17 domains |
| Outcome | ⛔ halted ~6 min in | 2 of 3 RD-quorum reviewer models in an upstream empty-response outage; 67% of reviews degraded to single-reviewer |
| Partial writes before halt | ~27 accepts | ~54 generated · 45 reviewed · ~27 refined+accepted · 63 StandardNameReview nodes |
| Spend (approx, interrupted) | ≈ $6.17 | cost_is_exact=False — run interrupted before cost finalization; ≈ $1.43 (23%) wasted on the two dead reviewer models' empty responses |
| Selection after halt | 1,409 (−28) | The ~27 fixed names dropped out of the defect predicates — sticky (a re-run will not revisit them) |
| Re-run | queued | f-docs-campaign-rotations stays open; re-run when OpenRouter reviewer availability recovers |
Root cause — upstream reviewer-quorum outage
The docs review stage runs a 3-model RD-quorum (review-and-defend). During rotation 1 two of the three models returned empty completions (finish_reason='stop', no content) across dozens of names, sustained for the whole run window and surviving the 5-attempt retry ladder:
| Reviewer model | Quorum role | Status during rotation 1 | Failed cycles | Spend (wasted) |
|---|---|---|---|---|
openai/gpt-5.6-luna | primary | ✅ working (also the docs-gen seat) | 0 | $2.28 (productive: gen+review) |
qwen/qwen3.7-max | secondary | ❌ empty responses | 34 | $1.07 |
minimax/minimax-m3 | tertiary | ❌ empty after 5 retries (ERROR+traceback) | 19 | $0.36 |
openai/gpt-5.5 | authoritative escalation + refine seat | ✅ working | 0 | $2.46 (productive: refine+tie-break) |
15 of the failures exhausted the retry ladder and surfaced as ERROR — LLM failed after 5 attempts. This is the same class of external transient the plan already noted at the §2 launch day (K3 unavailable on 429s) — a provider-side availability problem, not a defect in the campaign engine.
Why the quorum degradation is a checkpoint miss
With two secondaries down, the RD-quorum collapsed toward single-reviewer decisions:
single_review (1 model)30quorum_consensus (≥2)12authoritative_escalation3The plan's §6 risk register names "review quorum unchanged" as the load-bearing mitigation against refine churn writing degraded documentation into the accepted catalog. A 1-of-3 effective quorum does not satisfy that invariant. Two consequences drove the halt decision:
- Quality risk (hard to reverse): each accept is an accepted-name documentation rewrite on the live catalog. Under a full rotation ~200 would have landed under the weakened gate; halting bounded that to ~27. The primary reviewer (Luna, the strongest model and the production docs-gen seat) still gated every accept and the deterministic ISN audit re-stamped each batch, so the 27 are not unreviewed — but they lack the two secondary quorum votes.
- Invalid baseline: a checkpoint's acceptance rate / cost-per-name only means something under the intended quorum. Numbers gathered mid-outage cannot certify the healthy-path rotation cadence, so continuing would have burned ~$40 for an uninterpretable result.
The ~27 sticky accepts
The names accepted during the outage window had their banned prose removed, so they no longer match the campaign's defect predicates and a re-run will not re-select them. They carry full provenance — StandardNameReview nodes stamped method=single_review plus DocsRevision events — so they are individually identifiable if the lead wants them re-reviewed under a restored quorum. Recommended disposition: leave them (primary-reviewer + score-gate + deterministic-audit accepted) unless a spot-check of their diffs surfaces a quality concern; a targeted re-review query on method=single_review reviews since 2026-07-18 can force them back through the quorum if desired.
Checkpoint verdict (rotation 1)
| Gate axis | Threshold | Rotation 1 | Verdict |
|---|---|---|---|
| Review quorum intact | 3-model RD-quorum | 1-of-3 effective (67% single_review) | ❌ MISS — external outage |
| Docs acceptance | ≥ 90% | not validly measurable under degraded quorum | — void |
| Banned prose reintroduced | 0 | 0 observed in the partial window | PASS (partial) |
| Name-identity drift | 0 | 0 (docs-only refine events on unchanged ids) | PASS (partial) |
| Refine claim-race waste (gate a) | < 5% | 0% observed — the launch-gate fix held; waste here was upstream reviewer spend, a different axis | PASS |
| Cost/name | ≈ $0.20 | not validly measurable (retry waste from outage) | — void |
Disposition: HALT + re-run. The miss is on an external precondition, so the response is to re-run rotation 1 unchanged once the two reviewer models recover — not to change the campaign. The stratified selection is self-pruning, so the re-run re-stratifies over the remaining 1,409 and the ~27 already-fixed names stay out.
Incidental catalog-health signals (startup ledger, pre-existing)
The run's always-on startup audits surfaced pre-existing conditions unrelated to the campaign, logged here for the record: 24 live names with no recoverable producing source; 10 live names still fed by DD paths absent from the current DD (re-anchor or retire); and 2 flux_at_wall parents with heterogeneous child units (m^-2.s^-1 vs W.m^-2) skipped pending manual review. None block the campaign; each is a candidate for a separate provenance/units pass.
What's next
Re-run rotation 1 when OpenRouter reviewer availability recovers (verify with a 3-name smoke first — see the evergreen §5 NEXT card). The command and checkpoint criteria are unchanged; f-docs-campaign-rotations remains the driver. If the outage persists, the lead may elect a different --reviewer-profile (e.g. opus-only) for the re-run, accepting that it establishes a different quality baseline than the default 3-model quorum.