Two coupled deliverables. First, a complete role-by-role benchmark of the GPT-5.6 family (Sol, Terra, Luna) against the models currently holding each pipeline seat, so model selection is an evidence-based decision rather than a compose-only extrapolation. Second, a global-refine capability: a supported, budgeted way to run bounded quality campaigns over the accepted catalog (the immediate driver is the documentation-simplification backlog — ~1,250 accepted entries predating the strict-normative policy — plus the 129 accepted names carrying audit findings from the first full validation sweep). The two are sequenced deliberately: the refine campaign is the largest planned LLM spend, so the model seats it uses should be settled first — a cheaper, quality-equivalent refine model directly reduces the campaign's cost.
✓ rotations running clean 2026-07-20 — LLM-adjudicated gate, rotation 1 full 199/200 accept, no false-positive halt
§5 — Documentation-simplification campaign (pilot → scale)
Pilot converged 2026-07-17 (96% acceptance, zero prose reintroduction, zero drift; §5 pilot landed · before/after evidence), then forensics exposed two launch gates — both now fixed and validated live (§5 launch gates landed).
Gate (a) refine claim-race livelock (53% of pilot spend): per-node claim_seq + settle re-read in both winner verifiers, replica cap ceil(scope/2) on scoped drains, paid-call-without-persist tripwire (imas-codex af69e3e2).
Gate (b) stale audit stamps: every campaign batch now re-runs the deterministic ISN audit through an id-scoped LLM-free drain (fa503f83); the 25 pilot names re-stamped — perturbed_particle_energy correctly re-quarantined.
All 4 unit-defect quarantines resolved at root: 3 parents had unit '1' mis-inherited from normalized_* children (seeder exclusion e73331a9 + scoped repair db261bca/fab5af85 + docs re-aligned via sn edit, all accepted); the 4th was an audit false positive on the canonical transport-velocity form (exemption b4e38135). All four now valid.
Smoke (fresh stratified 25, run 9aa921c3): 24/25 accepted, 0 reintroduction, 0 drift, refine waste 0.0%, re-audit 25 cleared/0 re-quarantined, $0.18/name — confirms the ≈$300–380 basis for the 1,540-name docs-fixable spec.
Gate hardened to LLM adjudication (2026-07-20). Two earlier rotation attempts halted on the convergence
gate — first a reviewer-quorum outage (rotation 1 halted;
resolved by the resilient claude-sonnet-5 + grok-4.5 blind pair + AIMD governor, see
SN model selection + reviewer-resilience), then a prose-predicate false positive
(rotation 1 re-run: the estimator_recipe
pattern over-flagged legitimate "is derived from [related quantity]" provenance). Rather than chase regex arms, the banned-prose
convergence gate is now LLM-adjudicated (imas-codex 1c2e166a + fbd6a51b): a grep flags a
candidate, a quorum-independent adjudicator (gpt-5.6-luna) rules each flag legitimate-or-not, so definitional prose no
longer halts the run. Seats held; lead accepts ≈$0.29/name (premium blind pair).
Rotations running clean — see the rotation checkpoint ledger.
Docs campaign COMPLETE — docs-axis defects fully drained (1,047 → 0) across five clean rotations, no
false-positive halt — R1 (200: 199, $0.292, 1,047→889), R2 (450: 446, $0.274, 889→502), R3 (450: 441, $0.336, 502→115),
R4 (91: 87, $0.35, 116→59), R5 (9: 9, docs-fixable 9→0). Across all: gate adjudicated every grep flag as legitimate (0 genuine
reintroductions), 0 name drift, 0 outages, refine waste <5%, quorum intact; every checkpoint axis PASS. A retry-classification
gap on upstream 500s was fixed en route (cbd9cb6f). The genuine residue is 6 US-spelling renames
(*_strain_gauge→*_strain_gage), routed through the normal compose→review pipeline (never direct-accepted);
the other 52 flags were an artifact of an arbitrary 70-char length cap now removed as an anti-pattern (imas-codex
7e583184) — a name is as short as possible while retaining full meaning, and no shorter, so length is not a defect.
Full ledger: rotation checkpoint ledger.
Expert-review reply logged (2026-07-21): the campaign change log now lives in
expert-review-reply §5 (every change carries a SHA or graph-event reference) and
review comment 1 is closed at scale; plan status → shipped. Remaining catalog work is release close-out (ISNC export + RC),
tracked in catalog-expert-review-remediation.
Follow-on policy plan: the refine seat annotates a unit on nearly every LaTeX symbol; units are recoverable
(structured unit field / linked names), so the non-redundant target is symbol identity, not units.
Scoped + priced in sn-symbol-definition-policy (prompt + latex_def_check
fix; ~$75–120 marginal cleanup, or $0 if absorbed by the running campaign). Not a halt condition for the current rotations.
§ Followups
Run the full GPT-5.6 role benchmarks
Priority work: extend sn bench with per-role modes (refine-from-critique, breaker independence, docs generation, classifier gold set) and measure Luna/Terra (+Sol for docs only) against each incumbent with production prompts and reasoning-effort settings. ≤ $40 total.
Project: imas-standard-names
Plan: model-selection-and-global-refine (http://localhost:8765/imas-standard-names/model-selection-and-global-refine.html)
Section: §2
Tier: opus
Context
Compose-only benchmarking (2026-07-17, imas-codex 80c1935) showed Luna quality-tied at 41% lower cost and Sol overpriced for structured tasks. Each remaining pipeline seat needs its own measured row before any switch.
State to read (CODE / FILES / DATA)
imas-codex: imas_codex/standard_names/benchmark*.py, sn bench CLI, pyproject [sn-refine]/[sn-review]/[sn-docs] model config, StandardNameReview records on the live graph (read-only) for the refine-from-critique corpus, definitions/physics/domain_gold_set.json.
Scope locks / constraints (non-decision)
Extend sn bench (no one-off scripts); model-id-based identifiers; production prompts + per-seat reasoning-effort; held-out samples for judging; budget ceiling $40 across all roles; no pipeline seat/config switches in this followup — evidence only. Live graph reads OK, no writes.
Done-when
1. per-role benchmark modes committed with tests; measured table (quality + cost) per §2 row in the benchmark reference
2. switch/hold/needs-more-data recommendation per seat recorded in the plan
3. followup written into plan + this one resolved
Harness landed (imas-codex 23e6f99d, d4994f97) + refine-context fidelity fix 0a05c173 (bench was starving the refine prompt of the closed-vocabulary listing production injects). Execution NOT complete: the post-fix refine run saved an EMPTY report (n=0 all models, sn_rolebench_refine_20260717T090827.json) and the remaining 4 roles are unrun. Superseded by f-role-benchmarks-2 which carries the diagnosis brief.
Design review: global-refine selector, gates, and surface decision
Bring the §4 design to the lead with the refine-surface and derived-parent-parse-gate decisions; include the reviewed dry-run manifest shape and the convergence-gate thresholds. No implementation before the decisions lock.
Project: imas-standard-names
Plan: model-selection-and-global-refine (http://localhost:8765/imas-standard-names/model-selection-and-global-refine.html)
Section: §4
Tier: opus
Context
The docs-simplification backlog (~1,250 entries) plus 129 audit-flagged accepted names need a bounded campaign engine. Guardrails: same pipeline, no naive regeneration, root-fix before bulk, pilot before scale.
State to read (CODE / FILES / DATA)
imas-codex sn run --families / DocsRevision machinery, edits scope, validation sweep results (plan research r-validation-backlog-swept in catalog-expert-review-remediation), review-pool quarantine skip behaviour.
Scope locks / constraints (non-decision)
No accept path bypassing review; docs campaigns never touch name identity; campaign manifest reviewed before execution; batches idempotent + abortable; convergence gate halts on threshold miss.
Done-when
1. surface + derived-parent decisions locked by the lead
2. selector/gate design recorded with thresholds and manifest shape
3. implementation followup queued
§4 landed — design executed under the locked extend-sn-run decision: campaign engine a1c8ca1e (selector/manifest/batching/convergence gate/provenance, 44 tests), derived-parent gate rescope + review-context injection 93276ebf (52/52 parse quarantines clear read-only, 4 genuine unit defects retained), revalidate sweep fix 68f8fa65; parse-error renames accepted 0.91/0.96. Archive: docs/archive/model-selection-and-global-refine-s4-landed.html
Lead: run the curative revalidation sweep (classifier-gated)
The fixed gate is live and the --revalidate sweep now covers quarantined names (68f8fa65), but the pooled run is blocked by the session permission classifier. The lead runs one command; 436 quarantined names re-stamp under the current grammar, clearing ~58 derived parents and the freshly renamed pair.
Run in imas-codex on main (or via ! in a Claude session):
imas-codex sn run --source dd --revalidate --only validate
Then verify: quarantined origin='derived' count should drop 62 -> 4 (residual genuine unit defects: effective_particle_energy, particle_mass, perturbed_particle_energy, effective_thermal_ion_charge_state_energy_velocity_due_to_convection). Record the post-sweep counts in the plan §4 landed summary.
Partially executed and superseded: first run exposed two more surface bugs (sweep dead code below pool routing — fixed 3a562921; pools have no validate worker — --only validate now routes to a dedicated LLM-free drain, 363a6ee1; legacy IMASNode source-scoping excluded 330/436 including all 62 derived parents — dropped in b68b1e52). Two interim runs re-confirmed 106 genuine quarantines idempotently. Final re-run still pending — superseded by f-revalidate-rerun.
Relocate banned-prose vocabulary out of benchmark_roles.py
campaign.py imports BANNED_PROSE_PATTERNS/banned_prose_findings from benchmark_roles.py — a benchmark module owns production selection vocab. Move to a neutral prose_policy module; both consumers import it.
Project: imas-standard-names / code: imas-codex
Section: §4 follow-on (code hygiene)
Tier: sonnet
Context
The campaign selector and the docs role benchmark share the banned-prose vocabulary, currently owned by imas_codex/standard_names/benchmark_roles.py. Production selection logic must not depend on a benchmark module.
Work
Create imas_codex/standard_names/prose_policy.py holding BANNED_PROSE_PATTERNS + banned_prose_findings; benchmark_roles.py and campaign.py import from it (re-export from benchmark_roles for back-compat if anything else imports it). Update tests.
Done-when
1. No production module imports vocab from benchmark_roles
2. test_campaign.py + test_benchmark_roles.py green
3. Committed and pushed to imas-codex main
Landed — imas-codex a469501d: imas_codex/standard_names/prose_policy.py now owns BANNED_PROSE_PATTERNS + banned_prose_findings; campaign.py and benchmark_roles.py import from it (back-compat re-exports retained in benchmark_roles). test_campaign + test_benchmark_roles 85/85 green.
Diagnose the empty refine bench report, then run all five role benches
GATES THE DOCS-REFINE CAMPAIGN (expert-review response). The sn bench --role harness is committed but the refine run judges zero cases and saves a hollow report; the other four roles are unrun. Diagnose, make n=0 fail loudly, re-run all five, write docs/sn-role-benchmarks.md with per-seat verdicts.
Project: imas-standard-names / code: imas-codex (branch main)
Plan: model-selection-and-global-refine §2
Tier: opus
Context
Role-bench harness committed: 23e6f99d (modes) + d4994f97 (corpus keys) + 0a05c173 (refine context now merges build_compose_context — without it the grammar-reference include rendered empty and every candidate invented unregistered tokens). Post-fix refine run STILL produced an empty report: ~/.local/share/imas-codex/benchmarks/sn_rolebench_refine_20260717T090827.json has n=0, cost=0, metrics={} for gpt-5.5/luna/terra despite 20 sample_ids. Suspects: (a) key mismatch between load_refine_corpus output and run_refine_bench after the 0a05c173 seam change; (b) every candidate failing RefinedName validation and being dropped uncounted; (c) judge-stage errors swallowed per-case.
Work
1. In imas_codex/standard_names/benchmark_roles.py: find why every model uniformly judged 0 cases; fix; make run_*_bench RAISE (or set report.error) when a model ends with n=0 — an empty report must never render as a clean table. Test the tripwire.
2. Run sequentially with log capture + exit sentinels, $40 total ceiling (cost telemetry from OpenRouter response_cost):
uv run imas-codex sn bench --role refine (verify n>0, cost>0 BEFORE proceeding)
then --role breaker-names, breaker-docs, docs, classifier
3. Write /home/ITER/mcintos/Code/imas-codex/docs/sn-role-benchmarks.md: summary table + per-seat measured tables (quality metrics, cost/item, incumbent marked) + verdict per seat: SWITCH(to X) / HOLD / NEEDS-MORE-DATA. Note the two harness fidelity fixes in a methods section.
4. Commit+push (explicit paths; no AI trailers; no plan refs in messages).
Done-when
1. five non-empty JSON reports on disk; docs/sn-role-benchmarks.md pushed
2. verdicts recorded in the plan (§2 collapse + seat-switches decision context)
3. this followup resolved; §3 seat-decision batch queued for the lead
All six seats measured (~$36): refine HOLD, names-breaker SWITCH luna, docs-breaker HOLD, docs-gen SWITCH luna, compose HOLD DSv4-local (free; tie with luna at n=17), classifier HOLD gpt-5.4 (82.3% vs luna 79.4%). Seat switches applied ce0144e1; decision locked. K3 unavailable launch-day (upstream 429s). Results doc: imas-codex docs/sn-role-benchmarks.md (final compose/classifier rows folding in). Archive: docs/archive/model-selection-and-global-refine-s2-s3-landed.html
Lead: re-run the revalidation sweep (now reaches the full backlog)
All four revalidation-surface fixes are on imas-codex main (68f8fa65, 3a562921, 363a6ee1, b68b1e52). One command re-stamps ~436 quarantined names under the fixed derived-parent gate; classifier-gated, so the lead runs it.
Run (or via ! in a Claude session):
cd ~/Code/imas-codex && uv run imas-codex sn run --source dd --revalidate --only validate
Expect: ~436 cleared; drain re-stamps all; quarantined origin='derived' drops 62 -> 4 (genuine unit defects: effective_particle_energy, particle_mass, perturbed_particle_energy, effective_thermal_ion_charge_state_energy_velocity_due_to_convection). Verify with a quarantined-by-origin count and record post-sweep numbers in the §4 landed archive.
Sweep executed 2026-07-17 (classifier allowed the LLM-free drain): 436 pending/quarantined re-stamped. Quarantined origin='derived' dropped 62 → 10 — the 4 expected unit defects plus 6 names whose latex_def_check/decomposition_audit findings are genuine (correct quarantines; they are docs-campaign targets, e.g. angle_of_iron_core_segment, ion_current_density). Accepted-stage quarantines now 125 total (109 catalog_edit + 10 derived + 4 pipeline + 2 none); remainder of the 384 re-quarantined are superseded/exhausted terminal-stage residue outside campaign scope. Campaign selection shrank 2,332 → 2,288.
Produce the expert-review reply document logging ALL changes
Lead mandate (2026-07-17): the expert catalog review needs a reply document that logs every change made in response — the §4/§5 campaign work here, the derived-parent gate rescope, the parse-error triages, the provenance repairs, and the docs-refine campaign results once run. Gated by the §2 bench verdicts → §3 seats → §5 campaign chain; draft structure can start now from the §4 landed archive.
Project: imas-standard-names
Plans: catalog-expert-review-remediation (reply doc home) + model-selection-and-global-refine (change log source)
Tier: opus
Context
The expert-review reply must enumerate every remediation change with evidence. Sources to fold in: docs/archive/model-selection-and-global-refine-s4-landed.html (campaign engine, gate rescope, parse-error triage, provenance-split repair, revalidation fixes with imas-codex SHAs a1c8ca1e/93276ebf/68f8fa65/3a562921/363a6ee1/b68b1e52 + repair change events), the docs-campaign pilot+batch results once §5 runs, post-sweep quarantine counts, and the existing reply-thread docs in the remediation plan (open item 1 = docs simplification).
Work
Author the reply as HTML in docs/ per reckon conventions (graphics where they communicate: before/after quarantine counts, campaign convergence per batch). Cross-link both plans. Close open item 1 of the prior reply thread when the campaign lands.
Done-when
1. reply document authored + audit-doc clean + pushed
2. every change in the log carries a SHA or graph-event reference
3. remediation plan followup resolved; RC release unblocked
Reply document authored — docs/expert-review-reply.html §5 is now a complete change log of the documentation-simplification campaign (review comment 1 at scale), every change carrying a commit SHA or graph-event reference (root-fixes 4b36befa/93276ebf/68f8fa65+3a562921+363a6ee1+b68b1e52/6ef0ab2c/decomp-audit-retirement e3d20e11→c6191e61→5a08c744/unit-root-fixes e73331a9+db261bca+fab5af85+b4e38135/prose-exemption 6c6ffec8→f1a4fbbc; engine a1c8ca1e/a469501d/af69e3e2/fa503f83/1c2e166a+fbd6a51b/cbe551cc/cbd9cb6f/3fefe5a8; seats ce0144e1). Disposition figure, point-1 cell and §4 open-items updated to close comment 1; both plans cross-linked; rotation ledger + role-benchmark archives linked. audit-doc clean; committed+pushed b42cb670. Docs-axis defect set drained 1,047→0 across 5 rotations (≈$365, 0 prose reintroduction, 0 name drift). Remaining is release close-out (ISNC export + catalog RC) plus 6 US-spelling renames (strain_gauge→strain_gage, via the compose→review pipeline, never direct-accepted); the 52 over-length flags were dropped when the arbitrary 70-char length cap was removed as an anti-pattern (imas-codex 7e583184). Both tracked in catalog-expert-review-remediation (f-final-editorial-export / f-close-out-residuals) — neither is an open review comment.
Harden RefinedName.kind: enum the schema field, enumerate kinds in the refine prompt
Bench diagnosis exposed a latent production fragility: RefinedName.kind is a free string with no schema enum, and the refine prompt never enumerates scalar/vector/metadata — the only in-prompt kind signal is the scored-examples include. If examples ever fail to load, production refine degrades to mass kind-validation failures exactly as the bench did (60/60 dropped).
Project: imas-standard-names / code: imas-codex (branch main)
Section: §2 follow-on (production hardening)
Tier: sonnet
Context
See docs/sn-role-benchmarks.md methods section (bench-runner flag, 2026-07-17): RefinedName.kind free string, refine prompt omits kind vocabulary; production conveys kind only via the _compose_scored_examples.md include populated by load_compose_examples.
Work
1. Constrain RefinedName.kind to the valid kind literals (scalar/vector/metadata — confirm the authoritative set from the ISN schema) so invalid kinds fail at parse with a clear message.
2. Enumerate the valid kinds explicitly in the refine prompt template so the signal does not depend on the examples include.
3. Regression test: refine context WITHOUT scored examples still yields schema-valid kind on a fixture case.
Done-when
1. enum + prompt change + test committed and pushed
2. bench refine mode re-run smoke (3 cases) confirms no regression
Implemented at source, wider than scoped (lead directive: never rely on examples for format — they are complementary): imas-codex 6ef0ab2c unifies the kind vocabulary on the ISN catalog Kind enum {scalar, vector, tensor, complex, metadata}. LinkML StandardNameKind trimmed to mirror ISN exactly (eigenfunction/spectrum retired — semantic categories, structurally scalar, zero live names); RefinedName.kind + StandardNameCandidate.kind are non-nullable Literals so the JSON schema carries a flat enum and structured output cannot emit an invalid kind; compose + refine prompts enumerate the kinds explicitly; derive_kind/recompute_parent_kind aligned; legacy graph values still collapse to scalar at export; three-way consistency test pins Literal == LinkML-generated enum == ISN Kind. Note: live compose already injected kind_definitions from ISN — the generate/import pattern was present; the LLM response models and refine prompt were the gaps.
Lead: approve the docs-campaign scale-up, then run the 23 batches
Pilot converged (96% acceptance, zero prose reintroduction, zero drift) but measured cost is $0.37–0.42/name → ≈$850–960 for the full 2,288-name selection, well above the pre-pilot guess. The lead picks: full spec (~$900), prose-only spec (1,384 names ≈ $520–580, defers pure-audit fixes), or domain tranches. Then one command runs the batches under the convergence gate.
Decision inputs: docs/archive/model-selection-and-global-refine-s5-pilot-landed.html (pilot metrics, per-phase cost, before/after samples, alternatives table).
If approved as piloted, run (or via ! in a Claude session):
cd ~/Code/imas-codex && uv run imas-codex sn run --source dd --campaign all --include-accepted --campaign-batch-cost-cap 50 --campaign-cost-ceiling 1000
Prose-only alternative: replace --campaign all with --campaign prose (1,384 names).
The campaign halts between batches on any gate miss (≥90% acceptance, zero banned-prose reintroduction, zero name drift) and prints --campaign-resume-from for the restart. After completion: verify zero accepted names match the prose predicates (sn run --campaign prose --dry-run → total 0), record final counts + spend in the §5 archive, then close expert-review reply open item 1 via f-expert-review-reply-doc.
Superseded by f-docs-campaign-batches-2: the audit-class breakdown (2,125 of 2,288 matched names carry ONLY name-axis decomposition findings a docs pass cannot clear) sharpened the recommendation to the docs-fixable spec — prose,audit:latex,audit:spelling,audit:length = 1,540 names / 16 batches ≈ $570–650. Full before/after evidence doc authored: docs/docs-campaign-before-after.html.
Lead: approve the docs-campaign scale-up (recommended spec: docs-fixable defects, 1,540 names ≈ $570–650)
Pilot converged (96% acceptance, zero prose reintroduction, zero drift; $0.37–0.42/name). Audit-class analysis shows the 'all' spec would spend ~$350 on ~750 names whose only defect is a name-axis decomposition finding a docs campaign cannot clear — recommended spec is prose + docs-axis audit checks: 1,540 names, 16 batches, ≈$570–650. Evidence: docs/docs-campaign-before-after.html (weakness taxonomy, before/after pairs, mechanism mapping).
Decision inputs: docs/docs-campaign-before-after.html (weaknesses, before/after evidence, spec comparison) + docs/archive/model-selection-and-global-refine-s5-pilot-landed.html (pilot metrics, per-phase cost).
Recommended (docs-fixable defects only), run (or via ! in a Claude session):
cd ~/Code/imas-codex && uv run imas-codex sn run --source dd --campaign "prose,audit:latex,audit:spelling,audit:length" --include-accepted --campaign-batch-cost-cap 50 --campaign-cost-ceiling 700
Alternatives: --campaign prose (1,504 names, leaves 36 docs-axis audit names for hand edits); --campaign all (2,288 names ≈ $850–960, adds ~750 name-axis-only names whose finding cannot clear).
The campaign halts between batches on any gate miss (≥90% acceptance, zero banned-prose reintroduction, zero name drift) and prints --campaign-resume-from. After completion: verify sn run --campaign prose --dry-run → total 0; record final counts + spend in the §5 archive; re-scope decomposition findings to the name-axis workstream explicitly; then close expert-review reply open item 1 via f-expert-review-reply-doc.
Superseded by f-docs-campaign-rotations: lead directive 2026-07-18 — run high-spend docs work in stratified rotations with an orchestrator review checkpoint between each, so storms like the refine claim-race livelock are caught after ~200 names, not 1,540. Cost basis also corrected: healthy-path ≈ $0.20/name post-livelock-fix → recommended spec ≈ $300–380 total.
LAUNCH GATE: fix the refine_docs claim-race livelock (53% of pilot spend wasted)
Pilot forensics (LLMCost run 02313084): 39 of 59 refine calls ($5.55 of $7.55) were paid gpt-5.5 calls whose persist_refined_docs no-oped with token/stage mismatch, concentrated on perturbed_particle_energy (27 calls) and vertical_coordinate_of_constraint_position (12). 20 refine replicas racing a 1–2-name eligible set defeat the _verify_docs_claim_winners window — each racer is momentarily the token holder at its own verify instant, all pay, only the last persist lands, and the sub-threshold score keeps the name eligible so the storm repeats.
Project: imas-standard-names / code: imas-codex (branch main)
Section: §5 launch gate (a)
Tier: opus
Evidence
Pilot log (persist_refined_docs: no-op … token/stage mismatch loops at 22:42–22:43) + LLMCost run 02313084 per-name refine call counts. Claim path: graph_ops.claim_refine_docs_batch → _claim_sn_atomic → _verify_docs_claim_winners; persist: persist_refined_docs (token+stage guarded — the guard is correct, the waste is upstream).
Work (in order of leverage)
1. Scale docs-pool replicas to scope size for scoped drains (scope_run_id set): replicas = min(configured, max(1, ceil(eligible/2))) — 20 replicas on a 25-name batch is the amplifier.
2. Close the verify race: after claim + verify, re-read the token once more after a short settle (or verify with a strictly-increasing claim sequence number) so at most one racer proceeds to the paid call.
3. Tripwire: count paid-call-with-failed-persist per pool per run; surface in the run summary and warn when >2%. Also rename/clarify the pool 'completed' counter semantics (it counts processed attempts, not persisted work).
4. Regression test with a mock pool: N concurrent claimers, 1 eligible item → exactly 1 paid call proceeds.
Done-when
1. re-run the 25-name pilot scope as a smoke (idempotent; names already accepted → tiny drain) with refine waste <5%
2. tests green, committed+pushed
3. this followup resolved; launch gate (b) f-campaign-audit-restamp still open or resolved before approval
Landed imas-codex af69e3e2: strictly-increasing per-node claim_seq stamped at claim + settle re-read in BOTH docs/names winner verifiers (superseded racer stands down before paying); docs-pool replicas capped at ceil(scope/2) on scoped drains (1-2-name set → 1 replica, zero concurrency); paid-call-without-persist tripwire in every run summary (warn >2%), per-pool 'completed' relabelled 'processed'. 17 new race tests. Smoke (fresh stratified 25, run 9aa921c3): refine waste 0.0% (15 paid calls, 0 wasted persists), $0.18/name vs $0.42 pilot — spend −56% at equal scope. Residual: settle reduces (not provably eliminates) the race for large unscoped runs; seed-level compare-and-set is the full close if the tripwire ever fires. Archive: docs/archive/model-selection-and-global-refine-s5-launch-gates-landed.html
LAUNCH GATE: re-stamp the deterministic audit after each campaign batch
The campaign's default_revalidate confirms lifted quarantines 'valid' on the banned-prose grep alone — the deterministic audit (latex_def_check, spelling, length, unit checks) is never re-run on refreshed docs. Pilot proof: all 25 pilot names carry validated_at timestamps predating the drain, and perturbed_particle_energy (genuine name_unit_consistency defect) is now stamped 'valid'.
Project: imas-standard-names / code: imas-codex (branch main)
Section: §5 launch gate (b)
Tier: sonnet
Context
campaign.py default_revalidate only re-quarantines on prose-grep reintroduction and confirms the rest valid. The LLM-free validation drain (sn run --revalidate --only validate machinery, 363a6ee1) already re-runs the full ISN audit and re-stamps validation_issues/validated_at — route each campaign batch through it.
Work
1. In CampaignRunner step 5, replace/extend default_revalidate: clear validated_at on the batch ids and invoke the validation drain scoped to them (injectable fn, mock-tested), so every refreshed doc gets a fresh deterministic audit stamp and genuine defects (e.g. unit inconsistencies) re-quarantine instead of washing to 'valid'.
2. Re-stamp the 25 pilot names the same way; verify perturbed_particle_energy re-quarantines on its unit defect and record post-stamp counts.
3. Keep the convergence gate reading the prose grep (fast) but add re-stamped audit deltas to the batch outcome for the archive.
Done-when
1. campaign batches leave no stale validated_at; test with a mock validate fn
2. pilot names re-stamped, counts recorded in the §5 archive
3. committed+pushed; this followup resolved
Landed imas-codex fa503f83: id-scoped LLM-free validation drain (drain_validation_for_ids) invoked in CampaignRunner step 5 on every touched batch id — clears validated_at then re-stamps through the full ISN audit before the prose re-quarantine lands on top; BatchOutcome/CampaignResult carry audit re-quarantined/cleared deltas; 95 tests. Pilot re-stamped live: 24 valid + perturbed_particle_energy correctly re-quarantined on its unit defect. The 4 unit-defect quarantines then resolved at ROOT: 3 parents had unit '1' mis-inherited from normalized_* children (seeder exclusion e73331a9, scoped repair db261bca/fab5af85, docs re-aligned via sn edit --axis docs, all accepted); the 4th was a name-unit audit false positive on the canonical transport-velocity form (exemption b4e38135). All four valid. Smoke batch loop: re-audit 25 cleared / 0 re-quarantined. Archive: docs/archive/model-selection-and-global-refine-s5-launch-gates-landed.html
Docs campaign in stratified rotations with review checkpoints (gated on both launch fixes)
After f-refine-claim-livelock and f-campaign-audit-restamp land: rotation 1 = stratified 200 names of the docs-fixable spec (~$40), then orchestrator review (accept rate, reintroductions, cost/name, paid-call-without-persist tripwire, sample diffs); then 3 rotations of ~450 (~$90 each) with the same checkpoint. Selection is self-pruning — fixed names stop matching the predicates, so each rotation re-runs the same command and --campaign-pilot re-stratifies over the remaining defect set.
Prereqs: f-refine-claim-livelock + f-campaign-audit-restamp resolved.
Rotation command (repeat until selection empties; N=200 first, then 450):
cd ~/Code/imas-codex && uv run imas-codex sn run --source dd --campaign "prose,audit:latex,audit:spelling,audit:length" --campaign-pilot N --include-accepted --campaign-batch-cost-cap 50 --campaign-cost-ceiling 120
Checkpoint after each rotation (orchestrator): accept rate ≥0.90, zero prose reintroduction, zero drift, cost/name ≈$0.20, refine paid-call-without-persist <5%, spot-check 5 before/after diffs across domains; record rotation row in the §5 archive. Halt + root-cause on any miss.
Done-when: sn run --campaign "prose,audit:latex,audit:spelling,audit:length" --dry-run → total 0; final counts + spend in §5 archive; then f-expert-review-reply-doc.
Docs campaign COMPLETE — docs-axis defects drained 1,047→0 across 5 clean rotations (R1 200/199, R2 450/446, R3 450/441, R4 91/87, R5 9/9), every checkpoint axis PASS (quorum intact, 0 drift, 0 outages, refine waste <5%), ~$365 total. LLM-adjudicated gate ran without false-positive halt. Only residue is 58 NAME-axis findings (52 name-length >70 chars + 6 *_strain_gauge gauge→gage) a docs campaign cannot clear — re-scoped to the name-axis workstream. From R4 on, docs ran on the new no-units prompt (sn-symbol-definition-policy shipped: 357-doc units cleanup → 0). Ledger: archive/model-selection-and-global-refine-s5-rotations-landed.html. Next: f-expert-review-reply-doc.
Deterministic decomposition re-parse triage (LLM-free) → sized rename queue
Verified 2026-07-18: flagged names parse cleanly under the current grammar with unchanged name strings (ion_current_density → subject=ION + base=current_density) — the stored decomposition is stale, not the name. A $0 script re-parses all 2,255 flagged accepted names and buckets: (a) re-parse slots the token → re-stamp stored segments + clear finding; (b) whitelisted lexicalised compound → suppress; (c) still absorbed post-parse (e.g. reference_magnetic_field) → rename queue, composed free on local DSv4 with review-only cost (~$0.06–0.10/name). Runs after canonical-qualifier-order residuals (f-cqo-003/004, f-cqo-normdir-shadow) so the parse is final.
Project: imas-standard-names / code: imas-codex (branch main)
Prereq: canonical-qualifier-order §3 residuals landed (grammar final).
Work
1. Script (sn CLI subcommand or audit-utils entry, not one-off): for each accepted name with a decomposition_audit finding, parse via imas_standard_names.grammar.parse_standard_name; bucket (a) token now slotted → update stored segment fields + re-stamp validation (LLM-free drain); (b) compound whitelist → suppress finding; (c) still absorbed → emit rename-queue manifest (name, absorbed tokens, suggested segmentation).
2. Dry-run manifest with bucket counts for lead review before any write; then apply (a)/(b) and re-run the validation drain.
3. Queue bucket (c) as a rename rotation: compose on local DSv4 (free), blind-pair review (only real cost), sn edit cascade mechanics, batched with checkpoints like the docs rotations.
Done-when: dry-run bucket counts recorded; (a)+(b) applied + re-stamped; rename queue sized with projected review cost; followup for the rename rotation queued.
SCRIPT + DRY-RUN MANIFEST LANDED (this run). Prereq met: canonical-qualifier-order §3 residuals are final (ISN d150b6c). Delivered imas-codex e3d20e11: imas_codex/standard_names/decomposition_triage.py (pure, LLM-free re-parse bucketing, unit-tested against real catalog names) + `sn decomp-triage` (read-only dry-run default; --apply path). The audit substring-scans a name's raw id, so it flags a closed token even when the grammar correctly slots it (ion_current_density→subject=ion) — confirmed the stored findings are stale, not the names. Deterministic rule: re-parse→ (a) DRAIN if the parsed physical_base has no leaked closed token (finding stale, clear+re-stamp); (b) SUPPRESS if a token remains inside a grammar-registered atomic base (convection_velocity, diffusion_coefficient) — legitimate; (c) RENAME if a token is absorbed into a non-registered compound (reference_magnetic_field, vacuum_magnetic_vector_potential). DRY-RUN over all 2252 flagged accepted names: drain 1809 / suppress 173 (1982 clearable free) / rename 270 / parse-fail 0 / non-canonical 0. Rename queue sized: ~$21.60 review (270 × $0.08; compose free on DSv4). Manifest: docs/archive/canonical-qualifier-order-manifests/decomp-triage-dry-run.json; full record docs/archive/canonical-qualifier-order-s3-landed.html. NOT applied — the write pass is gated on lead review of the bucket split per this followup's own design. Next queued: f-decomp-triage-apply.
Lead: apply the decomposition-triage free drain, then run the 270-name rename rotation
The dry-run manifest is signed off first (drain 1809 / suppress 173 = 1982 clearable free; rename 270 ≈ $21.60). Then: (1) apply the free drain (`sn decomp-triage --apply` clears the stale/lexicalised findings + re-stamps segment edges); (2) make decomposition_audit_check parse-aware (scan the parsed physical_base, exempt registered bases) so the drain sticks and buckets a/b never recur; (3) run the 270-name rename rotation — compose free on DSv4, blind-pair review, sn edit cascade mechanics, batched with checkpoints like the docs rotations.
Project: imas-standard-names / code: imas-codex (branch main)
Section: §5 follow-on — decomposition-triage apply + rename rotation
Tier: sonnet (opus for the audit-logic change)
Prereq: lead has reviewed docs/archive/canonical-qualifier-order-manifests/decomp-triage-dry-run.json bucket split.
Context
The re-parse triage (imas-codex e3d20e11, `sn decomp-triage`) buckets the 2252 stale decomposition_audit findings on accepted names: 1809 drain + 173 suppress (1982 clearable, no LLM, no name change) + 270 rename queue (genuine absorption). 0 parse-fail. This followup does the writes.
Work
1. FREE DRAIN (classifier-gated write): `cd ~/Code/imas-codex && uv run imas-codex sn decomp-triage --apply`. Clears the decomposition_audit lines from validation_issues for the drain/suppress buckets and re-stamps segment edges via rederive_structural_edges. Verify the flagged-accepted count drops ~2252 → ~270.
2. DURABLE AUDIT FIX (opus): make imas_codex/standard_names/audits.py::decomposition_audit_check parse-aware — scan the PARSED physical_base (not the raw id) and exempt a physical_base that is a grammar-registered atomic base. Add tests. This stops the drain/suppress class recurring on the next sn run (without it, clearing is futile). Commit+push.
3. RENAME ROTATION (270 names): the manifest's rename_queue carries per-name absorbed tokens + a naive segmentation suggestion. Compose the corrected names on local DSv4 (free), blind-pair review (the only real cost, ~$21.60), apply via sn edit cascade mechanics, batched with a checkpoint per ~90 names (accept rate, drift, before/after diffs) like the docs rotations. Some (c) entries may be legitimate atomic quantities the lead keeps (alfven_frequency, wave_electric_field) — the rotation adjudicates, does not blindly rename.
Done-when
1. free drain applied; flagged-accepted decomposition count down to the rename set; audit made parse-aware + tests green + pushed
2. rename rotation complete (or the residual keep-set recorded with rationale); final counts + spend in the §5 archive
3. this followup resolved
RESOLVED differently than scoped — the whole premise collapsed under the lead's controlled-base insight. The decomposition_audit was RETIRED (imas-codex 5a08c744), not made parse-aware, because it is OBSOLETE now that physical/geometric bases are a controlled vocabulary. Root cause: it compared a name's surface base PHRASE (model.physical_base, e.g. reference_magnetic_field, absorbed_power, nuclear_heating_power) against the registered base-TOKEN set — but the actual base token (ir.base.token: magnetic_field, power, heating_power) is ALWAYS registered (the parser raises UnknownBaseTokenError for a made-up base). The surface phrase legitimately carries glued kind-forming qualifiers (royal-order: ion_atomic_mass, wave_electric_field). So a composer cannot hide structure in the base slot — the failure mode the audit was built for cannot occur. Every one of its ~2252 findings was a historic false positive. There is NO rename rotation: the '270 rename queue' was itself false positives (kind-forming glue + binary-op nested args + upper_bound substring artifacts; verified 0 genuine after excluding the qualifier segment, skipping binary-ops, and exempting base-head tokens). Steps actually taken: (1) parse-aware audit landed as a stepping stone (c6191e61), free drain applied (1982 cleared, 2252→270), single-shot `sn decomp-triage` CLI+module DELETED per lead directive; (2) then the controlled-base realization → decomposition_audit_check + find_absorbed_closed_tokens primitive + tests removed, reviewer prompt reframed to 'base registry is grammar-enforced; emit vocab_gap when no base fits', all remaining decomposition_audit findings drained graph-wide (1641 cleared, 0 remain). Net: the decomposition audit and its scaffolding are gone; the base grammar itself is the guarantee. imas-codex commits e3d20e11 (superseded) → c6191e61 → 5a08c744.
Lead: narrow the estimator_recipe 'derived from' false positive, then resume rotations
Rotation 1's resilient re-run (run c355df39) validated the pipeline but the convergence gate halted after batch 1 on 2 FALSE-POSITIVE prose flags: 'is derived from [related quantity]' provenance matched the estimator_recipe pattern's 'derived ...from' arm. This recurs on every derived quantity, so it halts every rotation. Narrow the pattern (lead picks the fix), add a regression test, re-baseline budget to ~$0.29/name, then resume the stratified rotations. Doc-policy semantics call — is 'is derived from [related quantity]' acceptable normative prose? — hence the lead decides which fix.
Project: imas-standard-names / code: imas-codex (branch main)
Section: §5 — prose-predicate root-fix (gates the remaining rotations)
Tier: sonnet
Context
Rotation 1 resilient re-run (run c355df39, 2026-07-20; archive docs/archive/model-selection-and-global-refine-s5-rotation1-landed.html) proved the pipeline: 100% leaf quorum_consensus on the claude-sonnet-5 + grok-4.5 blind pair, 0 outages, 98% accept, 0 drift, 3.4% refine waste. It halted after batch 1 because the convergence gate saw banned prose reintroduced in 2 docs (line_averaged_hydrogen_density, ratio_of_tritium_density_to_deuterium_density). Both are FALSE POSITIVES: the estimator_recipe pattern in imas_codex/standard_names/prose_policy.py, `\bis (?:computed|calculated|estimated|obtained|derived) (?:as|by|from)\b`, matched 'It is derived from the local [hydrogen density](name:hydrogen_density)…' — legitimate provenance / cross-reference the refine model writes for any derived quantity, NOT a procedural estimator recipe. Evidence: of 2,200 accepted docs, 797 (36.2%) match estimator_recipe; a 12-doc sample of those are ALL genuine backlog recipes ('In practice this quantity is computed by …'), which trip the computed/obtained arms — the refine model strips those and only leaves the 'derived from' provenance residue.
Decision (lead) + Work
1. Pick the fix (doc-policy call): (a) drop `derived` from the verb alternation, keeping computed|calculated|estimated|obtained — the sampled genuine recipes all use those, so backlog coverage is retained; or (b) exempt 'derived from [name:link] / the local <quantity>' provenance specifically. Apply in prose_policy.py.
2. Regression test: banned_prose_findings() returns CLEAN on the two halted docs' text and still flags the 12 sampled backlog recipes (and the estimator_recipe class stays non-empty for 'is computed by' text). Commit+push (explicit paths, no AI trailers, no plan refs).
3. Re-baseline the budget to ≈$0.29/name (resilient premium review pair). ~1,300 remaining docs-fixable names ≈ $380 — the lead accepts the premium, trims the spec (e.g. prose-only), or caps rotations.
4. Resume the stratified rotations on the unchanged command (uv run imas-codex sn run --source dd --campaign "prose,audit:latex,audit:spelling,audit:length" --campaign-pilot 200 --include-accepted --campaign-batch-cost-cap 50 --campaign-cost-ceiling 120); the selection re-stratifies over the remaining ~1,300 and the 98 fixed names stay out.
Done-when
1. estimator_recipe narrowing committed + regression test green + pushed
2. budget re-baselined and recorded in the §5 archive
3. rotations resume; each rotation's checkpoint row recorded; this followup resolved when the pipeline runs a full rotation without a false-positive halt
Prose exemption LANDED (imas-codex 6c6ffec8 → narrowed f1a4fbbc). estimator_recipe no longer flags 'is/can be derived from … [label](name:id)' linked-quantity provenance (the parent/child form the refine seat writes), while 'derived from ' and all compute-verb recipes still flag — catalog-verified: the 2 halted docs scan clean, 20 procedure docs still flag; regression tests green. NB the first commit was a blanket 'derived from' exemption; the lead flagged it and it was narrowed to be name-link-conditioned (only 4 of 68 'derived from' accepted docs are linked provenance; ~20 are genuine procedure recipes). BUDGET decided: lead accepts ~$0.29/name as-is after a per-model cost review (rotation-1 split: gpt-5.5 refine+escalation $11.34, sonnet-5 review $9.04, grok $4.78, luna docs-gen $3.99). SEATS held after review: review blind pair claude-sonnet-5 (docs AUC 0.910, best) + grok-4.5 (cheap, names 0.920), justified on discrimination + 16/16 concurrency (old qwen+minimax pair non-viable under load); refine HOLD gpt-5.5 (resolution 0.874 / collateral 0.147 — luna 0.765/0.218 and terra 0.837/0.145 both measured worse or a wash; generate≠refine). Rotation resume carried by f-docs-campaign-rotations.