# v0.7 Retrospective — Beginner UX, made legible

*Period: 2026-04-20 (same-day continuation of v0.6.0).*
*Sub-releases: v0.7-A / v0.7-B / v0.7-C. None are git tags yet; this retro proposes tagging the three as `v0.7.0`.*
*Tests: 306 → **382** (+76, +25%). All green throughout.*

## Scope at a glance

| Sub-release | Theme | Primary deliverable | Tests Δ |
|---|---|---|---|
| v0.7-A | YAML-first declarative capability registry | `coderouter/data/model-capabilities.yaml` (bundled) + `~/.coderouter/model-capabilities.yaml` (user override), `CapabilityRegistry` with glob + first-match-per-flag semantics | +39 |
| v0.7-B | Live-probe diagnostic subcommand | `coderouter doctor --check-model <provider>` with 4 probes (auth / tool_calls / thinking / reasoning-leak), YAML patch emit, CI-friendly 0/2/1 exit codes | +37 |
| v0.7-C | Beginner UX — symptoms, probe, patch | README `Ollama beginner — 5 silent-fail symptoms` subsection + `ollama-hf-example` reference stanza + lunacode cross-link | ±0 |

The minor was scoped from plan.md §9.4 around a single operator phenotype: "I set up Ollama, pointed CodeRouter at it, and something is off — but nothing looks like an error". Five silent-fail symptoms were collected ahead of scoping (`num_ctx` truncation / tool-call incompetence / `<think>` leak / model-tag 404 / missing API key), and the three sub-releases were ordered to close the diagnostic loop: declare capabilities in data rather than Python literals (A), probe them at runtime against the declarations (B), document the observable symptoms so operators know which probe to run (C). Sub-release C is intentionally non-code — a pattern this minor validates but did not invent (v0.4's docs-pass and v0.5.1's closeout pack were earlier precedents).

## What happened, in order

1. **v0.7-A** externalized the `^claude-(opus-4|sonnet-4-6|sonnet-4-7|haiku-4)` regex literal that had lived inside `coderouter/routing/capability.py` since v0.5-A. The regex worked, but it was the wrong shape for two reasons that only become clear in aggregate: (a) adding a new Anthropic family meant a code change + release cycle, which is strictly more expensive than "edit one YAML line", and (b) the regex couldn't declare anything other than `thinking` — the capability gate was about to grow `tools` / `reasoning_passthrough` / `max_context_tokens` and the regex-as-registry approach didn't scale to multiple flags. The fix was a declarative YAML layer: `coderouter/data/model-capabilities.yaml` as a bundled default that ships with the package (encoding the v0.5-A regex 1:1 for drop-in compatibility) + an optional `~/.coderouter/model-capabilities.yaml` user override. Schema: `rules: [{match: <fnmatch>, kind: anthropic|openai_compat|any, capabilities: {...}}]` with first-match-per-flag semantics. Precedence: `providers.yaml capabilities.*` per-provider (explicit) > user YAML > bundled YAML > unset. The explicit-wins-over-glob ordering matters — it's the contract that makes glob defaults safe to evolve, because a user can always pin per-provider. Everything landed behind the existing `provider_supports_thinking(provider)` boundary so the capability gate's public API didn't move. Zero observable behavior change for any existing provider; +39 tests mostly verifying the precedence matrix.

2. **v0.7-B** was the largest sub-release of the minor, both in LOC and in design surface. Given v0.7-A's registry as a declarative source-of-truth, the natural next question was "how does an operator know when their declarations diverge from reality?". The answer: a live-probe subcommand that runs four small probes against one provider and diffs observed behavior against the registry + explicit `capabilities.*`. The four probes mirror the capability gates already implemented (auth is a structural precondition; tool_calls maps to v0.3-A tool-call repair; thinking maps to v0.5-A capability gate; reasoning-leak maps to v0.5-C passive strip), which is deliberate — each probe is an **observation hole** into a transformation CodeRouter already does. Three design calls shaped the rest of the implementation:

   - **Bypass the adapter layer and call httpx directly.** If the probe runs through the adapter, it observes post-transformation behavior — but the whole point of the probe is to diagnose transformations. The reasoning-leak probe needs to see the raw upstream body before the v0.5-C strip runs; the thinking probe needs to send Anthropic's native wire shape without OpenAI-shape round-trip; the tool_calls probe needs to distinguish "native `tool_calls`" from "text-JSON that v0.3-A would repair". All three require the adapter to be out of the way. This is the opposite of the usual "use the production code path" advice, and the tension was real — but the alternative (adapter-aware probes) leaks the adapter's internals into the test surface and couples the diagnostic to the adapter version.
   - **Auth short-circuit with SKIP preserved.** When the auth probe fails (401 / 403), the remaining three probes are pointless — the request never makes it to the model. But simply hiding them felt wrong: operators should see *what was not checked* so they can reason about whether fixing auth alone is sufficient. The compromise: each subsequent probe emits a `SKIP` verdict with `detail: "upstream auth failed"` so the transparency is preserved while the token cost is not. The same short-circuit fires for 404 (model-not-found), since that's structurally the same "request can't reach the model" case. It turned out to cleanly map symptom #4 and #5 from the plan.md table onto the same UX.
   - **Exit codes at Unix lint convention.** `0` match / `2` needs-tuning / `1` blocker. The temptation was to collapse tuning-needed into blocker — "patch required" is a kind of failure — but CI semantics want the distinction: `2` means "I have a mechanical fix, auto-PR it", `1` means "human intervention required". That's the `eslint --fix` / `flake8` convention, adopted here for the same reason. The precedence when multiple probes fail (blocker > tuning > ok → 1 > 2 > 0) was pinned in a dedicated `exit_code_for()` helper with three tests, partly because the precedence itself is a contract that downstream CI consumers will rely on.

   The rest of the implementation was straightforward: `ProbeVerdict` enum, `ProbeResult` / `DoctorReport` dataclasses, `format_report()` with `Exit: N` trailer for `grep`-ability, `_patch_providers_yaml_capability()` / `_patch_model_capabilities_yaml()` helpers that emit YAML parseable-by-test-suite, and a `run_check_model_sync()` wrapper for the CLI. CLI wiring via argparse subparser with three error-mapping classes (FileNotFoundError / KeyError / generic Exception → exit 1 + stderr). +37 tests total: 31 in `tests/test_doctor.py` covering each probe's verdict matrix, 6 in `tests/test_cli.py` for argument plumbing and error propagation.

3. **v0.7-C** was deliberately non-code and deliberately a sub-release. The precedent was v0.4's docs-pass + v0.5.1's closeout pack, both of which had been packed into the same "release" as code changes and got lost in review noise. v0.7-C separates the documentation work into its own versioned checkpoint: README `Ollama beginner — 5 silent-fail symptoms (v0.7-C)` subsection with one entry per symptom following a uniform 3–4-step template (cause / doctor command / YAML patch / fix command), plus a `#### HF-on-Ollama reference profile` sub-subsection that points at a new `ollama-hf-example` stanza in `examples/providers.yaml`. The stanza is commented-out on purpose — uncomment-to-use forces the operator through the `ollama pull` step and makes the `:<quant>` suffix failure mode concrete (symptom #4). Inline comments map each knob (`num_ctx` / `append_system_prompt` / `capabilities.tools` / `reasoning_passthrough`) back to its symptom, which is the smallest legible cross-reference from data to narrative. Cross-link to [lunacode](https://github.com/zephel01/lunacode)'s `MODEL_SETTINGS.md` was added to resolve an operator question that was going to come up anyway: "I use both CodeRouter and lunacode against the same Ollama — which file do I edit?" The answer is "both, at different granularities" (CodeRouter provider-level, lunacode model-level), and saying it explicitly in the README costs less than fielding the question.

## The minor's design through-lines

Four patterns emerged across v0.7 that are worth naming:

| Pattern | Instances in v0.7 | Cross-minor roots |
|---|---|---|
| Data-as-configuration (YAML over Python literals) | v0.7-A `model-capabilities.yaml` (bundled + user) | v0.5-A regex literal (the thing being replaced); spiritually adjacent to v0.6-D `mode_aliases` YAML block |
| Diagnostic surface that bypasses runtime transformations | v0.7-B direct httpx probes | New in v0.7; reveals transformations from v0.3-A / v0.5-A / v0.5-C |
| Dominant-signal short-circuit with SKIP preserved | v0.7-B auth failure → SKIP remaining probes | Philosophically adjacent to v0.6-C `chain-paid-gate-blocked` aggregate log (single loud signal > N quiet signals) |
| Non-code release as a sub-release boundary | v0.7-C docs + examples | v0.4 docs pass / v0.5.1 closeout pack; formalized here |

The first two are the durable ones. **Data-as-configuration** is the obvious conclusion from v0.7-A's motivation — regex literals in Python files are a maintenance anti-pattern once you realize you're using them as a registry — but naming it explicitly matters because v0.8+ will face the same decision for OpenRouter roster cron output, request-level mode overrides, and any other "lookup tables" that currently live in Python. Default to YAML with bundled + user layering, unless there's a specific reason not to.

**Diagnostic surface bypassing the runtime** is the more interesting pattern. CodeRouter has accumulated several transformations (v0.3-A tool-call repair, v0.4-A ChatRequest↔AnthropicRequest round-trip, v0.5-A thinking gate, v0.5-B cache_control strip in translation, v0.5-C reasoning field strip, v0.6-B provider-call overrides). Each transformation is correct in production but creates an observability hole — the operator can't tell from a request/response pair what the runtime did. `coderouter doctor` plants an observation point *before* the transformations run, so the probe sees what the upstream actually emits. A corollary: any future transformation (v1.0 output-cleaning for `<think>` tags is the near-term candidate) should come with a probe that verifies its trigger condition, not just the transformation output. Probe-alongside-transformation is a good habit to adopt before v1.0 lands its output cleaning.

## What worked

- **Externalization-before-probe was the right ordering.** If v0.7-B had landed before v0.7-A, the probe would have had to diff against a Python regex — which means the probe would need to import `capability.py`, which means the probe would couple to the runtime capability layer by construction. With v0.7-A first, the probe diffs against a `ResolvedCapabilities` object returned by a query-shaped `CapabilityRegistry.lookup(kind=..., model=...)` interface, and the runtime layer is a sibling consumer of the same registry. Small design call that paid off immediately in v0.7-B test mockability.
- **Probe coverage tracked the symptom table explicitly.** plan.md §9.4's 5-symptom table was written before any v0.7 code, and v0.7-B's four probes were designed to cover symptoms #2 (tool_calls) / #3 (thinking) / #4 (model-not-found) / #5 (auth) directly. Symptom #1 (`num_ctx` truncation) was intentionally held out because direct detection requires a long-prompt probe that doubles token cost; it surfaces indirectly through tool_calls "no tool_call emitted" and was documented as a follow-on. Pre-registered scope discipline: the table was the contract, and the code delivered against it.
- **Auth short-circuit's SKIP verdict was the right UX call.** Early design draft had auth failure simply halt the probe run with "auth failed, stopping" text. The review feedback that changed it: "I want to know what wasn't checked so I can reason about whether fixing auth alone is enough". SKIP with transparent reason preserves the operator's audit without wasting tokens. Same philosophy as v0.5-B's "observability-only" capability-degraded — failure mode is visible, not silent.
- **YAML patch emitters validated via round-trip parse.** Every `_patch_*` helper is tested by running `yaml.safe_load()` over its output and asserting the result is non-empty + structured correctly. This would be annoying to maintain for a dozen emitters but is cheap for the two here, and catches a class of bug (malformed YAML suggestion emitted to the user) that would be embarrassing in practice.
- **Non-code release as a sub-release worked.** v0.7-C was 0 LOC of production code, 0 LOC of test code, and still earned its own CHANGELOG entry, release history row, tag index row, and Status line. Making it a named sub-release signals that documentation quality is a first-class deliverable, not a chore. The alternative (packing it into v0.7-B's PR) would have buried the 5-symptom narrative in the diff review noise.
- **+76 tests vs +35 target.** Same 2x overshoot pattern as every recent minor. The denominator this time was `test_capability_registry.py` (+~30 for v0.7-A) and `test_doctor.py` (+31 for v0.7-B). Both are characterization-style test suites for new subsystems; the ratio of "designed-the-feature → wrote-the-test" was roughly 1:1 in LOC.
- **5-dep bound held.** v0.7-B's doctor module is pure-httpx + pyyaml + pydantic. No SDK creep despite the probe being a nontrivial network-client surface. Same restraint as v0.6.

## What was sharp

- **`num_ctx` probe is a real gap, not just a deferred nicety.** Symptom #1 is the single most common Ollama silent-fail for Claude Code workloads, and `coderouter doctor` detects it only indirectly. The right design is probably a two-shot probe that sends a long system prompt with a token-count marker in the first 1000 tokens and asks the model to echo it — if the marker is missing, `num_ctx` truncation is confirmed. This doubles the probe's token cost and wasn't justified for v0.7-B, but it's the shape that will close the loop for symptom #1. Held for v0.8.
- **The `--network` flag that wasn't.** plan.md §9.4 originally envisioned `coderouter doctor --network` as the flag that gates the live-probe mode, anticipating a future `coderouter doctor --static` that does offline lint. In v0.7-B the `--check-model <provider>` flag implies network usage by construction, making `--network` redundant. The flag was dropped, but the static-lint mode it was anticipating is still absent. If v0.8 introduces offline capability linting (e.g. "your `providers.yaml` references a model that doesn't match any registry rule, here's the diagnostic"), the flag should come back — not as `--network` but as a separate `doctor static-lint` subcommand so the two modes don't share argparse namespace.
- **Python 3.10 sandbox vs 3.11+ `datetime.UTC` still bites.** The v0.5-D OpenRouter roster diff test suite has been `--ignore`'d from every sandbox CI run since v0.5-D landed. The workaround is cheap but the accumulated cost of remembering it in every retro is not zero. Either (a) backport `datetime.UTC` via a compat shim in the roster module, or (b) bump the sandbox to Python 3.11+. The latter is the right answer; the former would be a code smell.
- **`ResolvedCapabilities` shape designed in v0.7-A without full knowledge of v0.7-B's needs.** The `CapabilityRegistry.lookup()` return shape was nailed down in v0.7-A and then slightly awkwardly used in v0.7-B — the probe wants to know "which rule matched this provider" for diagnostic messages, but `ResolvedCapabilities` only exposes the merged flags. Not painful (the doctor threads `provider.name` through for messaging), but the micro-friction is a reminder to over-design return shapes when the consumer surface is known to be growing. Kept in mind for `--json` output, which will need the same "what matched" metadata.
- **GitHub anchor fragility in README cross-links.** The 5-symptom subsection links to the Doctor subsection via `#doctor--coderouter-doctor---check-model-provider-v07-b` — an anchor generated from a heading with backticks, em-dash, angle brackets, and parentheses. If someone edits the Doctor heading casually, the link breaks silently. The right fix is either `{#doctor-v07b}` explicit anchors (not CommonMark, but GitHub-supported) or a flatter heading that's more edit-stable. Left unfixed for now; noted for v0.8 doc-pass.
- **Doc-edit touchpoint cost, still.** Same warning as the v0.6 retro: per sub-release, ~9 doc places need updating (plan.md state line / release history / tag index / test chart / §9.4 strike-through / CHANGELOG top / README Status / README feature bullet / "Coming next"). That's ~27 edits for v0.7-A/B/C combined, none of them individually painful but cumulatively real. A `scripts/release-close.py` that takes a version tag and a one-line summary and patches the 9 places would be a one-afternoon build with multi-release payback. Same as last time. Still unimplemented.
- **No real-machine re-verify.** v0.4 and v0.5 both had same-day real-machine verification passes. v0.7's changes are mostly data-layer (v0.7-A YAML) + diagnostic-layer (v0.7-B probe) + docs-layer (v0.7-C), which are lower risk, but the diagnostic layer specifically would benefit from a real-machine run against a live Ollama + OpenRouter + api.anthropic.com to confirm the verdict text is correct in all four probes. Deferred — `scripts/verify_v0_7.sh` or equivalent is a v0.7-D or v1.0 item.

## Follow-ons worth remembering

- **`coderouter doctor` `num_ctx` probe.** Long-prompt + echo-marker technique, gated behind a flag (`--deep` or `--probe num_ctx`) because it's the one probe that's not ≤ 100 tokens. Symptom #1 direct detection. v0.8 scope.
- **`coderouter doctor --json` output mode.** Machine-readable verdict + patches for CI auto-PR bots. Needs `ResolvedCapabilities` to carry "which rule matched" metadata, so couple with the design-cleanup above. v0.7-D or v0.8.
- **CI smoke workflow: weekly `doctor --check-model` over each free provider.** Symmetric with the v0.5-D OpenRouter roster cron — auto-PR for exit 2, open-issue for exit 1. Prerequisite: `--json` for structured consumption. The cron's existing file layout (`scripts/openrouter_roster_diff.py`, GitHub Actions workflow) is a good template.
- **HF-on-Ollama glob rules in bundled `model-capabilities.yaml`.** `hf.co/unsloth/*-Coder-*-GGUF:*` as `tools: true`, `hf.co/*/DeepSeek-R1-Distill-*:*` as `reasoning: leaky`. Decision deferred because it violates the provider-granularity principle — the same HF GGUF might be reliable via one Ollama install and unreliable via another, and declaring it at registry level removes the operator's per-deployment dissent. The reference stanza in `examples/providers.yaml` is the compromise: operator-level declaration, not registry-level. If the pattern accumulates across 10+ user-reported HF models in v0.8+ issues, revisit.
- **`reasoning` field strip granularity.** Current v0.5-C strip is all-or-nothing per-provider. Downstream of v0.7-B reasoning-leak probe, operators may want to keep some reasoning tags (e.g. from an R1-family distill where `<think>` is signal) while stripping others. v1.0 output-cleaning is the natural home; add a probe for it as a matter of course.
- **Real-machine re-verify for v0.7.** One afternoon: Claude Code + Ollama + `coderouter doctor --check-model ollama-qwen-coder-7b`, confirm verdict table text and exit code; intentionally misconfigure OPENROUTER_API_KEY, confirm AUTH_FAIL + SKIP cascade; intentionally use a non-pulled model tag, confirm UNSUPPORTED + pull-hint. Output of run goes into `docs/v0_7_verify_evidence.md` or equivalent.
- **Test-count auto-updater.** Third retro in a row to name this. `scripts/release-close.py` incorporating it is a candidate v0.8-adjacent chore.
- **v1.0 output-cleaning probe coupling.** As v1.0 lands `<think>` stripping, the `coderouter doctor` reasoning-leak probe should grow a "strip is configured, verify it's firing" verdict. Probe-alongside-transformation as a habit.

## Numbers

```
Tests:        306  →  382              (+76, +25%)
  v0.7-A:     306  →  345              (+39, registry + glob + precedence matrix)
  v0.7-B:     345  →  382              (+37, 4 probes + patch emitters + CLI wiring)
  v0.7-C:     382  →  382              (±0,  docs + examples only)
LOC (src):    ~4350 → ~5100            (+~750, concentrated in capability_registry.py + doctor.py + cli.py subcommand)
LOC (tests):  ~7250 → ~7950            (+~700, test_capability_registry.py + test_doctor.py + test_cli.py doctor block)
Docs (adds):  README +~130 lines       (Status bump + feature bullet + Doctor subsection + 5-symptom subsection + HF-on-Ollama sub-subsection + Coming-next trim)
              CHANGELOG +~180 lines    ([v0.7-A] + [v0.7-B] + [v0.7-C] entries with design notes + follow-ons)
              plan.md ~9 edits / sub   (state line / release history / tag index / test chart / §9.4 DoD strike-through × 3)
Commits:      3 on 2026-04-20          (one per sub-release + docs touches)
Runtime deps: 5    →  5                (unchanged — pyyaml was already present, no new SDK creep)
```

Baselines: v0.6.0 umbrella (306 tests). LOC numbers are rough; the commit log is the source of truth.

v0.7's LOC delta is meaningfully larger than v0.6's (+750 vs +250) because v0.7-B's `doctor.py` is a new top-level module with its own abstraction stack (probe enum + result dataclass + report dataclass + patch emitters + CLI wiring), and v0.7-A's `capability_registry.py` adds the Pydantic schema + loader + glob matcher layer. Test delta is proportionally similar (+700 vs +400) — the discipline of "characterization tests for every new subsystem" held.

## How to read this

This retro is the primary source for v0.8+ planning. Four constraints worth respecting when adding the next minor:

- **Data-as-configuration is the default for new lookup tables.** If a new feature needs a lookup from name-pattern to capability-flag (or similar), it goes in a bundled YAML file with an optional user override, not as a Python literal. Two precedents as of v0.7 (`model-capabilities.yaml` + `mode_aliases` block). The third instance should cement it as a convention.
- **Every new transformation gets a probe.** v0.7-B demonstrated that the runtime's transformations are what need diagnosing. A v1.0 output-cleaning feature ships with a probe; an eventual vision capability gate ships with a probe; a future `reasoning` field partial-strip ships with a probe. The cost is low (the probe is roughly the same LOC as the transformation being verified) and it replaces ad-hoc operator debugging sessions.
- **Non-code releases are legitimate sub-releases.** Docs + examples + retros can be versioned as C-suffixed sub-releases (v0.7-C, and presumably v0.8-C / v0.9-C when the pattern recurs) rather than packed into the preceding code release. The sub-release boundary gives docs the same review surface as code.
- **Exit-code precedence is `1` > `2` > `0`.** Any future CLI subcommand that reports diagnostics should adopt the same precedence, unless there's a specific reason (the reason must be documented in the subcommand's argparse help text).

The full sub-release detail lives in `CHANGELOG.md` (`[v0.7-A]` / `[v0.7-B]` / `[v0.7-C]`). This document is the narrative layer on top.
