Crew Fleet Return-on-Experience Review

§1 — Scope and method

This review measures the integrated fleet-dispatch framework — Claude orchestrator sessions driving Codex CLI workers through reckon crew into detached git worktrees — across every repository that has used it, and audits the framework code itself. Four evidence streams were combined: (1) the four committed run ledgers (docs/state/<project>/crew.json in reckon, imas-ambix, nova and imas-codex — 259 completed runs, 2026-08-12 → 08-16); (2) a full read of the crew engine (crew.py, ledger.py, _backends.py, budget.py, flight.py, calibration.py, capabilities.py, agent_context.py, the crew half of cli.py and their tests); (3) the orchestration contract (reckon-ship SKILL.md plus its references); and (4) the live on-disk state — flight config layers, live run pointers, worktree roots and the capabilities cache. Two runs were live during the review (nova, imas-codex); their ledgers are still growing, so their figures are floors.

§2 — Return on experience: the numbers

	reckon	imas-ambix	nova	imas-codex	combined
completed runs	80	45	116	18	259
plans covered	8	3	18	3	32
gate pass rate	88.8%	91.1%	89.7%	100%	90.3%
lines added	9,145	45,877	53,565	not recorded	108,587
tests added	227	89	330	33	679
commits	70	43	108	7	228
median wall minutes / run	5.4	17.6	13.3	8.6	9.8
median budget consumption	21%	23%	45%	34%	32%
stall-corrected worker-hours	7.7	20.6	35.1	2.8	66.2
mean concurrency while active	2.02	1.22	1.90	1.97	2.26
peak concurrency	5	3	9	3	13 (global, 08-12 17:34)
scope-change rate	5.0%	11.1%	4.3%	0%	5.4%

Token economics (correct per-session totals — the per-run counters are session-cumulative, see §8): 1.78 billion input tokens, 96.9% of them cache reads, and 5.4 million output tokens across 68 worker sessions, for 108k added lines and 679 tests. The cache-read fraction is what roster session reuse buys and is the framework's standout economic result. Median authored throughput is stable at 16–24 changed lines per worker-minute across three unrelated codebases — the most reproducible number in the dataset and the right basis for effort estimation.

Stacked timeline of concurrent crew workers across four projects, 2026-08-12 to 08-16 — Concurrent workers over the five active days, stacked by project (intervals capped at 3× budget so stalled streams do not paint phantom activity). The fleet is bursty: short waves of 2–6 workers, one 13-wide global peak on 08-12, and long fully-idle gaps. 48% of busy time runs at concurrency 1.

Budget consumption histogram, per-project wall-time distributions with stalls marked, and gate outcomes per project — Run economics. Left: the median run uses a third of its declared time budget; three-quarters finish under half — budgets are 2–4× oversized. Middle: wall-time distributions; open rings are the nine stalled runs (>3× budget) that alone account for 64 of the 130 recorded wall-hours. Right: gate outcomes — the ~90% pass rate is stable across four unrelated codebases.

Verdict. The framework works and is improving with each adopting project: imas-codex, onboarded in a single afternoon, achieved 100% gate pass, 100% outcome coverage, zero stalls, zero over-budget runs, a clean roster and a 96.5% cache-read fraction from a standing start. But the honest sustained parallelism is 2.26 workers, not the 13 the peak suggests, and the gap is structural, not accidental: the orchestration contract's wave barrier (dispatch a wave, wait for the slowest member) idles roughly 45% of purchased worker-time, and nothing refills a finished worker's slot.

§3 — What works: do not touch

Zero duplicate run ids in 259 records; one-commit-per-run discipline holds wherever code is written.
The ~90% gate pass rate is a property of the dispatch discipline (single deliverable, measurable done-when, exclusive write scope), stable across four codebases.
Roster session reuse delivers a 96.9% cache-read fraction; reckon's best members carry 17–20 runs on one session.
The worktree isolation model: no cross-worker file collision appears anywhere in four ledgers.
The budget design principle "absence of a signal is never read as exhaustion" holds in code (budget.py:486-491) — no path turns unknown headroom into a hold.
Codex CLI workers are the well-supported path (resume, stop, liveness, session continuity, off-stream headroom probe); the asymmetry many would expect is reversed — in-harness native workers are the degraded ones.

§4 — Defect inventory

Twenty-plus defects were verified against code paths; two were demonstrated by execution. Ranked by severity within each cluster; each row names the follow-on plan that owns the fix.

State-store integrity (plan: crew-state-integrity)

Defect	Location	Failure
A corrupt or merge-conflicted `crew.json` silently resets the ledger to empty; the next promotion overwrites all history (demonstrated)	`ledger.py:118-136`, `_store.py:226-231`	`JSONDecodeError` is swallowed to `({}, 0)`, indistinguishable from "no ledger yet"; version check passes at 0. Conflict markers from concurrent orchestrators are the realistic trigger.
Concurrent promotions lose run records	`_store.py:239-269`	Read–compare–write with no lock and a fixed temp-file name shared by all writers; two simultaneous `crew complete` calls can publish one writer's content under the other's version bump. The retry loop only fires on version-check failure — exactly the interleaving that doesn't happen.
A duplicate promotion leaves a live pointer no command can remove	`crew.py:1665-1670`	`append_run` raises before the pointer unlink; thereafter `complete` always raises and `recover` advises a promotion that cannot succeed. No discard command exists.
Live pointers are read-modify-written with no concurrency control	`crew.py:533-539, 1275-1351, 1449-1461`	observe/attach/stop/resume interleavings silently drop fields (e.g. a task binding); fixed temp name allows mixed-content publishes.

Dispatch and run lifecycle (plan: dispatch-lifecycle-safety)

Defect	Location	Failure
Two concurrent nodes on one roster member kill each other, and dispatch never checks	`crew.py:1116-1130`	The second `codex exec resume` dies at launch with no stream and no manifest, phase `orphaned` — a silently lost node. `list_live()` has the data; nothing consults it. The highest-value one-line guard in this review.
A failed dispatch rolls back the worktree but leaves the spawned worker running	`crew.py:1204-1232`	Pointer-write failure after spawn: detached worker burns quota in a deleted cwd, invisible to list/recover/stop.
Run-directory creation sits outside the rollback	`crew.py:1132-1134`	A failed `mkdir` orphans a registered worktree that blocks the node id from redispatch.
One malformed reset timestamp crashes observe, complete and recover (demonstrated)	`_backends.py:191-201`	`datetime.fromtimestamp` unguarded: a millisecond-epoch `resetsAt` makes a finished run unpromotable and takes down recovery for every live run.
`crew stop` is erased by the next observe; recover then calls the run abandoned	`crew.py:1322, 1734-1790`	The intentional-cancellation distinction does not survive one observation.
`killpg` on a possibly-recycled pid; `process_alive` returns True on `PermissionError`	`crew.py:1454, 584-585`	A long-abandoned pointer can signal an unrelated process group.
Resume orphans the original stream and breaks measurement	`cli.py:718-731`, `crew.py:1546`	`log_path` is overwritten per resume, so `stream.jsonl` drops out of completion measurement; lexicographic resume ordering misorders from the tenth resume; the first pid becomes unreachable by `crew stop`.
The MCP live view trusts stored liveness	`crew.py:1727-1729`, `mcp.py:2510-2516`	A worker that dies without a manifest reports `running` indefinitely on the surface the skill tells orchestrators to poll.
Naive `--completed-at` timestamp crashes promotion (demonstrated)	`crew.py:1525-1532`	Aware-minus-naive `TypeError` outside the guard that its sibling in `ledger.py:445-455` has.

Measurement (plan: run-measurement-truth)

Defect	Evidence	Consequence
`worker_seconds` is orchestrator wall-clock, with `completed_at` inferred from stream mtime in 249/259 runs	9 stalled runs contribute 64 of 130 recorded hours (worst: 26.7 h against a 45 m budget)	Every derived quantity — lines/minute, effort calibration, the live competence gate's speed factor 20.93 — is built on a corrupted clock.
Per-run token counters are session-cumulative	Monotone non-decreasing across all 33 multi-run sessions; naive summing over-reports 4.2× globally (8.0× in reckon)	No per-run cost attribution is currently possible from the ledger.
Three modules disagree on which completion sources are usable	`ledger.py:755-758`, `capabilities.py:150-154`, `calibration.py:77-78`; `complete()` writes a third value `provided`	Runs promoted with an explicit stamp silently drop out of the effort report and horizon derivation; the live capabilities cache excluded 63 of ~80 reckon runs.
`outcome` empty on 86% of runs and on 21 of 25 non-passing ones	imas-ambix and nova 100% empty; imas-codex proves 100% coverage is achievable	The ledger records that six consecutive nova nodes burned on one section without recording why.
`changed_lines` regressed to absent in the newest project; 12 older reckon records store git error strings	imas-codex 0/18 recorded; reckon strings like `fatal: bad revision '…none'`	Throughput measurement is silently partial.
248 of 259 manifests point into `/run/user` tmpfs	ledger `manifest_path` fields	The documented recovery order (manifest → logs → message) loses its first two rungs on reboot.
Identifier drift: sections `§N` vs `s6` vs empty; a nova node name reused across two runs	imas-codex 13/18 empty section	Cross-project rollups fail; retry lineage is untrackable.

Admission gates (plan: gate-signal-repair)

Defect	Location	Failure
The competence gate compares estimated hours against a horizon after dividing by a speed factor in the wrong units, and reads plan-level hours for node-level decisions	`crew.py:881-948`, `capabilities.py:156-227`	Live cache: horizon 4.0 h × speed 20.93 → refuses only above ~84 estimated hours (21× looser than named); the advice ("split into nodes no larger than 83.7 worker-hours") cannot be satisfied by any input the caller controls. The two calibration modules also define "speed" in opposite directions — wiring one into the other flips the gate from 21× loose to 21× strict.
The capabilities cache never refreshes and no CLI rebuilds it	`capabilities.py:252-261`	Written 08-12 recording ledger versions 77/46/0/0; real ledgers now 95/197/38/59. First dispatch after a cache wipe synchronously scans every mounted repo.
A content-free "known" headroom block outranks a real measurement	`_backends.py:528-541`, `budget.py:334-345`	An empty `rate_limit_info` mapping claims `known` and displaces an older genuine 99% reading — the exact failure the module docstring claims to prevent. The window identity (`rateLimitType`) is also dropped, so a monthly reading ≥95% would hold every claude dispatch until month-end with no fresher signal able to arrive.
CLI resume ignores flight config entirely	`cli.py:695`	`resume_plan` gets `config=None` → permissive fallback policy; a project's ceiling and exhausted-statuses are silently ignored on every resume.
A ledger bookkeeping failure aborts the dispatch it was recording	`crew.py:470-481`	Contradicts the stated principle that a pre-flight must never be stopped by its own instrument.
A competence refusal is reported as a configuration error	`cli.py:610-616`	Structured verdict discarded; exit 1 instead of a typed hold; `--dry-run` skips the check entirely, so a validated wave can still refuse at real dispatch.

§5 — The real caps on background workers

The question "do we have artificial caps?" has a precise answer: nothing in the code caps the number of concurrent workers. The caps that exist are of three kinds:

Cap	Value	Kind	Reality
`backends.*.concurrency`	codex 3, claude 3, native 4 (host `~/.config/reckon/flight.yaml`)	Dead config	Declared in the schema, shipped defaults and host layer — no runtime code reads it. Raising it changes nothing; it should be wired to a dispatch admission check or deleted.
Roster size + session reuse	9 members (reckon), 10 (imas-codex), 41 (nova)	De-facto hard cap	A member is a serial worker; the real parallelism ceiling for session-reusing backends is however many members are registered — enforced by nothing but prose (see the collision defect above). Dispatching without `--member` has no ceiling at all.
Wave barrier	"launch the wave together, then wait for all results"	Contract prose	The most expensive artificial serialization: max-of-wave wait vs mean run time idles ~45% of worker-time at 3-wide. Rolling slot refill costs no safety property — the safety-critical rule is only that a dependent wave must not build on unverified work.
Landing freeze	"Nothing else may be promoted, merged, dispatched, or opened before this write"	Contract prose	Safety-critical in its promote/merge half; artificially blocks unrelated dispatches on an `edit_plan` round-trip in the rest.
`time_budget`	25m on all three backends (host layer); fence default 25m	Hard refusal at dispatch	The backend default doubles as its ceiling: no node on this machine can declare more than 25m without a config edit, contradicting the contract's promise of longer budgets for unsplittable work. Meanwhile nothing enforces the budget at runtime — no watchdog, no timer — so the fleet's worst run ran 26.7 h against 45 m, undetected.
Wave-size guidance	"2–8", "3–8", "at most three" (three documents, three numbers)	Advisory prose	Never enforced; "at most three for compiled work" is unsupported by any argument and is the strictest number in the corpus.
Budget ceiling	dispatch holds at 95% (ceiling 100 − reserve 5)	Real, but blind for codex	Codex has never reported headroom in 253 runs (its stream carries tokens only) and its off-stream probe (`codex app-server` account read, verified working) runs only with `budget_check: true`, which no layer sets. No hold has ever been recorded in any project. Only claude can be held, and only when a run happens to emit a rate-limit event.
Competence horizon	derived; currently refuses above ~84 estimated hours	Real but broken	Undocumented in any skill file; unit-mismatched (§4); inert today because only 1 of 24 reckon plans declares effort hours.

To push more work through workers, the binding order is: fix the member-collision guard and ledger write races first (they are what a wider fleet will trip), then replace the wave barrier with rolling refill and narrow the landing freeze, then raise per-role time budgets to measured values (p75 of actual wall time per role, not a uniform 25m) with a real runtime watchdog, and register more roster members per backend. Forced parallelism is not a risk in the current design — there is no mechanism that creates parallelism at all beyond what plan prose exposes; the DAG only ever restricts.

§6 — Orchestrator context economy

The contract makes the orchestrator read ~24k tokens of skill/reference preamble before the first dispatch, then spend ~15–18k tokens per delivered node (dispatch echo, per-run observe polls, manifest read, merge audit, a mandatory full-plan raw read before every state write, evidence and evergreen edits, completion). An eight-node plan costs ≈180k orchestrator tokens — this is the context flooding to engineer away. The four highest-payoff compressions, none of which loses a safety property:

A version-only read view (or server-side version re-read in edit_plan): the mandatory raw read exists only to fetch version and discards 3–6k tokens of plan state per landing.
Poll the fleet, not the run: crew(view="live") costs ~800 tokens for the whole fleet vs 1.4k per run per observe; the skill prescribes both in different sections and should prescribe only the former (after the live view re-checks liveness).
Stop mandating reference files the engine already compiled in: compose_prompt emits the parallel-safety rules, manifest keys, fences and help protocol verbatim; two reference files exist to let an orchestrator hand-compose what reckon composes for it.
Delegate evidence authoring: the cumulative evidence record is a single-file exclusive write scope — a textbook worker node the contract currently assigns to the orchestrator.

§7 — Parameters in force (host: this workstation)

Parameter	Value	Source	Note
`default_backend`	codex	host flight.yaml	All four roles route to codex; `native` is defined but unreachable without an explicit role override.
`backends.codex`	gpt-5.6-sol, effort high, cli, worktree-full, session_reuse	host	253 of 259 runs identical — the ledger supports no comparative routing claim.
`backends.*.time_budget`	25m	host	Default doubles as ceiling; see §5.
`budget.utilisation_ceiling_pct` / `resume_reserve_pct`	100 / 5	shipped	Effective dispatch ceiling 95%; resume may spend the reserve — but CLI resume currently ignores the policy entirely (§4).
`backends.*.budget_check`	unset	—	The most consequential unset knob: the only way codex ever reports headroom. Cost when enabled: one `codex app-server` probe (≤20 s) per preflight and per dispatch; a probe-once-per-wave shape is not expressible today.
`fences.needs_help_after_failures`	2	shipped	Prompt text only; nothing enforces it.
`LOG_STALE_AFTER_SECONDS`	900 (hard-coded)	`crew.py:1697`	Freshness label only; never acted on.
probe timeouts	20 s / 10 s / 5 s (hard-coded)	`_backends.py`, `flight.py`	Probes only, never workers.
ledger write retries	5 attempts, no backoff	`ledger.py:349,588`	A wide simultaneous-completion wave can exhaust this.
capabilities `success_threshold` / `bin_width_hours`	0.8 / 1.0 (hard-coded)	`capabilities.py:129`	Feed the competence horizon; no CLI reaches them.
project-doc byte budget	32768 (codex host config raises to 131072)	`agent_context.py:13`	No config source at all for the claude agent — the same 42 KB AGENTS.md passes codex and fails claude today.

§8 — Operational findings beyond the code

Workspace accumulation: 27 leftover worktrees totalling 12 GB under ~/Code/.reckon-worktrees/ (including a suspicious nested reckon-worktrees/reckon-worktrees directory), plus 270 run-stream directories (129 MB) under ~/.config/reckon/crew/runs/. Conservative cleanup never garbage-collects and no crew command owns removal.
Live-pointer hygiene: all live pointers observed during this review sat at phase starting — orchestrators are not folding observations back; one nova pointer had a dead pid and a completed replacement run, uncollected for hours. recover classifies these correctly but nothing reaps them.
Dispatch prerequisite drift: _create_worktree requires a vendored skills/reckon-ship/scripts/worktree_fleet.py per repo; present in nova/imas-ambix, missing in imas-efit — undocumented, and the failure will present as a broken dispatch.
Roster metadata drift: 10 of nova's 41 members declare harness: claude while all 25 of their runs recorded codex; 12 of 69 members fleet-wide have never captured a session.
Config-home documentation: the real config home is ~/.config/reckon/ (mounts, flight, crew state, cache); repo AGENTS.md still names ~/docs-server/ as current with the rename "tracked" — stale and actively misleading during this review.
Sprint hygiene in this repo: two sprints flagged active simultaneously (S5 and S9, active_sprint_id=S5); S7 contains only shipped plans yet reads planned; one shipped plan is unscheduled. In imas-ambix the crew runs map to a sprint dated to start the day after its last run completed. Ledger↔sprint rollups cannot work until membership and dating are repaired.
Framework self-harm, two recorded cases: a read-only review sandbox rejected its own launch before any work; a coordinator cleanup-session deleted a worktree under a live run. Cleanup must consult live pointers first.

§9 — Follow-on work map

Six plans, organized in two sprints. Correctness first: the state-integrity and lifecycle defects are what a wider fleet will trip immediately, so they gate the throughput work.

Sprint	Plan	Owns
S10 — crew correctness	`crew-state-integrity`	Ledger and pointer storage safety: locked/atomic envelope writes, corrupt-file refusal, duplicate-promotion recovery, a discard command, retry backoff.
	`dispatch-lifecycle-safety`	Member-collision guard, spawn rollback kill, run-dir rollback, stop/observe/recover phase truth, pid identity, resume stream handling, timestamp guards, live-view liveness.
	`run-measurement-truth`	Worker time from stream events with stall exclusion, per-run token deltas, one completion-source vocabulary, mandatory outcomes on non-pass, durable manifests, changed-lines everywhere, identifier normalization.
S11 — fleet throughput	`gate-signal-repair`	Competence gate units + cache + CLI + exit codes; budget signal truth (content-free known, window identity, resume policy, codex budget_check with a probe-once-per-wave shape).
	`fleet-throughput-contract`	Rolling wave refill, narrowed landing freeze, per-role measured time budgets + runtime watchdog, wire-or-delete concurrency config, roles for test/doc workers, fleet polling + version-only reads, skill/CLI reconciliation.
	`crew-workspace-hygiene`	A garbage-collection surface: integrated-worktree removal, run-dir retention, dead-pointer reaping, live-pointer-aware cleanup, per-repo prerequisite checks.

Related existing threads left where they are: the cross-repository dispatch gap stays on execution-orchestration-hardening (f-eoh-cross-repo-node); the user-gated server restart stays on inflight-visibility; the sprint-hygiene repairs in §8 belong to the sprint owner, not to these plans.