Budget-aware dispatch — landed evidence
A pre-flight reads what earlier runs already recorded and decides, per backend, whether a wave may open. Holding costs nothing and creates nothing; a backend that publishes no headroom is never held. Ten declared measures produced evidence, 29 new tests cover them, and the suite is green at 1,657. Two results are worth reading before the rest: the investigation the plan left open is answered positively, and a held wave scheduled its own resumption and fired.
Declared measures and what each produced
| Measure | Evidence | Verdict |
|---|---|---|
| Headroom parsed where it exists | Recovered twice from real sources: a recorded
run stream yields headroom: known with utilisation and reset time, and a
live read of a second backend's account surface returned utilisation 0.0% resetting
2026-08-19T06:07:08Z — § reading | pass |
| Unknown is honest and non-blocking | A backend recording no headroom reads
unknown and the wave opens; the verdict says why. Four tests assert an opening
wave and none asserts a hold from silence — § unknown | pass |
| Exhaustion holds the wave | A recorded 100.0% holds the dispatch before any
worktree exists: no live pointer, no worktree named for the node in
git worktree list, and the node still validates as dispatchable —
§ hold | pass |
| Holds are per-backend | With one backend at 100.0% held, a node routed to the other dispatched and got its worktree in the same session — § hold | pass |
| Reserve protects the escape hatch | At 97.0% against a 100% ceiling and a 5-point reserve, a dispatch is held at the 95.0% effective ceiling while a resume is not; a resume is still held at 100.0% — § reserve | pass |
| Pre-flight is free | With subprocess.Popen and
subprocess.run both replaced by an assertion failure, the pre-flight still
returns its hold — it reads only the ledger and the live pointers —
§ free | pass |
| Harness capability quarantined | A regex over every skills/**.md
outside orchestrator-harness/ finds no harness-local primitive; two host files
exist and each states all four capabilities, including the ones it lacks —
§ quarantine | pass |
| Resume actually fires | A hold reporting a 79 s wait scheduled its own resumption through the host's self-scheduling primitive; it fired, and the resumed pre-flight returned clear — § resume | pass |
| Investigation answered | Answered positively, which the plan did not expect:
the account surface answers account/rateLimits/read with used percentages and
reset times, by a read that runs no model — § investigation | pass |
| Suite green | 1,657 pass, 29 of them new; lint clean; the generated schema artifacts regenerate byte-identically — § suite | pass |
The two ceilings, and why there are two
One utilisation axis carries the whole policy. A fresh dispatch stops at the ceiling less the reserve; answering a worker that stopped and asked for help may spend the reserve, because that is the expenditure it was withheld for. Spending the last of a quota on a new node instead strands the wave in its worst state — work in flight, and nothing left to unblock it with.
Measured: at 97.0% the dispatch verdict reports
effective_ceiling_pct: 95.0 against ceiling_pct: 100.0 and holds, while
the resume verdict reports effective_ceiling_pct: 100.0 and does not. At 100.0% both
hold.
What is read, and why a call is not made
The pre-flight reads the budget block that runs already recorded, from both homes a run's record occupies: the live pointers of runs still in flight, and the committed ledger for runs that finished. A completed record now carries that block, because promotion deletes the pointer that held it.
Two selection rules do the real work, and both were chosen against a failure:
- A known reading outranks any silence, however recent. An observation carrying no headroom carries no information, so taking "most recent" at face value would let one silent run erase a real exhaustion and open the wave the mechanism exists to hold. Measured: with a 99.0% record followed one second later by a silent one, the reader returns 99.0%.
- A window that has reset expires. The figure described a window; once that
window rolls over it describes nothing, and it degrades to unknown — which never holds.
Without this, a single exhausted record would hold a project forever. Measured: the same
100.0% record with a reset 60 s in the past reports
expired: true,headroom: unknown, and the wave opens.
Headroom is recovered from both backends, from real sources rather than hand-written fixtures:
| Source | Transport | Recovered |
|---|---|---|
| A recorded worker run stream | the machine-readable event stream a worker already writes | headroom: known, utilisation and reset time, from the
stream's rate-limit event |
| A live account-limit read | the backend's own account surface, over a line protocol | utilisation_pct: 0.0, resets_at:
2026-08-19T06:07:08Z, headroom: known |
The second row is a live result, run end-to-end through
_backends.probe_budget against a real backend. It costs no worker budget: the read
runs no model.
The investigation, answered positively
The plan recorded one backend as publishing per-turn token usage, no headroom, and no query command for it — while its interactive display shows rolling and weekly limits, so the information had to exist somewhere. The plan asked whether the app-server path exposes it, and allowed a negative answer as a valid outcome.
It is positive. The app server's protocol schema declares an
account/rateLimits/read request whose answer carries a rate-limit snapshot: per
metered window, a used percentage and a reset timestamp — the same two facts the other backend
publishes on its stream. A live exchange returned them.
So the asymmetry the plan describes is in the stream, not in the harness. Two mechanics make it a probe rather than a command whose output is read, and both were learned by the live attempt failing first:
- the server rejects requests until a handshake has been answered, so the exchange is a sequence, not one call;
- it exits the moment its input closes, so a plain
command < requestsredirect returns nothing at all — stdin has to stay open for the life of the exchange, and the reply is read on a thread because unrelated notifications arrive interleaved with it.
What this does not do is overturn the locked decision that the pre-flight reads the
ledger rather than making a call. That decision's rationale — a probe spends the very resource it
is measuring, most often when it is scarcest — is untouched by a read that runs no model, and §2
of the plan already sanctioned using a genuine status surface where one exists. So the ledger
stays the base case and the surface is opt-in per backend
(budget_check: true), which keeps the free path free: with no backend asking for it,
the pre-flight spawns nothing.
One consequence is worth recording because it inverts an expectation: on the two hosts measured, the harness that cannot schedule its own resumption is the one that can see its own headroom, and vice versa. Neither host is simply better instrumented than the other.
A hold creates nothing, fails nothing, and is per-backend
A held dispatch raises before the worktree is cut, which is the ordering that matters: a hold
that had already created a worktree would leave write scope claimed by a node nobody is running.
After a held dispatch, crew.list_live() is empty, no worktree named for the node
appears in git worktree list, and the node still passes all seven properties of the
task contract — it was never malformed, only waiting.
The distinction is carried into the command surface as its own exit code, because a caller that cannot tell a hold from a malformed node either rewrites work that was fine or abandons work that was only waiting:
| Exit | Meaning | What the caller does |
|---|---|---|
| 0 | succeeded | proceed |
| 1 | the configuration or request is wrong | fix the call |
| 2 | the node is not dispatchable, naming each failing property | reshape or split the node |
| 3 | held on budget, naming the backend, its utilisation and its reset | leave the node exactly as it is; retry after the reset |
Per-backend is not a refinement but the reason budget state is tracked per backend at all.
Measured: with one backend recorded at 100.0% and another at 10.0%, the report returned
held_backends: ["alpha"] and clear_backends: ["beta"], and a node routed
to the clear one dispatched and got its worktree in the same session.
Holds are recorded, and the record is a measurement
A hold produced no run, so nothing recorded that a wave was held, on which backend, or for
how long — the one measurement the mechanism generates was the one thing it did not keep. Holds
now sit in crew.json beside the runs, never among them: a hold has no worktree,
commit, gate or worker-time, and putting it in the run list would feed empty measurements to the
effort report that averages over it.
One record spans one continuous window, so an orchestrator checking every few minutes does not turn "how often were we held" into "how often did we look". Closing is what turns the record into a measurement: a test hold closed at 137 s of wall-clock against a 600 s predicted reset. The gap is the point — it is what reveals a resumption that fired late, and it is unavailable to anyone who restates the forecast instead of measuring.
Two defects surfaced in audit rather than in the gates, and both were fixed at the root:
| Found | Why it mattered | Resolution |
|---|---|---|
| Recording was wired into the shared pre-flight, so the read-only tool opened a hold and took the ledger from version 1 to 2 just by being asked | A reader that records invents history by looking at it, and the tool is documented as never mutating | Recording belongs to the commands that decide to open a wave; the read is now
v1→v1 with zero holds written while still reporting held: true |
resumption_fired was true whenever a resume-purpose check closed the
hold |
That is a stuck worker being answered, not a scheduled wave resuming — the field answered the opposite of the question the section asks | An explicit input, because a stuck-worker resume uses the same ceiling as a scheduled resumption without proving a scheduler fired |
A promoted signal keeps its producer
The reader matches a recorded budget block to a backend through the run's agent block, and a
record promoted without one keeps its headroom while losing the only thing that makes it
readable. Measured on this repository's own ledger: six of seventeen records carried no
agent.backend, and both records holding a genuinely known reading were
among them. Every real headroom signal the project had recorded was invisible to the mechanism
built to read it — and the loss was caused by promotion, not absence, since the same reading was
attributed from the live pointer minutes earlier.
That sits under the locked read-source decision rather than beside it: the ledger was chosen
over a probe because the ledger already holds what each backend emitted. It also revived the
failure the fail-safe posture exists to prevent, by another route — unknown came to
mean "recorded but unmatched" while still claiming to mean "nothing recorded", which is wrong in
the direction that opens waves it should hold.
The first fix was right by coincidence, which is why the gate changed. It recovered the backend by matching a configured name against a component of the manifest path — and that path runs through the host harness's own runtime directory. Tested as a counterfactual rather than argued: the identical reading, reached through a scratch directory named for a different harness, attributed to that other harness. The answer tracked where the orchestrator ran, not what produced the signal.
Attribution now walks a ladder of real evidence — a backend field written at promotion, the
agent block, the budget block, the roster member — and recovers a legacy record from the
normalised evidence its stream interpreter produced. Both known readings resolve to their
producing backend with attribution: budget-evidence; a misleading delivery path does
not change the answer; ambiguous producer evidence stays unattributed; and a record with nothing
recoverable reports source: unattributed-ledger, which a caller can tell apart from
nothing-was-recorded.
Silence is never read as exhaustion
The fail-safe direction is deliberately one-way. A false hold is invisible and stalls everything, while the failure it would prevent — a rejected call — is cheap and announces itself. So an unknown backend's verdict reads "headroom is unknown, and absence of a signal is never read as exhaustion" and the wave opens.
Three separate paths reach unknown and all of them open the wave: a backend whose recorded block carries no headroom, a project with no records at all, and a backend whose measured window has since reset. A fourth is the instrument itself — every failure path of the account-limit read returns an unknown block naming the reason rather than raising, because an instrument that fails must not become a hold.
Free, asserted rather than asserted about
"Costs nothing" is the kind of claim that decays silently, so it is pinned by making any
spawned process a test failure: with both subprocess.Popen and
subprocess.run replaced by an assertion, the pre-flight still returns its hold. The
only reads are the ledger file and the live pointers.
The opt-in surface is pinned from both sides too: with no backend setting
budget_check, the injected probe runner is never called; with one backend setting
it, it is called exactly once, for that backend.
The host harness is quarantined, and says what it cannot do
Whether an orchestrator can schedule its own resumption depends entirely on the harness it is
running inside. Naming that primitive in the skill or in a process reference would couple the
whole skill to one host, which is the portability a single skill exists to preserve — so it lives
in references/orchestrator-harness/<harness>.md, mirroring the worker-backend
files, and a test enforces the boundary rather than trusting reviewer attention.
Two files exist, and each states all four capabilities — including the ones it lacks, because an omission reads as "not investigated" rather than "absent":
| Capability | First host | Second host |
|---|---|---|
| Background dispatch | yes | partly — nothing binds a detached process to the session |
| Wake on completion | yes | no |
| Self-scheduling | yes, three forms | no — needs an external trigger |
| Budget visibility to itself | no | yes — its account surface answers a limits read |
The second file earns its place by being nearly the complement of the first. An orchestrator that assumed one host's capabilities on the other would either wait forever for a wake-up that never comes, or dispatch into a quota it could have measured.
A held wave resumed itself
Run end-to-end against a spent quota with a deliberately short window, so the whole cycle could be observed rather than argued about. The pre-flight held, reported the wait, the resumption was scheduled through the host's own primitive, it fired, and the resumed pre-flight returned clear.
held 15:32:26Z preflight exit 3 utilisation 100.0% ≥ 95.0% effective ceiling
resets_at 2026-08-12T15:33:56Z resume_after_seconds 79
scheduled 15:32:51Z wait 85s, through the host's own self-scheduling primitive
fired 15:34:17Z elapsed 86s
resumed preflight exit 0 held false clear [metered, native]
"the measured window reset at 2026-08-12T15:33:56Z, so the
recorded utilisation no longer describes it"
The resumed verdict is worth reading closely: the wave reopened because the recorded figure expired, not because a fresh measurement said there was headroom. That is the honest answer — the window rolled over, so the old figure describes nothing, and the next run will measure the new one. It is also what stops one exhausted record holding a project forever.
Nothing about the mechanism is harness-specific except the scheduling step: reckon computes
when — resume_after_seconds and resume_at, both derived from
the backend's own reported reset — and the host reference documents how. Where no such
capability exists, the orchestrator reports the reset and stops: degraded, not broken, and never
a reason to dispatch into the quota anyway.
Suite and generated artifacts
1,657 tests pass, 29 of them new: 21 on the hold decision and its command surface, 4 on the threshold policy resolving through the config layers, 3 on the quarantine boundary, and 1 on the read tool's new view. Lint is clean. The Pydantic model and JSON Schema derived from the LinkML source regenerate byte-identically, so the three new threshold slots and the new summary occasion are not hand-edits waiting to drift.
One pre-existing test changed rather than being added to: it asserted the read tool named four views, and there are now five.