Engineering report
A read-only audit of src/kiss/core, src/kiss/agents/sorcar,
src/kiss/server and src/kiss/agents/vscode recorded 102 findings.
Ninety-four were fixed, each behind a test that failed first. An independent second
reviewer then found 19 more, including three defects introduced by the fixes themselves.
If you are new to this codebase, here is the shape of it. Four
directories do most of the work. src/kiss/core holds the agent loop and one
adapter per model provider — Anthropic, Gemini, OpenAI-compatible endpoints over two
different wire protocols, and two command-line tools (claude and
codex) driven as child processes. src/kiss/agents/sorcar is the
agent that ships: it owns a SQLite task history, git worktrees, a Docker sandbox, MCP
tool servers and a parallel fan-out engine that runs sub-agents in threads.
src/kiss/server is the long-lived daemon: it accepts commands over a unix
socket and a WebSocket, runs tasks in worker threads, and pushes a stream of events back
out. src/kiss/agents/vscode is the client — a TypeScript extension host plus
a webview whose HTML and JavaScript are served, byte for byte, to the remote browser
client as well.
The brief named three defect classes and nothing else.
Ten read-only audits ran in parallel, one per slice of the four directories, each producing a numbered finding list with file and line references, a concrete failure scenario, and a proposed test. Auditors were also asked to record candidates they investigated and rejected, with the reasoning, so that a later reader could see the negative results instead of re-deriving them.
Eleven fix workstreams then took ownership of disjoint file sets. The rule for each finding was the same: write a real end-to-end test, watch it fail for the reason the audit gave, fix the root cause, watch it pass. Where a finding was real but could not be made to fail — a duplicated constant, a dead computation, a docstring — that was stated rather than dressed up as a reproduction.
A second model then reviewed the whole change set read-only, with no permission to edit anything, and was asked specifically to look for code the first pass had missed, wiring it had broken, and tests that could not fail. It reported 19 defects with reproductions. All 19 were real; three of them were introduced by the fixes. They were fixed in a second commit, again test-first.
| Findings recorded by the audits | 102 |
| … fixed, with a regression test | 94 |
| … deliberately left as they are (see Limits) | 2 |
| … partly addressed | 1 |
| … deferred past the first two rounds, then closed in a third | 5 |
| Further defects found by the independent review, all fixed | 19 |
| New test files | 71 |
| Non-test files changed (distinct, across both commits) | 46 |
| Lines added / removed across both commits | 27,526 / 3,101 |
Two of the 102 are duplicates: audits 08 and 09 independently reported the same two unrendered progress events, from opposite ends of the wire. Both audits were right, and the number is left as reported rather than quietly deflated.
Every audit also recorded candidates it investigated and dismissed, which is where a
good deal of the value sits. Four in the model-catalog audit alone: a structured tool
result would be JSON-encoded on one adapter and repr-formatted on three
others, but the only production producer of that dict always writes a string, so it is
latent API drift and not a live bug. Thinking-level aliases such as -xhigh
looked like they could miss the catalog, until parsing the shipped catalog showed all 74
alias keys present as first-class entries. During the fixing round, five more predicted
defects were investigated and rejected on evidence — including one the audit had rated
medium: a failed git cherry-pick was supposed to leave a repository wedged,
and five real-repository probes on git 2.50.1 showed --abort returning 0 and
fully restoring the tree every time, because the code always picks a commit range and a
range pick always writes an abortable sequencer. The guarantee was tightened anyway; the
claim was not made.
uv run check --full green: ruff, mypy, pyright, TypeScript typecheck,
eslint, stylelint, mdformat.What follows is grouped by the code that owned the bug, worst first. "Worst" means what a user would notice, not how clever the bug was.
Tasks can run in a git worktree. When one finishes, the user gets a bar offering merge or discard, and the worktree sits pending until they choose. The daemon decided who still owned that pending worktree using the current run's "use worktree" setting rather than asking whether the agent was still holding one. So if the next prompt on that tab ran without a worktree, the pending branch lost its owner, its preserve marker and nothing else — it stayed registered. A later worktree task's orphan-reclaim sweep then found it and squash-merged it into the working branch. The user had been asked, had implicitly declined by moving on, and the work landed regardless.
The end-to-end reproduction ends with the plainest assertion in the whole change set:
assert 'declined.txt' not in ['declined.txt', 'main_work.txt', 'seed.txt']
Three changes, all at the root: ownership is now decided by
agent._wt_pending, the release guard is no longer gated on the run's mode,
and the "what changed in this worktree" probe no longer returns an empty list for a tab
whose current run happens to be worktree-free — left alone, that last one would have made
the new release path discard real work instead of publishing it.
WorktreeSorcarAgent.auto_commit_enabled was hardcoded True,
and it guarded messages that blamed a --no-auto-commit command-line flag that
does not exist. The user's toggle never reached the agent at all. The visible consequence:
with auto-commit off, typing a follow-up prompt silently committed and squash-merged the
previous task. The toggle now arrives as a run(auto_commit=…)
argument, and an explicit click on the Merge button still commits, because refusing the
user's own merge click was itself a defect in the first attempt at this fix.
The independent review found two paths where the per-run value still lost to the value
persisted on disk: the post-task commit on a non-worktree run ran unconditionally, even
when the task had failed, and the pre-run retirement of a carried-over worktree obeyed
whatever the previous run had bound. Both now read the same
state.auto_commit_mode the runner uses for its own decision. Two existing
tests had written the old behaviour down as policy — one docstring said a dirty main tree
is "ALWAYS auto-committed … auto-commit toggle and task outcome notwithstanding" — which
contradicted the public API contract in server/sorcar.py. They were rewritten
against the contract.
When a parallel sub-agent is abandoned, the parent waits up to five seconds for it
before tearing down the worktree they share. The wait happened after the final
staging pass, so a child that wrote a file and finished inside that window had its work
staged by nobody and then removed with the directory. The reproduction abandons a real
child agent thread, releases it half a second into the parent's five-second wait, and
observes COMMITTED_AND_REMOVED with the file gone from disk and from the
branch. Cleanup now reclaims first, then commits.
On startup the daemon sweeps the task history for rows still marked running and rewrites them as "process killed", on the theory that a running row without a live task must be debris from a crash. Liveness was judged from the booting process's own in-memory registry, which knows nothing about any other process. Start a second daemon, or a second VS Code window, and the first one's running task was relabelled as killed — and stickily, because the rewrite is what the next sweep reads.
Liveness is now a fact on disk. Each process mints a token, holds an exclusive
flock on <KISS_HOME>/task-owners/<token>.lock for its
whole life, and stamps the token into every row it inserts. The kernel releases that lock
however violently the process dies, so a dead owner is provable rather than assumed. The
first implementation of this had its own bug, caught by its own test: it updated rows by
id, a TEXT primary key, and SQLite permits NULL there, so rows written by an
older release could never be targeted — 0 of 300 recovered. It works on
rowid now.
Three sweeps asked "is this task alive?" with a narrower test than the rest of the
codebase uses. A worker that had started but not yet flagged itself busy was invisible to
all three, which meant SIGTERM could abandon it mid-run, and the extension's
activeTasksQuery could report an idle daemon and restart it on top of a task
that had just begun. All three now use the canonical AgentState.busy().
Separately, stop_async() returned while an interactive merge was still
mutating git — measured at 9 ms, with the merge still committing. Merge threads are now
published on the agent state and joined on all three shutdown paths, with a 30-second
bound. They are waited for, never interrupted: half a stash-commit-checkout-merge is worse
than all of it.
VSCodeServer.__init__ called agent_states.clear() on a
process-global registry. Anything embedding two servers in one process — which the test
suite does routinely, and an embedder may do — lost every live task's state on the second
constructor call, including the exemption that keeps a live task out of the orphan sweep.
It is now a selective purge: busy entries stay, finished ones go.
Two independent paths did this. The daemon refuses a run while a merge is in flight,
and it sent the error without the matching status running:false — but the
client had already optimistically marked the tab as running when the user pressed Enter, so
the composer stayed locked with no agent behind it. The refusal now clears the flag first,
matching the sibling refusal a few hundred lines away that always did.
The second path was created by this very change set. Reconnect handling grew a
time-to-live and a queue cap, and dropped frames silently. The review found that a
run dropped by either rule left the tab spinning permanently, because the
extension host posts running:true before the daemon has seen anything. The
client now emits commandDropped(cmd, reason) for both discard paths, the
sidebar unwinds the tab with a warning naming the reason, and a dropped commit-message
request fails its promise and its countdown instead of hanging.
Open two VS Code windows on the same machine and both may decide the daemon needs
restarting. Each probes the port, kills what it finds, and bootstraps — including the
process the other one had just launched. The first fix was an O_EXCL lock
file. The review then showed that lock could be stolen from a live owner: staleness was
judged from the file's mtime after 120 seconds, while the startup verifier alone may run
for 180, plus however long a laptop was asleep. Worse, the release path unlinked the file
unconditionally, so once a lock had been broken the original owner's release deleted its
successor's lock and let a third window in. The lock now carries
{pid, token}, breaks only on a provably dead owner (an EPERM from
kill(pid, 0) counts as alive) or a 600-second backstop, and unlinks only on an
exact token match.
Clicking the sparkle in the SCM view twice discarded the real answer and left the box
empty, because the in-flight guard was registered after an await. It is now
registered synchronously and the second caller joins the first promise. The review then
found a second, quieter half: the command resolved the repository VS Code passed in, used
it to detect staged changes and to pick the input box, and then asked the sidebar to
generate — without passing it. The sidebar sent the workspace folder. In a multi-root or
submodule setup, clicking the sparkle in repository B asked the daemon to diff A. Both the
work directory and a per-repository tab id are now threaded through, which also stops two
repositories' concurrent requests from collapsing into one.
The daemon broadcast autocommit_progress ("Staging changes…", "Generating
commit message…", "Committing…") three times per non-worktree task, and
worktree_progress during a merge. Nothing outside the VS Code extension host
rendered either, so the remote browser and mobile clients showed a frozen UI through
however long a merge takes. The webview now renders both.
Then a follow-up audit of that fix found the harder half. Both flows run inside
except BaseException handlers that log and continue, and the progress events
are emitted from inside the call they wrap. A git binary that dies, an LLM call
that throws while writing the commit subject, or a stopped task unwinding through the
merge leaves the transcript reading "Staging changes…" for ever — not as a stale label, but
as if the operation were still running. The guaranteed events are the task-end ones, so the
line is now cleared from all four of them, routed to the owning tab's fragment rather than
whatever is on screen. Two of that suite's five cases are negative guards: another tab's
task ending must not cancel this tab's live merge line.
The webview's stream state machine existed in four hand-maintained copies: the visible
tab, background tabs, replay, and a fourth that only counted steps. They had already
drifted — the counter seeded one flag false where the renderer it shadowed
seeded it true. Collapsing them into one context object exposed a real bug
that none of the four had: the shared code adopted the daemon's step count from a
result event but had no equivalent rule for the live usage report, so after
every usage line the panel counting resumed from its own stale base. On a
run_parallel fan-out, where the monitor adds every sub-agent's steps to the
parent's, the header visibly went backwards:
thinking_start "Steps: 1"
usage_info "Steps: 43"
thinking_start "Steps: 2" <-- regression
run_parallel is on by default, so most tasks could show this. One rule in
the one shared place fixed it for the visible tab, background tabs and replay at once. The
ctx.stepCount > 0 guard in that rule is load-bearing, and the note in the
fix report says why: zero is also the flag that tells the renderer the first Thoughts panel
is still to be opened, and the monitor commonly reports a step in progress before the first
token arrives.
configData push painted over whatever the user was
typing in Settings. Edited fields are now tracked and skipped, and only touched fields are
sent — safe because the config save merges server-side.parseFloat(v) || 100
swallowed a legitimate 0. The box now shows the daemon's number and says nothing when it
has none.Every other network adapter reads the thread's stop signal and bounds a silent stream. Gemini read neither. Pressing Stop during a Gemini turn did nothing until the provider finished on its own; a wedged gateway held the turn open indefinitely. The adapter now iterates its stream through the shared abort wrapper, with two independent clocks. The second one is not belt-and-braces: the installed SDK skips blank lines while reading the event stream, so ordinary SSE keep-alives — and whatever an eager proxy emits to hold a connection open — are real bytes that reset the transport's read clock while yielding the adapter nothing. Reproduced before fixing, with a harness that sends nothing but blank lines:
generate() was still reading a keep-alive-only stream 20.0s later —
stream_stall_timeout=2.0s is not enforced at the event level, only at the byte level
Two more Gemini defects came with it. Reasoning text was copied into the assistant message on the tool-calling path but not on the plain path, so a tool-using conversation carried its own thoughts forward and paid for them as input tokens on every later step. And usage was read from the last chunk of the stream rather than from the chunk that carries usage. Both paths now share one parts parser, so they cannot disagree again.
claude and codex are driven as child processes. Six defects
sat in that plumbing, and they compounded.
kill(), so the abandoned reader really did survive — and emitted
['LATE'] into the next run's callbacks. Parsing now happens on the
calling thread; the background threads only move bytes, so an abandoned reader has
nothing to emit with.unreaped children: [29959] for codex and two for
claude, counted from real ps output and real /dev/fd
entries.stderr was piped and never drained, so a chatty CLI filled the pipe and
deadlocked. The failure was then reported to the user as a stall: timed out after
20s for a process that was perfectly healthy and had written 200 KiB of
diagnostics.cc/* step aborted the whole task where the equivalent Anthropic condition
would have retried the step. A stall is now a plain TimeoutError.{"tool_calls": …} JSON to the user while the
other stripped it. Both now use one filter that passes thinking through, streams text
incrementally, and buffers each top-level JSON object until it closes before deciding
whether to drop it.None on a JSON null instead of
zero.Then the independent review found the hole in the new supervisor. The turn deadline and the stop check lived in the read loop, and the prompt was written to the child's stdin before that loop was entered. A child that stays alive but never reads stdin fills the pipe buffer, and the write blocks with no timeout and no stop check — reproduced with a 2 MB prompt and a 0.2-second timeout, the writer thread still blocked 1.5 seconds later. The write now runs on its own thread, polled against the same two conditions the read loop obeys.
The abort wrapper accepted a stall_timeout, documented it, and never read
the watchdog's stalled flag — and no caller passed the argument anyway. A
stream that went quiet mid-answer therefore came back as a short but successful reply, and
the agent carried on with a truncated turn. It now raises a retryable timeout, and all
three call sites pass the bound. Related: the watchdog could fire up to one poll interval
after the caller was finished, shutting down a socket already back in the
connection pool. Claiming the abort and disarming it are now serialised under one lock — but
the socket teardown itself runs outside that lock, because the first version of this fix
held the lock across teardown and made Stop flaky one run in five.
The OpenAI-compatible stream loop closed its "thinking" bracket on normal exhaustion and on a stop or a stall, but not on an ordinary transport failure. The agent retries such failures inside the same run, so the retry's plain answer was printed inside the dead stream's Thinking block. The reproduction renders it through a real console printer and quotes the damage:
the answer was rendered inside the dead stream's thinking block —
no closing rule precedes it:
'──────── Thinking ────────\n\nLet me thinkTwo plus two is four'
The bracket now closes in a finally, and a redundant local flag that
shadowed the real one is gone. The test that had previously covered this area asserted the
leak as setup — "test setup no longer reproduces the leak" — which the turn-level
fix makes impossible; it was rewritten to assert the stronger guarantee.
The Chat Completions adapter delegates some turns to a cached Responses-protocol
sibling. That sibling captured its callbacks at construction, so on the second run of a
reused model it streamed into the previous run's printer — the tokens went nowhere
visible. It also rebuilt a whole HTTP client and connection pool before every delegated
step (five distinct TCP peers for five steps), never checked
finish_reason so truncated tool arguments silently became {},
raised a bare IndexError on an empty choices list, and kept a
raw-item cache that grew by one whole turn per tool call and survived a conversation
reset.
One shared cache decided whether an endpoint accepts reasoning_effort
alongside tools. It was an unsynchronised process-global read-modify-write straddling an
HTTP round trip, keyed by base URL alone — so two models on one endpoint contaminated each
other, and parallel sub-agents raced: last writer won. It is now locked, keyed by endpoint
and model, with compare-and-set semantics where a rejection is definitive.
Two JSON files were written with a truncating open and read with an unguarded
json.loads at import time. A reader that arrived mid-write got a parse error;
worse, the seeding of the user-model file was a check-then-act, so a concurrent importer
could silently lose every user-defined model. Both writes are now staged and published with
os.replace, and the reader retries briefly and then raises an error that names
the file. The review added the last piece: valid JSON of the wrong shape
(null, [], 42) escaped as a raw
TypeError from a function whose docstring promised a classified error.
Provider routing was the redundancy half of the same file. One registry was described in
the source as the single source of truth, and three hand-written prefix tables restated it
— one of them as an if-chain. A fourth copy lived in the test helper that decides which
live tests to skip. There is one lookup now, and measuring the helper against the catalog's
601 entries before changing it showed nine behavioural differences, all
codex/*, where the old copy demanded an API key that the Codex CLI does not
use.
The engine that runs sub-agents in a thread pool existed twice: once on the base agent, which around twenty third-party channel agents inherit, and once on the chat agent, which had received four fixes the base copy never got. The consequences were not cosmetic.
KeyboardInterrupt arm, so stopping one child discarded its
finished siblings' results.One engine now, parameterised. About 150 lines of the duplicate went, along with a duplicated stop-event class.
Beyond the orphan sweep, the history database had four more problems. Multi-statement
read-modify-write sequences ran in autocommit under an in-process lock only, so two
processes could exceed a hard cap — 101 rows where the cap is 100, reproduced with two real
processes released by a barrier. Ten no-op commit() calls that read as
durability guarantees were deleted and the sequences wrapped in BEGIN
IMMEDIATE. Events that the database permanently refused were acknowledged as
flushed, i.e. lost; they are now journalled beside their own database file and replayed
once writes are accepted. A cross-file idempotence check held only within one process. And
the flush waited by polling a counter five hundred times a second: an idle task's flush
took 29.9 seconds behind a busy one, now under 0.3.
The review then showed the journal replay was not safe between processes: two of them could persist the same batch twice, or one could delete events the other had just journalled. Replay now takes an inter-process lock and claims the journal by renaming it, so the file it deletes is exactly the file it read. A snapshot the database still refuses is renamed back.
In the sandboxed mode, Write returned "Successfully wrote 24 characters"
for a file that was never created, because the timeout path carries no exit code and success
was assumed. Bash ignored timeout_seconds entirely on the path
the product uses — sleep 30 with a 2-second timeout blocked for the full 30 —
and never truncated output, so a chatty command could flood the model's context. All four
output decodes were strict UTF-8, so a single binary byte raised
UnicodeDecodeError out of the tool, and a multi-byte character split across
two stream frames did the same. Write now proves success with a sentinel,
Bash honours both limits, and decoding is incremental with replacement.
The timeout kill took two attempts. The first used the pid from Docker's exec
inspection, which on macOS is a pid inside the Docker VM's namespace — it would have killed
nothing, or something unrelated. The exec is now tagged with a token in its environment and
the kill script matches /proc/<pid>/environ, using only shell builtins
and tr, because slim images have no ps or
pkill.
Each configuration revision spawned a new stdio child and never reaped the old one, and
the pool had no cap. After a server was killed, the connection object still held what looked
like a live session, so the next tool call blocked until the 305-second timeout rather than
reconnecting. Connections now carry a last-used stamp, idle ones are reaped, the pool is
LRU-trimmed to eight, an idle connection is pinged so a dead server surfaces promptly, and
a call reconnects on demand. Eviction is deliberately not by server name: two projects may
each configure a server called github, and evicting by name would make them
thrash.
The review found the flaw in that cap: it could tear down a connection with a tool call
in flight, stranding the call for the full 305 seconds — the reproduction took 121 seconds
to demonstrate. Connections now carry a lease counter, and leased connections are excluded
from both the idle sweep and the over-cap list. Separately, nothing in the daemon ever
disconnected these servers, so the children were reaped only by an atexit hook
that a kill never runs; a real stdio server was still answering pings 30 seconds after its
daemon stopped.
Chromium's user-data directory is machine-global; the guard around launching it was a thread lock. Two processes resolved the same profile, the second deleted a live singleton lock, and Chromium refused: Failed to create a ProcessSingleton for your profile directory … Aborting now to avoid profile corruption, exit 21. The launch now holds a machine-wide file lock across cleanup, resolution and launch.
The console printer is handed, unchanged, to every parallel sub-agent. Its mid-line, bash-streaming and current-block state was plain instance state, so siblings interleaved into each other's output. The reproduction caught a raw write spliced into a panel border:
'rawtextrawtext…rawtext╭──── Bash ────╮'
Per-thread state, plus one lock around the genuine shared cursor, fixed the rendering. The review then pointed out that the same class's token, cost and step offsets — the numbers printed in a result panel — were still process-shared, so the last sub-agent to start controlled every sibling's rendered totals:
the sibling's token offset won:
╭──── Result ────╮
│ early message │
╰── tokens=910 cost=$9.1000 ─╯
Before moving them, every reader and writer was traced to confirm each runs on the printing agent's own thread — including the live usage monitor, which prints from its own daemon thread but computes its totals itself and reads no offset, so the status line could not regress.
Agent trajectories were written with a truncating open("w") and streamed
into. A concurrent reader — the visualizer is one — saw partial files; the reproduction
observed truncated reads of 451,168, 121,325 and 95,863 bytes among others. There is now
one atomic write helper, and a duplicated 22-line copy of the same staging logic elsewhere
in core now delegates to it.
The review found the helper's own bug: it called os.write() once and
ignored the returned count. POSIX permits a short write, so the helper could atomically
publish truncated content — precisely the outcome it existed to prevent. Reproduced in a
child process with a real 1 KiB file-size limit and SIGXFSZ ignored, so that
write(2) returns its partial count instead of killing the process:
{'requested': 4096, 'published': 1024}. It now uses the buffered
pattern the model-catalog writer had already chosen — the inconsistency was inside the same
commit.
Separately, importing one core module registered a representer on PyYAML's process-global dumper, so merely importing the agent framework changed how an unrelated part of the host program serialised YAML. The style is now applied through a private dumper subclass, and only to genuinely multi-line strings rather than to every key and one-liner.
save_config wrote only keys present in its defaults table, so four live
settings — the tunnel token, skill and MCP permissions, and the email address — were
silently dropped on every save. Refreshing the config after saving an API key rebuilt the
config object and reverted a max_budget that had just been applied. The budget
default itself was written in four places with two different values, 100 and 200; there is
one authoritative constant now, and the last hardcoded literal in the daemon and the three
in the client were removed too. A retired key was still being read back and re-persisted.
And an artifact-directory setter with no production caller — the only runtime mutator of a
process-global — was deleted, which makes a whole class of mid-run relocation
unconstructible rather than merely untested.
No mocks, no patches of production code, no fakes, no test doubles. That constraint is
what makes a race reproducible at all: you cannot mock your way to a genuine interleaving,
and a patched subprocess.run tells you nothing about what a child process does
when its parent gives up on it. The new suites use, literally:
threading.Barrier to force
two probes into the same window, threading.Event rendezvous instead of
sleeps, and threading.enumerate() to prove a watchdog thread did not outlive
its stream.multiprocessing.Barrier to
release two processes into one read-modify-write, real SIGKILL to prove a
dead owner is detected, real /bin/ps and /dev/fd to count
unreaped children and leaked descriptors.PATH — a "CLI" that spawns
a grandchild inheriting stdout so kill() cannot close the pipe; one that
writes 200 KiB to stderr before its first stdout event; one that emits a complete turn,
closes stdout and then sleeps for a minute; one that never reads stdin at all. Also a
git shim that logs its argv and then execs the real binary, used to prove a
command is issued exactly once.ThreadingHTTPServer on an
ephemeral port serving genuine Chat Completions and Gemini SSE bodies to the real vendor
SDKs, with policies that cut a stream after a given event, send nothing but keep-alives,
or return a failed response.pre-commit hook that
blocks so a merge can be caught mid-flight, and a diff.external script that
blocks on a release file so each in-flight commit-message generation is one countable
child process.CREATE TRIGGER … RAISE(ABORT) to make
a table genuinely unwritable rather than pretending it is./bin/bash so the
fixes stay covered on a machine without a daemon), real MCP stdio servers
with a tool that holds a call open, and real unix domain sockets carrying
the daemon's real line protocol.chat.html,
api.js and main.js, driven with real message events and real
clicks, asserting on the real DOM and the real posted payloads.RLIMIT_FSIZE of 1024
bytes in a child that ignores SIGXFSZ, which is the only honest way to make
write(2) return a partial count.PyThreadState_SetAsyncExc to
abandon a running child agent, because that is what an interrupted parent does to it.Where determinism needed help, it came from widening a real window rather than from faking one: an environment-variable delay hook inside the read-modify-write, a printer whose delivery of one specific event is slow (modelling a momentarily busy subscriber), a holder thread that owns a lock for one second. One race resisted this. A plain fifty-iteration loop did not reproduce the file-picker ordering bug reliably, so that approach was discarded and replaced with the slow-subscriber printer, which does.
A test that passes after the fix proves nothing on its own if it also passes after the
fix is removed. Every new webview suite, and every new suite in the review round, was
mutation-proved: back up the production file, break the specific behaviour the test claims
to protect, re-run, restore, and verify the restore with diff -q or a
checksum. Eight mutations across the panel and settings suites, all caught. Twelve further
suites, one targeted mutation each, all caught. Two mutations were not caught and
led to strengthening the tests rather than to a shrug: a numeric step count was masked
because the trailing result event repeated the same number, so a mid-run assertion was
added; a background-tab path was masked for the same reason, so a case was generalised to
run without a trailing result. Two further uncaught mutations were investigated and shown to
be genuine redundancy — two independent paths both paint the step header correctly, and
which one covers a given transcript depends on whether that transcript ends in a result. It
would have been easy to write those off as coverage gaps.
One webview suite claimed to prove that a replayed transcript renders identically to the
live stream. It posted an event named taskEvents. The client handles
task_events, so the frame fell through the dispatcher's default arm and was
dropped: nothing was replayed, the live transcript stayed on screen, and the assertion
compared a snapshot with itself. It could not fail.
Correcting one character exposed two things. First a real bug — the backwards step
counter described earlier. Second, that the test's fixture was impossible: it replayed a
usage event, and usage events are not in the daemon's display whitelist, so they are
broadcast live and then discarded and can never appear in a stored transcript. Its
result event, meanwhile, omitted the step count the daemon always attaches. Fed
a realistic pair, the two paths agree exactly.
The suite now builds its stored transcript by piping the fixture through the daemon's own
filter function in a real subprocess, so the fixture cannot drift from the daemon, and it
asserts that the live panels have left the document — a structural guard against exactly the
vacuity it suffered from. A follow-up audit then checked the other twelve suites the same
way: every inbound event type against a 64-entry vocabulary extracted from the client, every
payload field name, and all 43 outbound command names against the server's catalog. The
conclusion, stated plainly in the notes, was that the taskEvents case was the
only vacuous test.
Fourteen tests matched regular expressions against media/main.js read as
text, and one Python harness ran extracted function bodies against a hand-built DOM
stub — which is why it could not see the module-scope wiring around them and had to be
patched every time the source moved. Collapsing the four stream-machine copies broke all of
them without breaking any behaviour. Rather than re-tune the regexes, each product rule they
protected was identified, checked against the new code, and rewritten as a JSDOM test that
loads the real page and clicks the real gear icon. One rule turned out to be new: replay had
never re-armed the Thoughts panel on a sub-session boundary, so a replayed transcript wrote
the next sub-session's thinking into the previous session's panel. The shared code fixed
that as a side effect.
Three more assertions-on-source went the same way. A test whose decisive line was
assert not hasattr(config_module, "set_artifact_base_dir") became a test that a
trajectory saved late lands in the same file the first save created — the real damage the
deleted setter could cause. Two that spied on subprocess.run arguments became
tests against a real hanging git shim and a real exec boundary. And one JS suite
that regexed a Python constant out of the daemon's source now calls the daemon's real filter
function instead.
Voice tests are environment-dependent and were not run. Twenty-three suites need a real microphone; the 166 figure quoted above is the non-voice set. One voice suite passes but leaks a real audio handle and never exits, which takes the sequential runner down — pre-existing, and no voice file was touched by any of this work.
Two Windows-only paths are unexercised. The file-lock based liveness
marker has no Windows equivalent, so two branches carry honest pragmas: without
flock, the code now reports "no owner" rather than reporting every crashed
process's rows as permanently live. Separately, one audit claimed an unbounded
communicate() hang in the git runner. On this project's CPython that branch is
Windows-only — a pre-fix POSIX run with a one-second budget returned in 1.04 seconds — so
the claim was corrected rather than repeated. The real POSIX damage was a surviving process
group and a git budget that had drifted to a tenth of the shared one.
Five findings survived the first two rounds and were closed in a third.
They are recorded here because they were open for most of this work, and because the second
of them turned out to be worse than its rating suggested. A HEIF converter that crashed left
a partial JPEG behind, and the next candidate's success check accepted it; the shared output
path is now cleared before each attempt. The two OpenAI transports disagreed about which
attachments to keep, and the stricter set is the correct one — both from OpenAI's
documented image-input formats and from the SDK's own Literal["wav", "mp3"]
— which means the looser transport had been putting invalid bodies on the wire. The
three model_config policies are now one rule, driven by each vendor's accepted
parameter set read from the SDK rather than a hand-written table, so seed and
top_k reach Gemini instead of vanishing and an unrecognised key is reported
instead of becoming a TypeError. The tool-result methods now share three
helpers: the base implementation no longer discards attachments, the two command-line
providers state in the prompt when they cannot show one, and an explicit call id is honoured
everywhere — proved by answering two outstanding calls in reverse order, which
positional matching gets wrong. The widened generate() signature is gone; both
command-line adapters read one flag set by the stream filter they already shared.
Three things were deliberately not changed. A diff_merge
module that an audit suspected was dead was kept, because three of its functions have live
callers; the real redundancy inside it — a drifted copy of the git runner — is what was
removed. An unknown-command branch that looks unreachable was kept, because the extension
probes for its exact error string to detect an old daemon. And an accessor with no
production reader was kept, with a docstring explaining why: it is the only way to observe a
registry-lifetime contract that another test already depends on, and deleting the field it
exposes would mean reverting a consistency fix made in the same commit.
Partly addressed. One adapter still hand-rolls the abort loop that the shared wrapper provides. Only the duplicated stall message was collapsed. The adapter needs the stream's final message after the loop, which the shared wrapper does not expose, so merging the two properly is a larger change than this pass took on.
Residual risk. The interesting one is not a defect but a property of the
method: several fixes moved state from process-wide to thread-local, and each of those is
correct only as long as every reader and writer stays on the owning thread. That was traced
case by case, and one of the fix reports says explicitly which sites must keep holding — but
a future contributor adding a helper thread that touches those offsets would reintroduce the
bug quietly. Two smaller items: ruff format reports pre-existing formatting
drift in eleven core files, which the project's own check script does not run and which was
left alone; and a transitional run left a zero-byte lock file beside the real history
database. It is inert and was deliberately not deleted, because unlinking a lock file while
it may be held is the exact race one of these fixes closes.
The three defect classes turned out to be the same defect wearing different clothes. Almost every race traced back to state that was shared because sharing it was invisible: a printer passed unchanged to sub-agent threads, a verdict cache keyed too coarsely, a liveness registry that only knew about its own process. Almost every inconsistency was a guarantee that one path implemented and its sibling did not — Stop, a stall bound, a timeout, an output cap, a rendered event. And the redundancies were where the drift lived: four copies of a state machine, two fan-out engines, three prefix tables beside the registry they restated, two git runners with different budgets. The bugs were in the copies that nobody was looking at.
The independent review earned its place. It found three defects that the fixes themselves had introduced — a bounded read phase with an unbounded write in front of it, an atomic write that could publish a truncated file, a queue cap that stranded a tab — and one test that had been passing for a while without being able to fail.