Chaos run · part two · the console

Clicking Every Button in Qanat

Part one broke the engine. This one drives the console in a real browser: every button, the form, the graph, the keyboard, and what the page does when the server stops answering. 20 problems.

Part one · the engine Part three · the workflow
Browser Chrome 152, 1440×900 Server qanat serve · 127.0.0.1:8421 Project qanat init --demo Console errors on load 0
58 fixed 6 partly fixed 9 still open

Everything below was fixed in v0.1.2, except where a badge says otherwise

Each finding carries its own status. Every fix was checked by re-running the script that found it, and the ones that mattered most are now tests — tests/test_audit_fixes.py holds one case per finding, so each of these mistakes fails loudly if it comes back. The full ledger, with the repro scripts, is in audit/.

One finding cannot be closed and is recorded as a limit rather than a fix: a step is arbitrary Python, so it can always read a source file off disk itself and reach data the as-of views hide. The three doors that were qanat's to close are closed.

The numbers

20 problems
34Buttons checked
21Form inputs
20Problems found
15Keyboard stops on load
2 very bad 8 bad 8 medium 2 small

Three themes.

One. The form does not check what you type. I set the trading cost to -500. Nothing stopped me. The alpha now shows +145.64% in the strategy book, next to the honest numbers, with no mark on it.

Two. The page keeps showing old data as if it were new. When the server hangs, the console still says "connected" and keeps drawing the last numbers it saw.

Three. The pipeline graph is a <canvas> with no keyboard path. Clicking a table is the main thing you do in this console, and you cannot do it without a mouse.

Credit where it is due: all 34 buttons are real <button> elements with real text. Every fetch() checks its status. The "server is down" banner, when you can see it, is honest and well written. This is not a careless front end — the gaps are specific.

How I tested it

three passes, run together
Live browser · me

Drove the real console

Opened qanat serve in Chrome. Clicked every button, opened the form, typed bad values, clicked the canvas, walked the tab order, pressed Escape, paused the server, then killed it.

Agent 1 · source

Read the front end for keyboard gaps

Audited all six console files for click handlers on non-focusable elements, missing labels, focus traps, and motion guards. Gave me file and line numbers to aim at.

Agent 2 · source

Read the front end for error paths

Traced every fetch, every catch, every polling loop, and every destructive action, to find where the UI would mislead rather than fail loudly.

I did not take the agents' word for it. Every finding below that came from a source audit, I then reproduced in the live browser. Two of their claims changed after I checked: Escape does work on the side panel (just not on the form), and the server-down banner is shown and well worded in the default layout. Those corrections are in the findings.

The main problems

8, worst first
Very bad U-01 fixed in 0.1.2 run form · api.py

A negative trading cost is accepted, and the fake result goes into the strategy book

What I did

In the "Run a backtest" form I set commission to -500 and slippage to -500, then pressed run it. Then I sent fee_bps: -9999 straight to the API.

What happened

browser input validity: valid   (no min attribute on #f-fee / #f-slip)
POST /api/backtest  fee_bps=-9999  -> 200 OK

strategy book now reads:
  momentum   net 1938.02%   3 runs · 60d
  low_vol    net 145.64%    2 runs · 5d   (its real result: -5.61%)

Why it matters

A negative cost means you get paid to trade. Turnover becomes profit. The more the alpha trades, the better it looks.

Nothing stops this at any layer. The number input has no min. The console does not check. And BacktestRequest in api.py declares fee_bps: float | None with no limit. The backend does check fee_bps < 0 — but only for the project file, in validate(), not for the per-run override.

The result then lands in the strategy book, which the code describes as "an alpha is in it because it produced a result, which is the only claim worth keeping". A run made at −9999 bps is not a claim worth keeping, and the book has no way to say so. It even becomes the headline number for that alpha, because the book shows the newest run.

How to fix it

  • Add ge=0 to fee_bps and slippage_bps in BacktestRequest. One line, and it closes the API, the console and MCP at once.
  • Add min="0" to #f-fee and #f-slip so the browser catches it before the request.
  • Show the run's conditions on the book row. If a run used costs other than the project's, say so beside the number.
Very bad U-02 fixed in 0.1.2 index.html · dag.js

The pipeline graph cannot be used without a mouse

What I did

Walked the whole tab order from the top of the page, then checked what the graph is made of.

What happened

tab stops on the whole page: 15
canvas reachable by Tab:     false
canvas attributes:           { role: null, aria-label: null, tabindex: null }

the whole page has:  aria-label x 0   role="dialog" x 0   aria-live x 0

Why it matters

The graph is one <canvas id="dagcv">. Every table in the project is painted on it, and clicking one is how you open its detail panel — the code wires this at dag.js:1075 and console.js:421. The canvas has no tabindex, no role, no aria-label, and no fallback content.

So there are 12 tables and 12 jobs in the demo project, and a keyboard user can reach exactly none of them. The hint text under the graph names only mouse moves: "click a table · drag to pan · scroll to zoom · double-click to fit". The fit button is the only one of the four with a real control.

A screen reader gets nothing at all: the whole middle of the app reads as empty.

How to fix it

  • Put a real list inside the canvas element as fallback content — one focusable <button> per table, visually hidden, that fires the same onSelect. The canvas stays the picture; the list is the keyboard path.
  • Give the canvas role="img" and an aria-label summarising the graph ("pipeline: 5 stages, 12 tables, 12 jobs").
  • Add arrow-key panning and +/- zoom once the canvas is focusable.
Bad U-03 fixed in 0.1.2 console.js · backtests.js

A hung server still reads "connected", with old numbers on screen

What I did

Sent SIGSTOP to the server. It keeps the port open but answers nothing — a hang, not a crash. Waited 9 seconds, about four poll cycles.

What happened

connection label:   "connected"
body class:         book-editing results-open      (no server-down)
down banner shown:  false
still showing the old book numbers: true

Why it matters

No fetch() in the console sets a timeout, and there is no AbortController anywhere. A request to a hung server does not fail — it just never returns. So the "disconnected" path never runs, the banner never appears, and every panel keeps showing its last good snapshot as if it were live.

A hang is the more common failure in practice. A crashed server is obvious; a wedged one is not. And qanat can wedge itself — part one showed a job with no timeout holding a worker forever.

Credit: when the server is truly dead, this all works. I killed it and got body.server-down, "disconnected", and an honest banner: "qanat is not answering on this address. The console keeps the last thing it saw, so what is on screen may be old." That is good writing. The gap is only the hang.

How to fix it

  • Wrap the shared api() helper in an AbortController with a timeout of a few seconds, and treat a timeout the same as a network failure.
  • Track the time of the last good response, and once it is older than two poll cycles, mark the panels stale even before the connection is declared dead.
Bad U-04 fixed in 0.1.2 theme.css:642 · index.html:36

When the server is down, the run form opens completely blank

What I did

Killed the server. Folded the graph (a normal thing to do while reading results). Then clicked RUN BACKTEST.

What happened

banner visible, graph open:   true
banner visible, graph folded: false
fold state saved to localStorage: "1"   (survives reload, forever)

modal open:            true
modal body length:     42 characters
warnbox element there: true
warnbox visible:       false
text the user sees:    ""

Why it matters

Two CSS rules combine badly.

The down banner lives inside <div class="view">, and .graph-folded .view { display: none } hides that whole container. The fold is saved to localStorage, so a user who folds the graph once loses the only explanation of a dead server on every visit after that.

Then body.server-down .warnbox { display: none } hides every warning box in the app while the server is down. The run form's failure path fills the modal with only a warnbox — so the modal opens, and it is empty. Same for the alpha editor and the source editor.

Net effect for that user: the server is dead, the console shows a small dot and one word, every panel shows old data, and clicking the main button opens a blank box.

How to fix it

  • Move #down-banner out of .view and into the top bar, so folding the graph cannot hide it.
  • Drop the body.server-down .warnbox { display: none } rule. If it exists to stop banner pile-up, scope it to the panels that duplicate the top-level banner, never to a modal whose only content is the warning.
  • Add a test that opens each modal with the server down and asserts the body is not empty.
Bad U-05 fixed in 0.1.2 backtests.js:915

The backtest-list error handler writes to an element that does not exist

What I did

Looked for #bt-list in the live DOM, because the error path writes to it.

What happened

document.getElementById('bt-list') -> null

the code that runs when /api/backtests fails:
  el('bt-list').innerHTML = '<div class="bt-empty bad">' + esc(e.message) + '</div>'

Why it matters

There is no id="bt-list" anywhere — not in index.html, not in the CSS. The runs-list panel it belonged to was removed, and this error path was left behind.

So when /api/backtests fails, the handler throws a TypeError on null.innerHTML, refresh() rejects, and the user sees no error at all. The results panel keeps drawing the last good run's equity curve and net figure, with nothing marking it stale. This repeats every 4 seconds for as long as the server is unhealthy.

How to fix it

  • Point the handler at a container that exists (#bt-detail), or delete the dead path along with the dead renderList() and renderRunsHead() functions.
  • Add a global window.onerror hook in development that logs to the console — a thrown TypeError in a polling loop should not be silent.
Bad U-06 fixed in 0.1.2 backtests.js:1354

One failed request stops live progress for the rest of the session

What I did

Traced the progress poller, which is what lights up the graph while a backtest walks the window.

What happened

async function tick() {
  try {
    var r = await fetch('/api/backtest/progress');
    if (!r.ok) return;              // no reschedule
    p = await r.json();
  } catch (e) { return; }          // no reschedule
  ...
  schedule(p.running ? 120 : 2500);   // the only reschedule, at the very end
}

Why it matters

Both early returns skip the only line that schedules the next tick. So a single 500, or one dropped connection, kills live progress permanently.

What the user sees: the graph stops lighting up, the running total freezes on whatever rebalance it last saw, and the "run finished, show me the result" handoff never fires. They watch a stalled progress bar for the rest of a long backtest. Only a page reload brings it back.

How to fix it

  • Wrap the body in try/finally and put schedule() in the finally, so every path reschedules.
  • Back off on repeated failures (2.5s → 5s → 10s) instead of stopping, and go back to fast polling on the first success.
Bad U-07 fixed in 0.1.2 api.py · retention.py

A typo in "rebalance" returns a 500 — and the error names the wrong field

What I did

Typed banana into the rebalance box, which is a free text field, and ran it.

What happened

POST /api/backtest  ->  500 Internal Server Error
body the user gets: "Internal Server Error"

server log:
  File "src/qanat/backtest.py", line 92, in dates
  File "src/qanat/retention.py", line 31, in parse_duration
  ValueError: retention must look like '7d' or '24h', got 'banana'

Why it matters

Two problems. First, /api/backtest only catches BacktestError. parse_duration raises a plain ValueError, so it escapes as a 500 and the user gets no explanation at all.

Second, even in the log the message is wrong for the situation. It says "retention must look like '7d'" — but the user typed a rebalance, not a retention policy. parse_duration lives in retention.py and is reused by the backtest for rebalance, purge and embargo, and its message never got updated. Anyone reading that log will go looking at the wrong setting.

How to fix it

  • Give parse_duration a field argument used in the message: "rebalance must look like '5d' or '1h', got 'banana'".
  • Catch ValueError alongside BacktestError in the endpoint and return 422 with the message.
  • Validate the duration in the form on blur, so the user never sends it.
Bad U-08 fixed in 0.1.2 alphaedit.js:155, :369

Delete has no confirmation, and sits right next to save

What I did

Searched the whole front end for confirm( and prompt(.

What happened

confirm() calls in the console: 0
prompt()  calls in the console: 0

DELETE /api/alphas/{id}   — fires straight from the click handler
DELETE /api/stages/{id}   — same

Why it matters

The delete button sits in the same row as save, and one click removes an alpha from the project — the server rewrites qanat.yaml. "Remove this stage" is the same. No question is asked.

The console already has everything it needs to write a good confirmation: the graph payload carries the row count for every table, and the stage panel has already worked out which tables belong to that stage. It could ask "remove features, which holds 3 tables and 412,908 rows?" and instead it asks nothing.

It also throws away the server's answer. Deleting an alpha returns a helpful note — "the script is still on disk, and so is anything it wrote" — and the console never reads it.

How to fix it

  • Add a two-step confirm on both deletes, naming the thing and its row count.
  • Show the server's note and warnings after a successful delete — they are already in the response.
  • Move delete out of the save row, or give it a different weight.

The other twelve

shorter, same evidence
ProblemHow badWhereID
Keyboard and screen reader
The run form is not a dialog. No role="dialog", no aria-modal. Focus never moves into it — after opening, the focused element is still the button behind the overlay. Tab walks straight out into the page underneath. Mediumindex.html:90U-09
partly fixed
Escape closes the wrong thing. One unconditional handler: if (e.key === 'Escape') closeDetail(). Pressing Escape over the run form closes the panel behind it and leaves the form up. I confirmed both: the side panel does close, the form does not. The side panel's button even advertises title="close (esc)", so the app teaches a habit that fails where it is needed most. Mediumconsole.js:467U-10
fixed in 0.1.2
No focus ring anywhere. The whole 642-line stylesheet has one :focus rule, and it is .rrow input:focus { outline: 0; border-color: #a2e65d66 } — a 1px border at 40% opacity, which is not a real indicator. Every other control falls back to the browser default on a near-black background. Badtheme.css:491U-11
fixed in 0.1.2
Hidden panels keep their buttons in the tab order. The book and the detail rail are hidden with width: 0; overflow: hidden, not display: none. I folded the book and counted: 7 of its buttons were still tabbable while invisible. With no focus ring, the user has no idea where they are. Badtheme.css:33, :522U-12
still open
"edit" on an alpha card is mouse-only. It is a <span> with a click handler, it is opacity: 0 until you hover the card, and it sits inside the card's own <button>. So a keyboard user who tabs to the card and presses Enter always gets the card action, never edit. There is no keyboard path to editing an alpha at all. Badbacktests.js:632U-13
still open
11 of 21 form inputs have no label. The shared row() helper puts the <label> beside the control with no for and no wrapping, so the link is visual only. A screen reader announces "edit text, blank" eleven times in the run form — no way to tell the seed from the commission. Mediumbacktests.js:1169U-14
fixed in 0.1.2
Sorting a table needs a mouse. The sort handler is bound to <th>, which is not focusable, with no aria-sort. Sorting is the only way to reorder a 50-row page of a table that may hold millions of rows. Mediumconsole.js:330U-15
still open
Chart drill-down is pointer-only. Clicking a bar opens one rebalance; dragging narrows the whole report. Both are on a plain <div>. The drill panel's own empty state says "Click a bar above" — for a keyboard user that panel can never be filled. Mediumbacktests.js:822U-16
still open
Errors and state
The run can be submitted twice, and the error is wiped. The form closes and the graph lights up before the server answers. A second submit returns a correct 409 "a replay is already running in this project" — but it is inserted into a panel that the 4-second repaint overwrites. The user gets a ~4 second flash, then a lit graph and the previous run's numbers. Badbacktests.js:1207U-17
partly fixed
Errors are shown as raw JSON. All four fetch wrappers do throw new Error(await r.text()), so the user is shown {"detail":"a replay is already running in this project"} verbatim. A Pydantic 422 is worse — a wall of nested JSON. The message is not lost, just never unwrapped. Mediumconsole.js:43U-18
fixed in 0.1.2
Two polling loops overlap and none back off. Four timers at 2.5s, 4s, 5s and 120ms. The 4s and 5s ones both call refresh(), so every cycle sends duplicate GET /api/backtests and GET /api/alphas, with no reentrancy guard — they can interleave and both write the same globals. With the server down the console fires roughly 2.5 requests a second, forever. Mediumbacktests.js:1296U-19
fixed in 0.1.2
Dead affordances. Every run-log row has cursor: pointer and no click handler — I clicked one and nothing happened. There is also no prefers-reduced-motion guard anywhere, including on an infinite pulse animation and a sweep that re-fires once per rebalance during a replay. Smalltheme.css:132, :334U-20
fixed in 0.1.2

What was already right

worth protecting

What I would build

6 changes

Numbers the form cannot lie with

Constrain costs at the model (ge=0), at the input (min="0"), and validate durations on blur. Then show every run's conditions on its book row, so a run at unusual costs can never be read as a normal one.

fixes U-01, U-07

A keyboard path into the graph

Put a visually hidden list of real buttons inside the canvas — one per table, firing the same select. The picture stays for mouse users; the list makes the app operable. This is the single biggest gap in the console.

fixes U-02, helps U-15, U-16

One honest "is this current?" signal

Timeout every fetch, track the last good response, and put the staleness marker in the top bar where nothing can hide it. A hung server and a dead one should look the same to the reader.

fixes U-03, U-04

Errors that survive the next repaint

One toast area outside the polled panels, with role="alert". Unwrap {"detail": ...} into plain text. Then no error can be erased four seconds after it appears.

fixes U-05, U-17, U-18

Ask before deleting

Both deletes name the thing and its row count before doing it, and show the server's note afterwards. The data for a good confirmation is already in the payload the console has.

fixes U-08

A focus pass

One visible :focus-visible ring on every control, display: none for folded panels instead of width: 0, dialog semantics and focus trapping on the two modals, and labels wired with for. Most of this is an afternoon in one CSS file and one helper function.

fixes U-09 to U-14, U-20
Order I would fix them in.
  1. U-01. One line — ge=0 on two fields. A backtesting tool that reports +1938% because someone typed a minus sign is the most serious thing on this page.
  2. U-05 and U-06. Both are a few lines, and both currently make failures invisible.
  3. U-04. Move one element and delete one CSS rule.
  4. U-02. The keyboard path into the graph. Bigger, but it is what makes the console usable by everyone rather than most people.