harp — Real-time streaming & back-patch dictation

Goal

Turn harp into a local-first, real-time dictation engine: as you speak, transcribed text appears in the focused application with a few-seconds delay — typed key-by-key as if you were typing it — and earlier text is silently corrected (delete + retype) when later transcription passes revise it. This is the "Whisper Flow" effect. Real-time matters because a 2-hour dictation (e.g. a meeting) cannot wait until the end for its transcript. This spec covers slice A only: the streaming core + back-patch typing inside the existing keyboard daemon.

Locked decisions

1. Stable-prefix + volatile tail. Only the unstable tail (last few seconds) is ever rewritten. Once a segment is old enough and agrees across passes it is committed and never touched again — bounds the blast radius of the dangerous backspace path over a 2-hour session.
2. Approach 1 — re-decode rolling window + LocalAgreement-2. Re-transcribe a sliding window each tick; commit the longest prefix that agrees across two successive hypotheses. De-facto standard for local streaming Whisper; zero new dependencies.
3. Drop command mode + cloud LLM entirely. Remove api.py, LLMClient, Ctrl+Shift+Space, the openai dependency and the llm_* / command_prompt / send_clipboard config. harp becomes a pure local dictation engine. (Folds the old "slice C" cleanup into this work since the streaming rewrite touches the same stop path.)
4. Streaming is the only mode. No batch fallback — the commit policy degenerates correctly for short utterances (everything commits on stop). One activation: Ctrl+Space toggles a streaming session; hold-to-talk variant analogous.
5. Streaming core is I/O-free. StreamingTranscriber depends only on an injected transcribe() callable — so the future REST server (slice B, Magpie substrate) feeds it identically to the daemon. That reuse boundary is the reason it is a standalone module now.
6. Cadence defaults are empirical. Slide interval / window / model defaults will be tuned by measurement per model size and CUDA availability, not guessed up front.

Components

Data flow

1. mic → AudioStreamer callback appends PCM to buffer (existing).
2. Daemon slide loop (every slides while RECORDING) pulls newly-arrived samples and calls transcriber.feed(pcm).
3. transcriber.step() re-decodes the rolling window via LocalWhisperEngine.transcribe wrapped in run_in_executor (keeps the event loop responsive); returns TranscriptState(committed, tail).
4. IncrementalTyper.update(committed + tail) diffs against what is already on screen and emits the minimal backspace + type sequence into the focused window.
5. Ctrl+Space again → one final feed/step, commit the remaining tail, end the session.

def step(self) -> TranscriptState:
    # re-decode window [commit_point - overlap : now]
    hyp = self._transcribe(self._window(), self._committed_text, self.lang)
    agreed = longest_common_prefix(hyp, self._prev_hyp)   # LocalAgreement-2
    self._commit(agreed)            # append to committed, trim audio buffer
    self._prev_hyp = hyp
    return TranscriptState(self._committed_text, hyp[len(agreed):])

Commit policy (LocalAgreement-2)

• Keep the previous hypothesis. Each tick, the longest common prefix of the new and previous hypotheses is "agreed" and appended to the immutable committed text.
• After a commit, audio before (commit_point − overlap) is dropped so decode cost stays bounded regardless of session length.
• The committed text is fed back as initial_prompt for continuity across the trim boundary.
• On session stop, the current tail is force-committed (no second hypothesis needed — recording is over).
• Tunable knobs (defaults set empirically): slide_interval, window, overlap, agreement passes (default 2).

Removal scope

Delete: src/harp/api.py, LLMClient usage in daemon.py, the Ctrl+Shift+Space command-mode branch, _get_clipboard_context, the openai dependency in pyproject.toml, and the llm_api_key / llm_base_url / llm_model / command_prompt / send_clipboard fields in config.py + their CLI options in __main__.py.

Keep: copy_result (clipboard copy of the final text — independent of the cloud), all local_* Whisper config, model-management CLI, the evdev grab / uinput passthrough machinery.

Replace: the half-built continuous mode + _background_transcription_loop + the batch finalize block in _stop_recording are superseded by the streaming loop and removed.

Error handling

• Decode exception in a step → skip that step, keep the buffer, dim-log; never crash the session (existing pattern).
• Decode slower than slide → coalesce; never queue ticks. Effective cadence degrades to decode time; warn once.
• User types mid-session → pause_typing defers emission; reconcile in one batch when quiet.
• Backspace cap exceeded (e.g. language flip rewrites a huge tail) → stop rewriting history, switch to append-only for the rest of the session, desktop-notify.
• Blind-typing caveat → uinput cannot observe the target app, so a manual edit or focus change inside the typed region mid-dictation is undetectable. Bounded by the backspace cap; documented as "do not hand-edit while dictating". Detection is out of scope for slice A.

Testing

Tests are written and validated inline (not delegated — they are the verification layer).

• StreamingTranscriber units — scripted fake transcribe() returning a controlled hypothesis sequence; assert commit/tail evolution, LocalAgreement-2 correctness, buffer trim, prompt continuity. Pure, fast, no audio.
• IncrementalTyper units — recording fake typer capturing (backspace_count, typed_str) ops; assert minimal diff, committed-prefix preservation, pause deferral, cap-exceeded fallback.
• Daemon streaming loop — async test with fake streamer + transcriber + typer; assert final committed text on stop and no crash when a transcribe step raises.
• Integration — existing voice-integration-test pattern over a real short WAV (marked integration); assert final text close to expected.
• Live manual run — the empirical cadence-tuning step: measure decode time per model/device, set sensible defaults, verify the feel (latency, no corruption).
• Removal verification — openai absent from pyproject.toml, no LLMClient import, full suite green, ruff clean.

Non-goals

• Slice B — REST server mode. Own spec/plan/build cycle. The I/O-free core is designed so B reuses it; B is not built here.
• Slice D — Magpie integration. Downstream consumer of B; separate cycle.
• Approach 2 (word-timestamp tail-only re-decode) deferred — a later optimization if decode cost dominates.
• Detecting manual edits / focus changes in the target app.
• Local instruction LLM to replace command mode — explicitly not wanted in this vision.