Theme:

Redact PII — wiring spec

Data flow · states · failure modes · what’s borrowed vs invented · every user-facing string across CLI, mac app, and SPA.

The thesis, up front: almost the entire failure/reporting/journalling apparatus already works for PII through the existing central machinery. The only genuinely new code is the Privacy tab (UI), one piiEnabled UserDefaults key + one env line, and two isinstance lines in the shared exception classifier. No new stage, event, manifest field, orchestration primitive, progress signal, colour, or glyph. (Delivering the detection stack itself — the on-demand Background Assets fetch + [pii] extra + find_pii_stack() — is a separate workstream; see docs/design-redact-pii.md.)

1 · Data flow & the states the data can be in

FullTranscript[]
merged, still contains PII
pii_enabled?
settings, default false
OFF — the default path (pipeline.py:1282)
shallow copy → PiiCleanTranscript
segments passed through · pii_entities_found = 0
downstream (topics…)
no files written · stage not marked in manifest
ON — success (pipeline.py:1264)
mark_stage_running
STAGE_PII_REMOVAL
remove_pii()
Presidio analyse + anonymise, language="en"
PiiCleanTranscript[] + PiiRedaction[]
write + mark_complete
transcripts-cooked/*.txt·md · pii_summary.txt (hidden, re-id key)
ON — failure (F1–F5, see §2)
remove_pii() raises
no stage try/except
propagates unchanged
run_lifecycle catch-all
BaseException → categorise_exception → RunFailedEvent (run_lifecycle.py:799)
row · popover · CLI banner · status page
manifest stage stays “running” — same as every other stage on abandon

Manifest stage states for PII: absent (off) · running · complete · running-then-abandoned (failure — re-runs next time, never poisons cache). Cache-bust contract: the topic stage hashes pii_enabled (pipeline.py:1309), so toggling correctly invalidates everything downstream. Nothing to add.

2 · Failure modes — nailed to the source

IDTriggerRaise siteException (base)Classified TODAYTargetRetry?
F1Presidio not installed_init_presidio import ImportError ✓ MISSING_DEP
isinstance(ImportError)
MISSING_DEPno
F2Model missing from bundle (desktop)_ensure_spacy_model FrozenSidecarError
(RuntimeError)
✗ UNKNOWN
+ engineer text (“PyInstaller datas”)
MISSING_DEPno
F3Model download failed (CLI)ensure_spacy_model PackageInstallError
(RuntimeError)
✗ UNKNOWN
“failed (exit 1)” ∉ _NETWORK_RE
MISSING_DEPyes
F4Disk full writing cooked / summarywrite_cooked_* OSError errno 28 ✓ DISK DISKyes
F5Presidio analyse/anonymise error_redact_text (various) ~ UNKNOWN
acceptable — genuinely rare/opaque
UNKNOWN

The whole gap is two rows. F1 and F4 already classify correctly through the existing catch-all. Only F2 and F3 — the two most likely real-world cases — fall to UNKNOWN and leak engineer-worded text, because categorise_exception’s isinstance ladder doesn’t know these two custom exception types. Fix = extend that one central function; see §3.

3 · Borrow ledger — invent as little as possible

MechanismTemplate / source (file:line)PII todayVerdict
Terminus RunFailedEventrun_lifecycle.py:799 (BaseException)already covers PIIBORROW · 0 change
Exception → Cause classifycategorise_exception:351F2/F3 → UNKNOWNEXTEND · +2 isinstance
Cause → CLI failure bannercli.py:822 MISSING_DEP branchrenders on categoryBORROW · 0 change
Cause → desktop row .failedProjectRow.swift:245renders on categoryBORROW · 0 change
Cause → desktop popover + fixHintProjectDiagnosticPopover.swift · PipelineRunner.swift:1709.missingDep hint existsBORROW · 0 change
Cause → SPA status pageserver/status_page.py:152renders terminusBORROW · 0 change
Manifest running/completemark_stage_* — pipeline.py:1265,1279already calledBORROW · 0 change
Cache-bust provenancepipeline.py:1309 (pii_enabled hash)already hashedBORROW · 0 change
Doctor health checkdoctor.py:616 check_piiexistsBORROW · 0 change
CLI capability matrix rowpipeline_view/cli.py:48–50PII row + reasons existBORROW · reword lg→sm*
CLI success + download line_print_step · preflight.pii.downloadingexistBORROW · 0 change
Per-stage progress event_emit_stage_entry / RunProgressSubtitle.swift:22deliberately folded into neighboursBORROW · leave as-is
Settings → env → sidecarBristlenoseShared.swift:226–274no PII keyNEW · 1 key + 1 line
Native Settings tabSettingsView.swift:41 TabViewno Privacy tabNEW · tab instance

Everything genuinely NEW — and why it earns its place

New thingKindJustification / what it borrows
Privacy Settings tabUIPII has no control anywhere. Exact TabView + in-cell-subtitle idiom of the three existing tabs — a new instance, not a new pattern.
piiEnabled UserDefaults + env linestateSame persist-then-inject pattern as whisperBackend, palette et al. 1 read + 1 env[…] line.
+2 isinstance in categorise_exceptionlogicWithout it F2/F3 → UNKNOWN + engineer text. Extends the existing isinstance ladder in the one central classifier; benefits every caller.
3 Cause messages + tab stringscopyShown for approval in §4. Re-id-safe constant strings (no str(exc)).
Orchestration / events / journallingNONENo new stage, event type, manifest field, or progress signal. pii_summary.txt already written; s07 already logs.
UX tokens (colour / glyph / spacing)NONEReuses MessageKind.error glyph, system red, existing categories. Zero new tokens.

* Phase 0 — model choice: the name is split across 5 sites — sm: s07 constant + the “~12 MB, one-off” download copy; lg: the spec-excludes entry, Presidio’s default engine, the matrix reason string. In the on-demand world (Apple-Hosted Background Assets, not bundled) this is a download-time choice, not a bundle one: sm-stack ≈150 MB vs lg-stack ≈550 MB, both Apple-hosted. Recommend sm (faster first-enable, ~1 F1-point lower NER) and align the three lg sites. Full spec: docs/design-redact-pii.md.

4 · Every user-facing string, all three surfaces

EXISTS ships today, unchanged   NEW / REWORD proposed — approve or adjust the text.

⌨️ CLI
$ bristlenose run interviews/ --redact-pii
    Downloading PII detector (~12 MB, one-off)…    ← EXISTS · preflight.pii.downloading
   Redacted PII (47 entities)  [3.4s]     ← EXISTS · _print_step

# failure (F2/F3) — after the fix, classified MISSING_DEP:
   The language model for redaction isn’t installed.   ← NEW copy (F2/F3 Cause.message)
    Run bristlenose doctor to check your setup.       ← reuses MISSING_DEP banner shape
EXISTS
“Downloading PII detector (~12 MB, one-off)…”
preflight.json:40 · will need lg/sm reconcile if Phase 0 picks lg
EXISTS
“Redacted PII (47 entities)”
pipeline.py:1276 · count via count_noun
EXISTS
matrix reasons: “presidio_analyzer not installed” · “spaCy en_core_web_lg not installed” · “PII redaction is off”
pipeline_view/cli.py:47–50 · reword lg→sm* if Phase 0
NEW
F2/F3 Cause.message: “The language model for redaction isn’t installed.” / F3 “Couldn’t download the redaction model.”
rendered by the existing MISSING_DEP branch, cli.py:822
🖥️ Mac app
Failed run — normal .failed row
Ikea Kitchen Study
The language model for redaction isn’t installed.
Red MessageKind.error glyph + Cause.message. Right-click → Retry. No new UI.
NEW
Tab label “Privacy”
reuses the SPA token settings.json:48 · SF Symbol hand.raised
NEW
Toggle “Redact personal information” (approved)
NEW
Subtitle “Automatically remove names, emails, phone numbers, and other personal details from transcripts before they are analysed. Redaction imperfect – do check your transcripts. Applies to the next analysis, onward.” (approved)
NEW
Caveat (non-English UI only) “Detection is best for English. Redaction in other languages less thorough.” (approved)
EXISTS
Link “How redaction works →” → redact-pii.html
href already shipped · WelcomeHomeView.swift:45
EXISTS
Popover header “Run failed” + fixHint “Setup needed — a required tool isn’t installed.”
desktop.json:284 · PipelineRunner.swift:1709 (.missingDep)
EXISTS
Long-form intro “PII redaction is opt-in. It’s off by default because false positives… uses Microsoft Presidio…”
desktop.json:360 · available if the tab wants a footer/help
🌐 SPA (web report / serve)
EXISTS
Config-reference (read-only): category “Privacy” · “PII redaction” · “PII LLM pass” · “Custom names to redact”
settings.json:84–87 — documents the env vars; not an interactive control
EXISTS
Failure → server-rendered status page “Run failed” + Cause.message
status_page.py:152 — same terminus, no new code
FLAG
? The config-reference still lists inert PII LLM pass / Custom names. Recommend dropping or marking them — they warn and do nothing. Your call; SPA-only.

The interactive toggle is native-only (like Whisper backend, palette, etc.) — the web SPA keeps its read-only config-reference. No new SPA control in this plan.

5 · Build order

  1. Phase 0 — pick the model (recommend sm); align s07 constant · Presidio NlpEngine config · spec · matrix reason string · download copy. Prove redaction in a real unpacked layout, not just the dev venv. (Delivery of that stack — the on-demand Background Assets fetch — is Phase 2; see docs/design-redact-pii.md.)
  2. Classifier — +2 isinstance in categorise_exception (FrozenSidecarError, PackageInstallError → MISSING_DEP) with clean re-id-safe messages. Unit test: F1–F5 each map to the target category.
  3. Privacy tab — new PrivacySettingsView + piiEnabled UserDefaults key + one env["BRISTLENOSE_PII_ENABLED"] line in BristlenoseShared.swift. Register the tab in SettingsView.
  4. Copy / i18n — the ~5 new strings across the 20 full locales (deferrable per the hand-tune-first convention). SPA inert-var cleanup if approved.