Metadata-Version: 2.4
Name: shadow-clerk
Version: 0.6.0
Summary: Real-time audio recording and transcription tool for web meetings
Author-email: Atsushi Kato <ktat.is@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/edocode/shadow-clerk
Project-URL: Repository, https://github.com/edocode/shadow-clerk
Project-URL: Issues, https://github.com/edocode/shadow-clerk/issues
Keywords: meeting,transcription,speech-to-text,whisper,real-time,audio,japanese,asr,translation
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: End Users/Desktop
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Sound/Audio :: Capture/Recording
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Classifier: Topic :: Office/Business
Classifier: Natural Language :: English
Classifier: Natural Language :: Japanese
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: faster-whisper>=1.0.0
Requires-Dist: sounddevice>=0.4.6
Requires-Dist: webrtcvad-wheels>=2.0.11
Requires-Dist: numpy>=1.24.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: openai>=1.0.0
Requires-Dist: pynput>=1.7.6
Requires-Dist: evdev>=1.6.0; sys_platform == "linux"
Requires-Dist: PyAudioWPatch>=0.2.12; sys_platform == "win32"
Requires-Dist: pywinpty>=2.0.13; sys_platform == "win32"
Requires-Dist: langdetect>=1.0.9
Requires-Dist: pyte>=0.8.2
Requires-Dist: markdown-it-py>=3.0
Requires-Dist: onnxruntime>=1.30.0
Provides-Extra: spell-check
Requires-Dist: transformers>=4.30.0; extra == "spell-check"
Requires-Dist: torch>=2.0.0; extra == "spell-check"
Requires-Dist: sentencepiece>=0.1.99; extra == "spell-check"
Provides-Extra: reazonspeech
Requires-Dist: sherpa-onnx>=1.12.28; extra == "reazonspeech"
Provides-Extra: moonshine
Requires-Dist: moonshine-voice>=0.1.5; extra == "moonshine"
Provides-Extra: gcal
Requires-Dist: google-auth-oauthlib>=1.0.0; extra == "gcal"
Requires-Dist: google-api-python-client>=2.0.0; extra == "gcal"
Provides-Extra: diarization
Requires-Dist: onnxruntime>=1.16.0; extra == "diarization"
Requires-Dist: kaldi-native-fbank>=1.17.0; extra == "diarization"
Provides-Extra: pronounce
Requires-Dist: transformers>=4.30.0; extra == "pronounce"
Requires-Dist: torch>=2.0.0; extra == "pronounce"
Requires-Dist: espeakng-loader>=0.2.4; extra == "pronounce"
Requires-Dist: phonemizer-fork>=3.3.2; extra == "pronounce"
Provides-Extra: kana-hint
Requires-Dist: transformers>=4.30.0; extra == "kana-hint"
Requires-Dist: torch>=2.0.0; extra == "kana-hint"
Requires-Dist: sudachipy>=0.6.8; extra == "kana-hint"
Requires-Dist: sudachidict-core>=20240409; extra == "kana-hint"
Dynamic: license-file

# Shadow Clerk

[![PyPI version](https://img.shields.io/pypi/v/shadow-clerk.svg)](https://pypi.org/project/shadow-clerk/)
[![Python versions](https://img.shields.io/pypi/pyversions/shadow-clerk.svg)](https://pypi.org/project/shadow-clerk/)

A tool that records web meeting audio (your microphone and the speaker output) in real time and transcribes it, with live translation and meeting minutes. On top of that it can:

- run an AI assistant (Claude Code or Codex) beside the meeting in the **AI Console**, which points out open questions as the meeting goes and can write the minutes;
- let you **talk a topic through with Claude** by voice (VOICEVOX speech), and on Linux with PipeWire send Claude's voice into a meeting app;
- start and end meeting sessions from **Google Calendar**;
- put **browser screenshots** on the transcript timeline with a Chrome extension.

It runs on Linux and Windows. Transcription and LibreTranslate translation work fully offline; the rest is optional.

Available on PyPI: <https://pypi.org/project/shadow-clerk/>

## Privacy and Usage Notes

- **Audio is processed locally.** Transcription runs on-device using Whisper or another local ASR model. Raw audio exists only in memory during processing and is never sent to any external server or written to disk.
- **Transcripts are stored locally.** Text files are saved to your machine only (`~/.local/share/shadow-clerk/`). They are not uploaded anywhere automatically.
- **LLM calls send transcript text externally.** When you use an LLM (Claude, OpenAI, etc.) for translation or meeting minutes, the transcript text is sent to that provider's API. Anthropic and OpenAI do not use API requests for model training by default, but check your plan's data policy. **Claude Code Team or Enterprise plans provide contractual guarantees of no data retention.**
- **Consider consent from meeting participants.** Recording a conversation without the knowledge of others may raise legal or ethical concerns depending on your jurisdiction and context. It is generally advisable to inform participants that the conversation is being recorded and to obtain their consent where appropriate.

## Contents

- [Platform support](#platform-support)
- [Features and requirements](#features-and-requirements)
- [Setup](#setup)
- [Docker](#docker)
- [Usage](#usage)
  - [Starting the daemon](#starting-the-daemon) · [Recording & transcription](#recording--transcription) · [Audio device selection](#audio-device-selection) · [Audio level meters](#audio-level-meters)
  - [Dashboard](#dashboard) · [Voice commands](#voice-commands) · [CLI options](#cli-options) · [clerk-util subcommands](#clerk-util-subcommands)
  - [Translation & Summary Providers](#translation--summary-providers) · [Talk with Claude](#talk-with-claude) · [Meeting minutes](#meeting-minutes) · [AI Console](#ai-console) · [Browser screenshots](#browser-screenshots)
- [Configuration](#configuration)
- [File structure](#file-structure)
- [Troubleshooting](#troubleshooting)
- [Building standalone binaries](#building-standalone-binaries)

## Platform support

| OS | Status | Notes |
|----|--------|-------|
| Linux (PipeWire/PulseAudio) | Supported | Primary development target |
| Windows 10/11 | Supported | Monitor capture via WASAPI loopback (default playback device) |
| macOS | Not supported yet | Requires a virtual audio driver (e.g. BlackHole) — not implemented |

### Windows-specific notes

#### Prerequisites (PowerShell, uv, Git)

A fresh Windows install has neither `uv` nor `git`. Run every command below in **PowerShell**:

- **Open PowerShell**: press the Windows key, type `powershell`, and choose **Windows PowerShell** (or right-click the Start button → **Terminal**). No administrator rights are needed.
- **Install uv** (pick one):
  ```powershell
  winget install --id=astral-sh.uv -e
  # or the official installer:
  powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
  ```
- **Install Git** (needed for the ReazonSpeech k2 package, which is fetched from a Git repository):
  ```powershell
  winget install --id Git.Git -e
  ```
- **Close and reopen PowerShell** so the new `PATH` takes effect, then check with `uv --version` and `git --version`.

If `winget` is not found, install **App Installer** from the Microsoft Store, or download uv from <https://docs.astral.sh/uv/getting-started/installation/> and Git from <https://git-scm.com/download/win>.

#### Install

Recommended install (explicit Windows deps):

```powershell
uv python install 3.13
uv tool install --python 3.13 --with PyAudioWPatch --with sherpa-onnx --with "reazonspeech-k2-asr @ git+https://github.com/reazon-research/ReazonSpeech.git#subdirectory=pkg/k2-asr" "shadow-clerk[spell-check,gcal,reazonspeech,moonshine,diarization,pronounce,kana-hint]"
# Add the tool directory (%USERPROFILE%\.local\bin) to PATH, then reopen PowerShell:
uv tool update-shell
```

Why explicit `--python` and `--with`:

- **`--python 3.13` (uv-managed Python)**: Microsoft Store Python runs in an AppContainer sandbox that redirects `%APPDATA%\shadow-clerk` to `%LOCALAPPDATA%\Packages\PythonSoftwareFoundation.Python.X.YY_<id>\LocalCache\Roaming\shadow-clerk\`. The package id changes when the Python minor version is upgraded, silently moving the data directory and orphaning existing transcripts/config. uv-managed Python avoids the sandbox. Daemon startup also logs a WARNING when Store Python is detected.
- **`--with PyAudioWPatch`**: WASAPI loopback monitor capture uses [PyAudioWPatch](https://github.com/s0d3s/PyAudioWPatch). It's declared as a Windows-only dep in `pyproject.toml` but some uv versions don't reliably resolve PEP 508 markers from local-editable installs, so passing it explicitly is safer.
- **`--with sherpa-onnx`** (ReazonSpeech only): Same reason — ensures uv picks the Windows wheel (with `onnxruntime.dll`) rather than a stale resolution to the Linux wheel.

Other notes:

- **Microphone permission**: Allow mic access for the terminal you launch from (Windows Settings → Privacy → Microphone).
- **Monitor capture**: Uses WASAPI loopback on the system default playback device. Switching the default device in Windows sound settings switches what gets captured.
- **Data directory**: `%APPDATA%\shadow-clerk` (the `~/.local/share/shadow-clerk` paths in the rest of this README map to that on Windows). Override with `SHADOW_CLERK_DATA_DIR` if needed.
- **Remote Desktop (RDP)**: When running inside an RDP session, the host's "Remote Audio" virtual device is auto-skipped (it would either segfault or capture nothing useful). The daemon falls back to a non-RDP loopback device if available; otherwise monitor capture is disabled and only the mic is recorded.
- **`voice_command_key`**: The default `f23` is a Linux/xremap convention. On Windows set it to `null` (disable PTT) or to one of `menu`/`ctrl_r`/`ctrl_l`/`alt_r`/`alt_l`/`shift_r`/`shift_l` in `config.yaml`.
- **Stopping the daemon**: `clerk-util stop` works (Windows path uses `taskkill`). `clerk-util start` runs the daemon in the foreground with Ctrl+C handling, mirroring Linux. `clerk-daemon --daemon` detaches instead: there is no `fork()`, so it relaunches itself with `DETACHED_PROCESS` and the parent exits.
- **AI Console**: uses ConPTY through [pywinpty](https://github.com/andfoy/pywinpty), declared as a Windows-only dependency. Without it the daemon still runs; only the console reports the missing package and refuses to start.
- **Talk with Claude**: sending Claude's voice into a meeting app needs PipeWire, so that option is disabled on Windows.
- **Standalone binary**: `clerk-daemon.exe` and `clerk-util.exe` can be built with PyInstaller. See [Building standalone binaries](#building-standalone-binaries).

#### Desktop shortcuts

To start and stop the daemon without typing commands, create shortcuts once. Run this in PowerShell after installing:

```powershell
$exe = Join-Path (uv tool dir --bin) "clerk-util.exe"
$shell = New-Object -ComObject WScript.Shell
foreach ($s in @(@("Shadow Clerk", "start -d"), @("Shadow Clerk (stop)", "stop"))) {
    $lnk = $shell.CreateShortcut("$([Environment]::GetFolderPath('Desktop'))\$($s[0]).lnk")
    $lnk.TargetPath = $exe
    $lnk.Arguments = $s[1]
    $lnk.WindowStyle = 7  # minimized
    $lnk.Save()
}
```

**Shadow Clerk** starts the daemon in the background; open the dashboard at <http://localhost:8765>. **Shadow Clerk (stop)** stops it.

To create one by hand instead: right-click the desktop → **New** → **Shortcut**, enter `%USERPROFILE%\.local\bin\clerk-util.exe start -d` (`uv tool dir --bin` prints the folder if it differs), and name it.

To start the daemon when you sign in, copy the **Shadow Clerk** shortcut into the Startup folder (press Win+R and enter `shell:startup` to open it).

## Features and requirements

| Feature | Requires | Quality | Speed | Related settings |
|---|---|:---:|:---:|---|
| Transcription (default) | faster-whisper (included) | 3 | 4 | `default_model`, `default_language` |
| Transcription (Kotoba-Whisper) | Same (auto-downloaded on first use) | 5 | 3 | `japanese_asr_model: kotoba-whisper` |
| Transcription (ReazonSpeech) | `uv sync --extra reazonspeech` | 5 | 4 | `japanese_asr_model: reazonspeech-k2` |
| Transcription (Moonshine Voice) | `uv sync --extra moonshine` | 4 | 5 | `asr_engine: moonshine` |
| Interim transcription | Same | 2 | 5 | `interim_transcription: true`, `interim_model` |
| Translation (LibreTranslate) | LibreTranslate server | 2 | 4 | `translation_provider: libretranslate` |
| Translation (OpenAI compatible API) | OpenAI compatible API | 3-5 | 2-5 | `translation_provider: api`, `api_endpoint`, `api_model` |
| Translation (Claude) | Claude Code CLI (`claude -p`) | 5 | 2 | `translation_provider: claude` |
| Language detection (pre-translation) | langdetect (included) | — | — | Automatically detects source language to select correct prompt |
| Summary (Claude) | Claude Code CLI (`claude -p`) | 5 | 3 | `llm_provider: claude` |
| Summary (OpenAI compatible API) | OpenAI compatible API | 3-5 | 2-5 | `llm_provider: api`, `api_endpoint`, `api_model` |
| Voice commands (PTT) | None (built-in) | — | — | `voice_command_key` |
| Voice commands (LLM matching) | OpenAI compatible API | — | — | `llm_provider: api`, `api_endpoint`, `api_model` |
| Spell check (pre-translation) | transformers (auto-downloaded on first use) | — | — | `libretranslate_spell_check: true` |
| [AI Console](#ai-console) (meeting assistant) | Claude Code or Codex CLI, skill from `clerk-util install-skill` | — | — | `auto_analyze`, `ai_assistant_command` |
| [Talk with Claude](#talk-with-claude) | Claude Code CLI, a running VOICEVOX engine | — | — | `talk_*` |
| Send Claude's voice into a meeting app | Linux with PipeWire (`pw-dump`, `pw-link`, `pw-cat`) | — | — | `talk_route_app` |
| [Speaker diarization](#optional-speaker-diarization) (`[相手A]`, `[相手B]`… on the remote side, naming, voiceprints, names during the meeting) | `diarization` extra | — | — | `speaker_diarization`, `speaker_enrollment` |
| [Google Calendar](#optional-google-calendar-integration) (auto start/end, expected attendees) | `gcal` extra, OAuth credentials | — | — | `gcal_integration`, `gcal_credentials_file` |
| [Pronunciation feedback](#optional-pronunciation-feedback) (phoneme-level, English, for `clerk-practice`) | `pronounce` extra | — | — | `pronounce_enabled` |
| [Mishearing hints](#optional-mishearing-hints) (`リプロイ(でぷろい?)`, Japanese) | `kana-hint` extra | — | — | `kana_hint` |
| [Browser screenshots](#browser-screenshots) in the transcript | Chrome extension in `extension/` | — | — | — |

**Minimal setup without LLM:** Transcription + LibreTranslate translation requires no external API or Claude Code. Everything runs locally.

See the [Feature Tour](docs/feature-tour.en.md) for a visual walkthrough with screenshots (taken of the previous dashboard, now at `/legacy`).

## Setup

### 1. System packages (Linux only)

```bash
sudo apt install libportaudio2 portaudio19-dev
```

On Windows no system package is needed — the `sounddevice` wheel ships PortAudio. Use the install command in [Windows-specific notes](#windows-specific-notes) instead of step 2.

### 2. Install

Install from PyPI:

|  | Command |
|---|---|
| Basic | `uv tool install shadow-clerk` |
| + ReazonSpeech | `uv tool install "shadow-clerk[reazonspeech]" --with "reazonspeech-k2-asr @ git+https://github.com/reazon-research/ReazonSpeech.git#subdirectory=pkg/k2-asr"` |
| + Spell check | `uv tool install "shadow-clerk[spell-check]"` |
| + Both (ReazonSpeech + Spell check) | `uv tool install "shadow-clerk[spell-check,reazonspeech]" --with "reazonspeech-k2-asr @ git+https://github.com/reazon-research/ReazonSpeech.git#subdirectory=pkg/k2-asr"` |
| + Moonshine Voice | `uv tool install "shadow-clerk[moonshine]"` |
| + Google Calendar | `uv tool install "shadow-clerk[gcal]"` |
| + Speaker diarization | `uv tool install "shadow-clerk[diarization]"` |
| + Pronunciation feedback | `uv tool install "shadow-clerk[pronounce]"` |
| + Mishearing hints | `uv tool install "shadow-clerk[kana-hint]"` |
| All | `uv tool install "shadow-clerk[spell-check,gcal,reazonspeech,moonshine,diarization,pronounce,kana-hint]" --with "reazonspeech-k2-asr @ git+https://github.com/reazon-research/ReazonSpeech.git#subdirectory=pkg/k2-asr"` |

> **Note:** `uv tool install` maintains a single environment per tool. When reinstalling with different extras, use `--force` — without it, `uv tool install` reports "already installed" and does not add the extra. Only the extras specified in the command are included; previously installed extras are removed.

> **Why `reazonspeech-k2-asr` is `--with`'d separately:** The `reazonspeech-k2-asr` package is only distributed via Git (not on PyPI), and PyPI rejects direct URL references in package metadata, so it cannot be declared inside the `reazonspeech` extra. Pass it via `--with` whenever you need ReazonSpeech k2.

Alternative: `pipx install shadow-clerk` (or `pip install shadow-clerk` inside a venv) also works.

### 2a. For development

```bash
git clone https://github.com/edocode/shadow-clerk.git
cd shadow-clerk
```

|  | Command |
|---|---|
| Basic | `uv sync` |
| + ReazonSpeech | `uv sync --extra reazonspeech` |
| + Spell check | `uv sync --extra spell-check` |
| + Both (ReazonSpeech + Spell check) | `uv sync --extra spell-check --extra reazonspeech` |
| + Moonshine Voice | `uv sync --extra moonshine` |
| + Google Calendar | `uv sync --extra gcal` |
| + Speaker diarization | `uv sync --extra diarization` |
| + Pronunciation feedback | `uv sync --extra pronounce` |
| + Mishearing hints | `uv sync --extra kana-hint` |
| All | `uv sync --extra spell-check --extra gcal --extra reazonspeech --extra moonshine --extra diarization --extra pronounce --extra kana-hint` |

**Each `uv sync` defines the whole environment, and extras do not accumulate.**
`uv sync --extra gcal` after `uv sync --extra reazonspeech` removes ReazonSpeech
again — name every extra you want in the same command, as in the last row above.
For the same reason, anything installed with `uv pip install` is undeclared and
is removed by the next `uv sync`, so run it last.

This is all you need for transcription. The following optional extras are available:

### Optional: Japanese ASR models

**Kotoba-Whisper** — No extra install required. The model is auto-downloaded on first use. Just set:

```yaml
# config.yaml
japanese_asr_model: kotoba-whisper
```

**ReazonSpeech k2** — Requires the `reazonspeech` extra plus the `reazonspeech-k2-asr` package, which is only distributed via Git (not on PyPI), so it must be installed separately:

```bash
uv tool install "shadow-clerk[reazonspeech]" \
  --with "reazonspeech-k2-asr @ git+https://github.com/reazon-research/ReazonSpeech.git#subdirectory=pkg/k2-asr"
# or for development — in this order, and do not run uv sync afterwards:
uv sync --extra reazonspeech
uv pip install "reazonspeech-k2-asr @ git+https://github.com/reazon-research/ReazonSpeech.git#subdirectory=pkg/k2-asr"
```

`reazonspeech-k2-asr` cannot be declared in `pyproject.toml` (PyPI rejects direct
URL references in metadata), so `uv sync` does not know about it and removes it.
Check what you have with:

```bash
uv run python -c "import sherpa_onnx, reazonspeech.k2.asr; print('ok')"
```

```yaml
# config.yaml
japanese_asr_model: reazonspeech-k2
```

### Optional: Mishearing hints

Japanese only. A second model (a hiragana wav2vec2 CTC model, 1.2GB, downloaded on first
use) listens to the same audio and writes down what it hears in kana. Where that disagrees with
the reading of the transcribed word, the heard kana is appended to the word:

```
きょうの午後にリプロイ(でぷろい?)しますか
```

The transcript is never rewritten — which word was meant is left to the reader (or to the
`clerk-meeting-helper` skill, which knows the format). A hint is added only when a CTC score says
the heard kana fits the audio clearly better than the transcribed reading. Translation strips the
hints before sending text to the translator. No audio is stored.

```bash
uv sync --extra kana-hint      # transformers, torch, SudachiPy + SudachiDict (morphological analysis)
```

Enable "Mishearing hints" in the settings (`kana_hint: true`). It does nothing when the
transcription language is not `ja`. On CPU it adds about 0.2–0.5 s per line.

### Optional: Spell check (pre-translation correction)

Requires the `spell-check` extra (installs `transformers`, `torch`, `sentencepiece`):

```bash
uv tool install "shadow-clerk[spell-check]"
# or for development:
uv sync --extra spell-check
```

```yaml
# config.yaml
libretranslate_spell_check: true
spell_check_model: mbyhphat/t5-japanese-typo-correction  # default
```

The spell check model is auto-downloaded on first use. It corrects Japanese speech recognition typos before sending text to LibreTranslate.

### Optional: Speaker diarization

Tells the remote participants apart in the transcript. Requires the `diarization` extra (installs `onnxruntime` and `kaldi-native-fbank`):

```bash
uv tool install "shadow-clerk[diarization]"
# or for development:
uv sync --extra diarization
```

```yaml
# config.yaml (or the dashboard settings modal, Transcription section)
speaker_diarization: true   # default false
diarization_threshold: 0.70 # cosine similarity (default 0.70)
diarization_min_sec: 1.0    # shortest utterance to identify, in seconds (default 1.0)
```

The wespeaker ResNet34 ONNX model is downloaded on first use into `wespeaker_cache/` in the data directory.

- Utterances from the remote (monitor) side get `[相手A]`, `[相手B]`… labels. `[相手?]` means the utterance was too short (under `diarization_min_sec`) or could not be identified. Your own microphone is not labelled.
- `diarization_threshold`: lower values avoid splitting one person into several labels but merge different people more often. Real meetings often need about 0.50–0.65.
- Labels are unique per transcript file: when the output file changes, the in-meeting clusters are discarded and numbering continues from the file's highest existing label.

#### Naming speakers and voiceprints

```yaml
# config.yaml (or the dashboard settings modal)
speaker_enrollment: true      # default false (requires speaker_diarization: true)
voiceprint_max_samples: 200   # per person (default 200)
voiceprint_min_sec: 3.0       # shorter utterances are never added to voiceprints (default 3.0)
voiceprint_cluster_min_samples: 10  # people with fewer samples are not split into clusters (default 10)
voiceprint_cluster_distance: 0.6    # average-linkage cosine distance for merging clusters (default 0.6)
voiceprint_main_similarity: 0.5     # a cluster this similar to the main one counts as the usual voice (default 0.5)
voiceprint_outlier_max: 2           # largest cluster that can be excluded automatically (default 2)
voiceprint_outlier_ratio: 0.03      # ... and its largest share of the person's samples (default 0.03)
```

- With `speaker_enrollment: true`, the diarization embeddings are saved per transcript in `transcript-*.voices.jsonl`.
- After the meeting, the 👥 button in the dashboard lets you name labels such as `相手A` (for example `太郎`). Several labels may get the same name. Expected attendees and already registered people are offered as candidates, and you can also type any name.
- Naming rewrites the labels in the transcript and its translations. If `speaker_enrollment` is enabled, that voice is also stored as a voiceprint in `voiceprints.json` (up to `voiceprint_max_samples` per person).
- Your own voice: set `self_name` ("your name" in the settings modal), turn on diarization and enrollment, then press "Record my voice" under registered people and talk into your mic for one minute. Stretches of at least `voiceprint_min_sec` become your baseline. When you join from a meeting room through someone else's mic, your voice arriving on the speaker side is then named too.
  - Once the baseline has 3 samples or more (voiceprints taken by naming or imported count too), a voiceprint is also taken from your mic about once a minute. Your mic also picks up other people (in the room, or when your PC is the room's mic), so a mic utterance is kept only when it scores at least `voiceprint_high` against your baseline and beats everyone else by `voiceprint_margin`. Without a baseline nothing is taken automatically.
  - The latest 30 of each kind are kept, apart from voiceprints taken by naming. In `voiceprints.json`, baseline samples are marked `"origin": "enroll"` and automatic ones `"origin": "mic"`.
- Every assigned name is remembered (with its group, person flag and group color, but no voice data, regardless of `speaker_enrollment`) in `speaker_names.json` and offered as a suggestion in every meeting. A row that already has a name shows it as a placeholder; leave the field empty to keep it. Deleting a person from the dashboard removes their name too.
- A person's **group** (a company or organization) is an attribute of the person, set once in the "Registered people" list at the bottom of the naming modal (everyone in the name book plus everyone with voiceprints; edit the group field there). Untick "Person (enroll voiceprint)" in the naming or change-speaker modal to treat a label as a company or role name (for example `acme`, `先方営業`): only then does a group field appear there, and the label gets a group and a color but never enrolls a voiceprint. Group and person status change only when you edit them; otherwise the stored values are kept.
- Select transcript lines and press 👤 ("change speaker") to reassign them to another speaker (same name, group and person options). Shift+click a checkbox to select every line from the previously clicked one. The modal asks what to change and shows how many lines that is: **the selected lines only** (default), **everything between the two lines** (when exactly two are selected; every remote-side line in that time range, whoever the speaker, plus the `[自分]` lines when one of the two is a `[自分]` line), or **all lines of this speaker** (the labels of the selected lines across the whole meeting, the same as naming them with 👥: recorded in the history and undoable; selected `[相手?]` / `[相手]` / `[自分]` lines, which cannot be named as a label, are reassigned one by one). `[自分]` lines can be reassigned too, for when other people talk into your mic (in the same room, or when your PC is the room's mic); no voiceprint is taken from them, since mic lines keep no embedding. `[Claude]` lines cannot be reassigned.
- Speakers are colored by group in the transcript. Set `self_group` ("your own group" in the settings modal) and `[自分]` takes that group's color; each group gets an automatic, persistent color.
- Minutes (both the LLM method and the AI assistant method) receive a "speakers and groups" block (added only when at least one group or `self_group` exists). With two or more groups they are asked to organize requests, agreements and action items by group. The `clerk-meeting-helper` skill (1.1.1) reads it from `GET /api/speaker-roster`.
- After saving names or groups, if the meeting has minutes the dashboard asks whether to regenerate them. If the meeting has AI-assistant output (advice or analysis files), the assistant rebuilds them (started first if needed, with an LLM fallback); otherwise the LLM does. While a different meeting is in progress the LLM is always used, so the live assistant is not interrupted.
- Already-named speakers stay in the list and can be renamed (for a typo, say); their voiceprints from that meeting move to the new name.
- Utterances shorter than `voiceprint_min_sec` still get the name in the transcript, but are not added to the voiceprint (short replies such as "yes" make unreliable voiceprints). `[相手?]` lines (too short to identify) can be named with 👤 the same way; they never had an embedding. The utterance length is stored as `dur` in `*.voices.jsonl`; older rows without it are treated as long enough.
- **Undoing a naming:** each label naming is recorded in the meeting's `transcript-*.naming.jsonl`, listed newest first under "Naming history" in the naming modal (when, `label → name`, lines and voiceprints). "Undo" puts the original label back on the lines that still carry that name (lines moved to someone else later are left alone), removes the voiceprints that naming added and gives back the voiceprints it took from a previously named person (unless that person was deleted, or the voiceprint was deleted with "Delete" in the clusters view). Voiceprints that naming added but that are already gone (dropped by `voiceprint_max_samples`, deleted) or moved to someone else are left alone; the result says how many. The history is kept only for namings made with this version; it follows the meeting when it is renamed or deleted.
- **Reviewing a person's voiceprints:** "Details" next to a registered person splits their voiceprints into clusters (people with fewer than `voiceprint_cluster_min_samples` samples are shown as one cluster). Each cluster shows its kind, number of samples, the meetings (and original labels when the history knows them) and an example utterance:
  - **Usual voice** — the main cluster. No action.
  - **Different voice** — far from the main cluster: another person mixed in, or the same person on a different line or device. It is still used; you decide. "Delete" removes those samples from the voiceprint store only (the transcript keeps the name; the sidecar rows get `noenroll` so they are not enrolled again). "Move to…" moves them to another person (with name suggestions) and renames the matching transcript lines that still carry the old name; meetings being recorded or translated are not rewritten and are counted in the result.
  - **Excluded (laughter etc.)** — a tiny, distant cluster (at most `voiceprint_outlier_max` samples and `voiceprint_outlier_ratio` of the person's total) is excluded from matching automatically. "Restore" brings it back, and it is never excluded automatically again.
  The exclusion is stored per sample as `status` in `voiceprints.json` and is recomputed whenever samples are added to or removed from that person (naming, undo, delete, move). A restored sample is shown under the usual voice.
- Naming and line reassignment are refused for a file being translated, and when the name book or voiceprint store is unreadable. The file currently being recorded is never rewritten (that would break the live AI-assistant stream's read offsets):
  - **During a meeting**, 👥 and 👤 save the names *provisionally* in `transcript-*.provisional.json`. The transcript, translations and voiceprints are left alone; the dashboard only shows the provisional names (dotted underline). Reopening the modal shows them for editing; clearing a name (👥) or saving 👤 with an empty name cancels them. When the meeting ends — by the button, the voice command, the calendar or a skill — they are applied with the same processing as a normal naming (line reassignments first, then labels), before the automatic minutes. Whatever cannot be applied stays in the file and is logged.
  - **The daily file** cannot be named while it is recorded; wait until the date changes or extract it as a meeting with ⏱ first.
- A file whose translation is running (one-shot or the continuous loop) cannot be named until it finishes; provisional names wait for it (up to 2 minutes) when the meeting ends.
- Merging lines into an existing meeting can mix two different speakers under the same label; name such a file carefully (or extract into a new meeting instead).

#### Identifying speakers during the meeting

```yaml
# config.yaml (or the dashboard settings modal)
voiceprint_high: 0.60                  # score for a confident name (default 0.60)
voiceprint_low: 0.50                   # score for a tentative "name?" (default 0.50)
voiceprint_margin: 0.05                # a confident name must beat the runner-up by this much (default 0.05)
voiceprint_non_attendee_penalty: 0.05  # added to both scores for people not in the meeting's attendees (default 0.05)
voiceprint_min_utterances: 2           # a label needs this many utterances before any name is shown (default 2)
```

- With both `speaker_diarization` and `speaker_enrollment` on, each remote-side label (`相手A`…) is matched against the registered voiceprints while recording (samples excluded as outliers are not used). The mic side is never matched; it stays `[自分]`.
- The result has three levels, and a wrong name is avoided rather than a right one maximized:
  - **Confident** — score ≥ `voiceprint_high` and ahead of the runner-up by `voiceprint_margin`: the line is written as `[alice]`.
  - **Tentative** — score ≥ `voiceprint_low`: `[alice?]`.
  - **None** — the meeting label stays, `[相手A]`.
  People not in the meeting's attendee list (or every candidate, when the meeting has none) need `voiceprint_non_attendee_penalty` more on both scores. No name is shown until the label has `voiceprint_min_utterances` utterances.
- The match is redone each time the label gets a new utterance and applies to the following lines; lines already written are not rewritten. If the voiceprint store cannot be read, no name is shown and recording goes on.
- In the transcript, `[alice?]` takes `alice`'s group color. Minutes and `GET /api/speaker-roster` treat `alice?` as unconfirmed (`tentative: true`, no group); the LLM is told not to state it is that person.
- **Identified names never enter the voiceprints by themselves.** They are added only when you confirm them in the naming modal after the meeting:
  - An `alice?` label is listed as "Unconfirmed", with `alice` shown greyed out in the name field and an unticked "Confirm as alice" box. The row is saved only when you tick the box or type a name; saving other rows leaves it alone. Then it is named like any other label (`alice?` → `alice`), its voiceprints are added and it is recorded in the history, so it can be undone.
  - A confident `[alice]` shares the label of `alice` named by hand. When some of its lines were identified during the meeting, the row shows "Confirm N identified lines and add to voiceprints". Tick it and save to add those lines to `alice`'s voiceprints. The transcript is not changed; the history entry has no lines and only the added voiceprints, so undo removes just those and marks the lines as identified-but-unconfirmed again. Saving only the group or person flag of such a label does not add the identified lines.
  - Giving the label another name works as usual: all its lines, identified ones included, move to that name and its voiceprints. Undoing it marks the identified lines that went back to the label as unconfirmed again.
  - Names cannot end in `?`, so a real name is never mistaken for a tentative one.
- In `*.voices.jsonl`, `label` is the label shown in the transcript (`alice`, `alice?` or `相手A`), `cluster` is the meeting label it was grouped under, and `auto: true` marks a name that came from identification and has not been confirmed yet. Naming or confirming clears `auto`; undoing that naming sets it again. After a restart on the same file, numbering also counts `cluster`, so a `相手A` shown as `alice` is not reused for someone else. Older rows without these fields work as before.

**Sharing voiceprints:** someone who uses diarization a lot can hand their voiceprints to someone who has few. In "Registered people", check people (or pick a group with "Select by group…") and press "Export" to save one JSON file; the receiver presses "Import", chooses the file, and for each person checks whether to import and edits the name to import as (an existing name adds to that person, a new name creates one). The file holds the voiceprints used for matching (excluded samples are left out), each person's group and color, and the embedding model; it holds no meeting names. Imported samples are marked `"origin": "import"` / `"source": "import"`, importing the same file twice adds nothing, and a file made with a different model is refused. People new to the receiver get the file's group and color; existing people keep theirs. The same is available as `clerk-util voiceprints-export` / `voiceprints-import`. **Get each person's consent before passing their voiceprints on.**

**Privacy:** voiceprints are biometric data. They stay inside the data directory unless you export them, and are never passed to an LLM. Audio is not stored. Deleting a person from the dashboard removes their stored voiceprints and every embedding named after them in the per-meeting `*.voices.jsonl` files; deleting `voiceprints.json` and all `*.voices.jsonl` files erases everything. `*.naming.jsonl` holds only labels, names and timestamps, no voice data.

### Optional: Google Calendar integration

Automatically starts and ends meeting sessions based on your Google Calendar schedule. Requires the `gcal` extra:

```bash
uv tool install "shadow-clerk[gcal]"
# or for development:
uv sync --extra gcal
```

Then authenticate and configure:

```bash
# One-time OAuth setup (opens browser)
clerk-util gcal-auth ~/credentials.json

# Enable in config (gcal-auth already does this when it succeeds)
clerk-util write-config-value gcal_integration true
clerk-util write-config-value gcal_credentials_file ~/credentials.json
```

When enabled, clerk-daemon polls Google Calendar every 60 seconds. Events automatically trigger `start_meeting` / `end_meeting`, creating transcript files named `transcript-YYYYMMDDHHMM@EventTitle.txt`. The event's invitees are saved as the meeting's expected attendees and shown above its summary, and **More ▾ → 📅 Today's schedule** in the dashboard lists today's events.

See [docs/google-calendar-setup.md](docs/google-calendar-setup.md) for full setup instructions including how to obtain `credentials.json` from Google Cloud Console.

Add the following options if you need translation or summarization.

### Optional: Pronunciation feedback

Phoneme-level pronunciation feedback for `clerk-practice` (English only).

```bash
uv sync --extra pronounce
```

Downloads a 1.2GB phoneme recognition model on first use. `espeak-ng` ships as a
wheel (`espeakng-loader`), so no system package is needed. Note that
`phonemizer-fork` installs under the `phonemizer` import name and conflicts with
the original `phonemizer` package.

Enable it in Settings → Pronunciation feedback.

### 3. (Optional) LibreTranslate setup

Local translation without LLM. Install via Docker or pip:

```bash
# Docker (recommended)
docker run -d -p 5000:5000 libretranslate/libretranslate

# Or pip
pip install libretranslate
libretranslate --host 0.0.0.0 --port 5000
```

Configuration:

```yaml
# config.yaml
translation_provider: libretranslate
libretranslate_endpoint: http://localhost:5000
```

### 4. (Optional) OpenAI compatible API setup

Used for translation, summarization, and LLM voice command matching:

```yaml
# config.yaml — OpenAI
llm_provider: api
api_endpoint: https://api.openai.com/v1
api_model: gpt-4o
# Add SHADOW_CLERK_API_KEY=sk-... to ~/.local/share/shadow-clerk/.env
```

```yaml
# config.yaml — Ollama (local)
llm_provider: api
api_endpoint: http://localhost:11434/v1
api_model: llama3
```

### 5. (Optional) Use Claude CLI as the LLM provider

If you have Claude Code installed (`claude` on your `$PATH`), shadow-clerk can shell out to `claude -p` for translation and summarization. Set in `config.yaml`:

```yaml
llm_provider: claude
claude_cli_model: haiku   # or sonnet / opus / a full model id
# translation_claude_model: haiku  # translation only; unset = claude_cli_model
# claude_cli_path: claude  # full path if not on $PATH
```

This uses your existing Claude Code OAuth login. No extra setup needed — translation and summarization run inside the daemon as background threads, no Claude Code session required.

## Docker

Docker を使うと、Python や PortAudio のセットアップなしに shadow-clerk を動かせます。Linux ホストで PulseAudio / PipeWire が動いていることが前提です。

### shadow-clerk 単体

```bash
# イメージをビルドして起動（フォアグラウンド）
docker build -t shadow-clerk .
docker run --rm \
  -p 8765:8765 \
  -v "$HOME/.local/share/shadow-clerk:/data" \
  -v /run/user/$(id -u)/pulse:/run/pulse \
  -e SHADOW_CLERK_DATA_DIR=/data \
  -e PULSE_SERVER=unix:/run/pulse/native \
  shadow-clerk
```

ダッシュボードは <http://localhost:8765> で開けます。

### VOICEVOX 込みで docker-compose

`docker-compose.yml` は shadow-clerk と [VOICEVOX Engine](https://github.com/VOICEVOX/voicevox_engine) をまとめて起動します。

```bash
# 起動（初回はビルドに数分かかります）
docker compose up
```

起動後、ダッシュボードの **Settings → talk_voicevox_url** を `http://voicevox:50021` に変更するか、`data/config.yaml` に次の行を追加してください：

```yaml
talk_voicevox_url: http://voicevox:50021
```

ローカルインストール時と同じ `~/.local/share/shadow-clerk/` をマウントするため、設定・文字起こし・翻訳・要約を共有できます。

#### PulseAudio ソケットのパスについて

`docker-compose.yml` は `${UID}` を参照します（未設定時は `1000`）。シェルが `UID` をエクスポートしていない場合は次のように指定してください：

```bash
UID=$(id -u) docker compose up
```

または `.env` ファイルを作成しておくと毎回不要です：

```
UID=1001   # id -u の値に合わせる
```

#### PipeWire を使っている場合

PipeWire の PulseAudio 互換ソケットを使います。パスを確認して `docker-compose.yml` の `volumes` と `PULSE_SERVER` を置き換えてください：

```bash
ls /run/user/$(id -u)/pulse/native   # このファイルがあれば PipeWire が PulseAudio 互換ソケットを提供中
```

ソケットが存在すれば、上記の PulseAudio 設定がそのまま動きます。

#### GPU（CUDA）でより速い文字起こし

`docker-compose.yml` の `shadow-clerk` サービスに次を追加し、`clerk-daemon` のオプションに `--device cuda` を加えてください：

```yaml
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]
    command: ["--dashboard-host", "0.0.0.0", "--device", "cuda"]
```

## Usage

### Starting the daemon

If you installed via `uv tool install`:

```bash
clerk-daemon
```

For development (`uv sync`):

```bash
uv run clerk-daemon
```

> **Note:** `uv run` uses the project `.venv`, while `uv tool install` uses its own isolated environment. Make sure extras (e.g. `spell-check`, `reazonspeech`) are installed in the matching environment.

The dashboard is served at <http://localhost:8765>.

### Recording & transcription

```bash
# Basic (record mic + system audio, auto-transcribe)
clerk-daemon

# List available devices
clerk-daemon --list-devices

# With options
clerk-daemon \
  --language ja \
  --model small \
  --output ~/my-transcript.txt \
  --verbose

# In the background (logs go to daemon.log in the data directory)
clerk-daemon -d
```

Press `Ctrl+C` to stop recording (or `clerk-util stop` for a background daemon).

### Audio device selection

`mic_device` / `monitor_device` pin capture to a device by **name**, not device number — device numbers shift between daemon runs and can even change while the daemon is running. `null` (the default) follows the OS default device. Pick a device from the dashboard settings panel.

If the configured device disappears (unplugged, sink removed), the daemon falls back to the automatic device and keeps recording, then switches back on its own once the device reappears. The configured value is never overwritten by the fallback. If the device never actually disappeared — another application just grabbed it exclusively — the daemon won't retry on its own; use "Refresh list" to force a retry once it's free.

The `--mic` / `--monitor` CLI flags (device numbers, see [CLI options](#cli-options)) take priority over `mic_device` / `monitor_device`. While a flag is in effect, the corresponding dropdown is disabled on the dashboard.

A device connected after the daemon started won't appear in the dropdown until the list is refreshed ("Refresh list" in the settings panel); refreshing briefly interrupts both capture streams.

### Audio level meters

The dashboard header shows a level bar for each capture path (mic/speaker), next to the mute buttons. The bars use the crest factor — peak level divided by RMS — to tell real speech from noise: speech runs 3–10 or higher, while steady electrical noise (a dead built-in mic emitting only hum, say) sits at 1–2.

A bar highlights yellow after 10 seconds of loud-but-flat audio (steady noise, no speech dynamics) — this applies to both mic and monitor. The mic bar additionally turns red after 30 seconds of exact silence: a live microphone always has some noise floor, so true zero means nothing is reaching it at all (e.g. a powered-off headset whose dongle still shows up as a device). The monitor skips this check — a sink monitor reads back exactly zero whenever nothing is playing, which is its normal idle state, not a fault. Both bars also get a fallback outline when the configured device for that path is unavailable and the automatic one is in use, matching the fallback described above.

### Dashboard

The dashboard (<http://localhost:8765>, `--dashboard-port` to change) shows the transcript and translation live and is where most features are driven from. It listens on `127.0.0.1` only; pass `--dashboard-host 0.0.0.0` to reach it from another machine. Write APIs stay restricted to same-origin requests from localhost either way.

| Area | What it holds |
|---|---|
| Header | Status ("Transcribing (today's notes)", "In a meeting", "Disconnected"; click it to open the live transcript while another file is open), **▶ Start meeting** (meeting name, language and the AI assistant in one dialog), **🌐 Translate**, **Talk with Claude**, **🗣** the recognition language (`auto` / `ja` / `en` …, switches immediately), **Your voice** / **Their voice** mute buttons with [level meters](#audio-level-meters) (muting does not stop the recording), **More ▾** (glossary, 📅 today's schedule when the calendar integration is on, PTT, voice commands, spoken language, per-meeting settings, ⚙ settings, ❓ help, and a link to the previous dashboard; PTT, voice commands, spoken language and the link appear in **Advanced** only), **🪟 View ▾** (what to show in the centre, the side panels and text size), **Simple** / **Advanced**, and the light/dark theme switch |
| Left pane: **Dates** / **Meetings** / **🔍 Search** | Daily transcripts by date; meeting files grouped by name (sort by name or newest, rename a meeting group); search by keyword, target and year/month/day/hour across transcripts, translations and summaries, with the hits grouped by meeting |
| Centre | The open transcript. **🪟 View** switches it between Transcript, + Translation, Translation only, Minutes & AI advice (large) and the AI assistant screen. A daily transcript is shown in conversation chunks. Click (Shift+click for a range) to select lines, then **✂ Save as a meeting** (as one meeting, or split at pauses in the conversation; a whole daily transcript can be split too), **👤 Fix the speaker**, **📋 Copy** or **🗑 Delete**. **👥 Name the speakers** names the labels, **📂 Meeting name** assigns the file to an existing or new meeting, 🗑 deletes the file or moves a meeting back into the daily transcript |
| Right pane: **📝 Minutes** / **💡 AI advice** | The minutes (copy, reload, regenerate; expected attendees from the calendar on top). **AI advice** shows the [AI assistant](#ai-console)'s advice and analysis and starts, stops or switches it. Only one assistant runs at a time |
| Bottom drawer (**Advanced** only) | The two [AI Console](#ai-console) terminals (meeting assistant and [talk](#talk-with-claude)), the activity log and voice-command responses |

**Simple** hides the tools most people don't need (PTT, voice commands, spoken language, file deletion and the bottom drawer); **Advanced** shows everything. Both the mode and the theme are remembered by the browser.

The previous dashboard is still served at <http://localhost:8765/legacy> (**More ▾ → Previous dashboard** in **Advanced**) while the new one settles in; it will be removed later. `/next`, where the new dashboard was tried out, redirects to `/`. One thing only the previous dashboard can do: launch the AI assistant without a prompt (to `/resume` an earlier session).

A **Welcome** dialog appears on first run: it offers to install the skill for an AI agent and points to the settings worth checking first ("Do not show this again" hides it). When the bundled skill is newer than the installed copy, a dialog offers to update it.

### Voice commands

#### Push-to-Talk (recommended)

Hold down the push-to-talk key while speaking a command — no wake word needed:

```
[Hold PTT key] "start translation" → Translation starts
[Hold PTT key] "start meeting"     → Meeting session starts
```

The key is `voice_command_key` in `config.yaml`. The default `f23` suits a keyboard remapper (e.g. xremap mapping the Menu key to F23); other values are `menu` (the Menu key next to Right Alt), `ctrl_r`, `ctrl_l`, `alt_r`, `alt_l`, `shift_r`, `shift_l`. Set to `null` to disable. On Windows, see [Windows-specific notes](#windows-specific-notes).

#### Prefix mode (fallback)

During recording, say the wake word (default: "sheruku" / "シェルク") followed by a command for hands-free control:

| Voice command | Action |
|---|---|
| "sheruku, start meeting" | Start a new meeting session |
| "sheruku, end meeting" | End the meeting session |
| "sheruku, language ja" | Switch transcription language to Japanese |
| "sheruku, language en" | Switch transcription language to English |
| "sheruku, unset language" | Reset to auto-detect |
| "sheruku, start translation" | Start the translation loop |
| "sheruku, stop translation" | Stop the translation loop |

The separator (comma, space) between the wake word and command is optional. The wake word can be changed via `wake_word` in `config.yaml`.

#### Custom voice commands

You can register custom voice commands in `config.yaml` under `custom_commands` (or with **More ▾ → Voice commands** on the dashboard, shown in **Advanced** only). They are evaluated after built-in commands:

```yaml
custom_commands:
  - pattern: "youtube"
    action: "xdg-open https://www.youtube.com"
  - pattern: "gmail|mail"
    action: "xdg-open https://mail.google.com"
```

- `pattern`: Regular expression (case-insensitive)
- `action`: Shell command to execute

Start the meeting analysis by voice. This pattern fires when you say it while
holding the PTT key, with or without a leading `クラーク`:

```yaml
custom_commands:
  - pattern: (クラーク|クラーク、)?分析(開始|して)
    action: curl -sX POST localhost:8765/api/console/start
```

#### LLM fallback

If a voice command doesn't match any built-in or custom command and an LLM is configured (`api_endpoint`, or `llm_provider: claude`), the utterance is sent to the LLM as a query. The response is printed to stdout and saved to `.clerk_response`.

```
"sheruku, what is 1+1?" → LLM returns the answer
```

### CLI options

`clerk-daemon` options:

| Option | Description | Default |
|---|---|---|
| `--output`, `-o` | Output file path | `~/.local/share/shadow-clerk/transcript-YYYYMMDD.txt` |
| `--model`, `-m` | Whisper model size (`tiny`, `base`, `small`, `medium`, `large-v3`) | `small` |
| `--language`, `-l` | Language code (`ja`, `en`, etc.). Auto-detect if omitted | Auto |
| `--mic` | Microphone device number | Auto-detect (or `mic_device` config) |
| `--monitor` | Monitor device number (sounddevice) | Auto-detect (or `monitor_device` config) |
| `--backend` | Audio backend (`auto`, `pipewire`, `pulseaudio`, `sounddevice`, `wasapi`) | `auto` |
| `--list-devices` | List devices and exit | - |
| `--verbose`, `-v` | Verbose logging | - |
| `--dashboard` / `--no-dashboard` | Enable/disable dashboard | Enabled |
| `--dashboard-port` | Dashboard port number (falls back to `dashboard_port` in `config.yaml`, which `clerk-util` also reads) | `8765` |
| `--dashboard-host` | Dashboard bind address. Only loopback by default; pass `0.0.0.0` to expose it on the LAN (write APIs still accept localhost only) | `127.0.0.1` |
| `--beam-size` | Whisper beam size (`1`=fast, `5`=accurate) | `5` |
| `--compute-type` | Whisper compute precision (`int8`, `float16`, `float32`) | `int8` |
| `--device` | Whisper device (`cpu`, `cuda`) | `cpu` |
| `--daemon`, `-d` | Run in the background; logs go to `daemon.log` in the data directory | - |

### clerk-util subcommands

`clerk-util` manages the daemon and the data directory. Run `clerk-util help` for the same list.

| Subcommand | Description |
|---|---|
| `start [opts]` | Run clerk-daemon with these options, in the foreground (add `-d` to run it in the background) |
| `stop` | Stop clerk-daemon (SIGTERM on Linux, `taskkill` on Windows) |
| `restart [opts]` | Stop clerk-daemon, wait for it to exit, then start it with these options |
| `recorder-status` | Print `running` or `stopped` |
| `gui [opts]` | Open a small window showing the daemon's state and the tail of `daemon.log`, with start / stop / restart buttons. `clerk-util --gui` works too |
| `command <cmd>` | Send a command to the running daemon: `start_meeting`, `end_meeting`, `translate_start`, `translate_stop`, ... |
| `summarize [DATE\|FILE] [--mode full\|update]` | Generate minutes (`full` by default). DATE is `YYYYMMDD` or `YYYYMMDDHHMM[@name]`, FILE a `transcript-*.txt`; without either, the current meeting or today |
| `ls` | List the data directory (and the output directory if it differs) |
| `read-config` | Print `config.yaml` (writes one with the defaults if missing) |
| `write-config-value <key> <value>` | Change one key in `config.yaml` |
| `gcal-auth <credentials.json> [token_file]` | Google Calendar OAuth; on success it also enables `gcal_integration` |
| `voiceprints-export <out.json> [name ...]` | Export voiceprints to pass to someone else (everyone with voiceprints when no name is given) |
| `voiceprints-import <file.json> [name[=to] ...]` | Import exported voiceprints, optionally only some people or under another name |
| `install-skill [--target claude\|agents\|<path>] [--link] [--force]` | Install the bundled skills for an AI agent (see [AI Console](#ai-console)) |
| `run-llm <args...>` | Run the LLM client directly (`translate`, `query`, `match-command`, `summarize`, `spell-check`) |
| `help` | Show usage |

### Translation & Summary Providers

Translation and summary each support multiple providers with different operation modes:

#### Claude mode (`translation_provider: claude` / `llm_provider: claude`)

clerk-daemon shells out to `claude -p` per request, reusing your existing Claude Code OAuth login.

- **Highest quality** — especially for Japanese homophone correction (ja→ja)
- **Requires `claude` on PATH** — found automatically if Claude Code is installed
- **No Claude Code session required** — the daemon spawns `claude -p` per job, so you don't need to keep a Claude Code terminal open
- **Translation and summary both run inside daemon threads** — same plumbing as api / libretranslate
- **Cost tracking**: `claude -p --output-format json` returns `total_cost_usd`, which is logged by the daemon

```yaml
# config.yaml
translation_provider: claude   # Translation by Claude
llm_provider: claude           # Summary by Claude (default)
claude_cli_path: claude        # full path if not on PATH
claude_cli_model: haiku        # haiku / sonnet / opus or a full model id
translation_claude_model: null # Model for translation only (null = claude_cli_model)
```

#### API mode (`translation_provider: api` / `llm_provider: api`)

clerk-daemon calls an external API (OpenAI-compatible) internally. Claude Code is not required.

- **Works without Claude Code** — clerk-daemon handles translation and summary on its own
- **Quality depends on model** — high-end models (GPT-4o) produce good results; smaller models may struggle with Japanese correction
- **How translation works**: An internal thread in clerk-daemon processes translation. Started/stopped via voice commands or dashboard
- **Summary works similarly**: `clerk-util summarize` generates minutes via the external API

```yaml
# config.yaml
translation_provider: api     # Translation via external API
llm_provider: api             # Summary via external API
api_endpoint: https://api.openai.com/v1
api_model: gpt-4o
```

#### LibreTranslate mode (`translation_provider: libretranslate`)

Translation only. Runs locally without any external API or Claude Code (summary still needs `llm_provider`).

#### Recommended configurations

| Use case | Translation | Summary | Notes |
|---|---|---|---|
| Best quality (Claude CLI) | `translation_provider: claude` | `llm_provider: claude` | Highest quality, needs `claude` CLI |
| Autonomous (external API) | `translation_provider: api` | `llm_provider: api` | OpenAI-compatible, quality varies by model |
| Fully local | `translation_provider: libretranslate` | — | No LLM needed, lower quality |
| Hybrid | `translation_provider: api` | `llm_provider: claude` | Auto translation + high-quality summary |

### Talk with Claude

Talk mode lets you discuss a topic with Claude by voice. Claude asks questions and replies
through text-to-speech; your answers are transcribed as usual. Both sides are written to the
transcript (`[Claude]` lines are Claude's), so you can read the discussion back later.

Requirements:

- [Claude Code](https://claude.com/claude-code) CLI (`claude_cli_path`)
- A running [VOICEVOX](https://voicevox.hiroshiba.jp/) engine (default `http://localhost:50021`).
  The engine runs as a separate process and is not bundled. To start it, either:
  - Download and launch the **VOICEVOX app** from the official site (GUI + engine).
  - Or download the **engine-only** package (no GUI) from the [GitHub releases](https://github.com/VOICEVOX/voicevox_engine/releases) and run `./run` (Linux) / `run.exe` (Windows).
  - Or run via Docker: `docker run --rm -p 50021:50021 voicevox/voicevox_engine:cpu-ubuntu20.04-latest`

  Follow the terms of the voice you use; the dashboard shows the required credit (`VOICEVOX:<name>`) while talk mode is on.
  When talk mode starts, words VOICEVOX is known to misread (bundled `voicevox_dict.tsv`) are added to the engine's user dictionary, unless the same word is already registered there.
- Headphones. While talk mode is on, the monitor channel is not transcribed, so Claude's own
  voice does not come back as `[Others]` lines.

Click **Talk with Claude** in the dashboard header, enter a topic (or leave it empty), pick a
persona, and start. The **mode** select fills these in from a saved mode: besides **Free talk**, it ships with **英語の練習** (English practice), **Japanese Lesson** and **会議の振り返り** (meeting retrospective), which can be deleted like your own modes. **Skill** makes Claude switch to that skill first (`clerk-practice`, `clerk-conversation-retrospective` or `voicevox-sing`), with the topic as what you want from it. **Voice settings** in the same dialog picks the VOICEVOX speaker and adjusts speed, pitch, intonation and volume, with a preview button; changes apply from the next sentence. Unticking **Use the glossary** turns off the glossary replacements in the transcript for that conversation, and Claude does not read the glossary either (for conversations that do not need it, such as language practice; saved with presets). **Reply early** (on by default) lets Claude answer from the in-progress transcription before the final line arrives, which is faster but may answer a misheard line; untick it to have Claude wait for final lines. By default Claude runs in a second AI Console (the **Talk with Claude screen** in the bottom drawer) with the `clerk-talk` skill,
so it can edit files and run commands with the usual permission prompts, and the meeting helper keeps
running in the **AI assistant screen**. Set `talk_engine: headless` to use a background `claude -p` process
instead: it answers a little faster but cannot ask for permission, so it only gets the tools in
`talk_allowed_tools`. Install the skills with `clerk-util install-skill` (all of them are installed together).
The `clerk-talk` skill pre-approves only its own `curl` calls to `http://localhost` (and `Monitor` and `Agent`); if
Claude Code still asks, allow them once, or add the same rules to your Claude Code settings.
Screen captures taken with the [browser extension](#browser-screenshots) during talk mode are looked at by a background subagent, so Claude can follow "this box here".

**Sending Claude's voice to a meeting (Linux, PipeWire).** In the start dialog, pick the meeting app under
**Send Claude's voice to**. shadow-clerk plays the speech through a named PipeWire stream and links it into that
app's microphone input with `pw-link`, so the other participants hear Claude mixed with your microphone, and you
still hear it in your headset. If the app reopens its microphone, the link is restored automatically. Muting
yourself in the meeting app mutes Claude too, because the voice goes into the same input. Needs `pw-dump`,
`pw-link` and `pw-cat`; on other platforms the option is disabled. With a route set, speech plays to PipeWire's
default output and `talk_output_devices` is not used.

While a meeting app is chosen, the other participants are transcribed as `[Others]` again, so Claude can follow
them. Claude's own voice also reaches your headset and therefore the monitor channel; monitor lines that overlap
Claude's speech in time (or start within `talk_echo_tail_sec` after it) are dropped, whatever they say, because short
replies and fillers can't be told apart from Claude's voice by text. If someone talks over Claude, their words in
that overlap are dropped too. Without a meeting app chosen, the monitor stays muted during talk mode as before.
Interim (live) monitor text is not shown while Claude is speaking.

With a meeting app chosen, Claude also waits for the floor: while the interim (live) transcription of the monitor
has text, someone else is talking, so a new reply waits until they finish (at most `talk_floor_wait_sec`) before it is
spoken. This is much earlier than waiting for the finished transcript line. If anyone's line is transcribed while it
waits, the reply is not spoken, because the topic may have moved on: `/api/say` answers
`{"status": "held", "heard": [...lines...]}` and Claude rethinks it (the headless engine drops the rest of that turn and
gets the lines with a note). Before speaking in a meeting, Claude says a short "Um," and pauses for half a second; if
someone else is heard in that pause, both started at once, so Claude says "Sorry, go ahead" and waits for them. The
talk skill also passes the time of the newest line it has read (`after`), so a reply written before a newer line
arrived is held instead of spoken. Interim transcription runs for this even
when `interim_transcription` is off (it is then not shown on the dashboard). `GET /api/speaking` returns the flag
(`{"speaking": true, "sources": ["monitor"]}`); the talk skill also uses it before raising a point on its own.
It also returns the latest interim text of each source still speaking (`"interim": {"monitor": "..."}`), so Claude
can start drafting a reply before the line is finalized. For this, the microphone also gets interim transcription during
talk mode, with or without a meeting app (it is never shown on the dashboard).

Your own voice counts too: while the microphone's voice detection hears you speaking (muted mic and push-to-talk
excluded), a reply waits for you to finish, even without a meeting app chosen. Use earphones, since Claude's own
voice leaking into the microphone would count as you speaking.

You can also make Claude speak any text:

```bash
curl -s -X POST localhost:8765/api/say -d '{"text":"Hello"}'
```

Claude never stops listening on its own: when the conversation seems to be over, it asks first, and only on a
clear yes does it end talk mode with `POST /api/talk-end`. That call lets the sentence being spoken finish first.

If you want to speak another language (for example to practise English while the recognition language is `ja`,
which would turn English into katakana), just tell Claude: it switches the recognition language with
`POST /api/language` (`{"language": "en"}`, or `"auto"`) and switches back before the conversation ends.

**Language practice.** In talk mode, say that you want to practise a language ("I want to practise English").
Claude switches to the bundled `clerk-practice` skill: it starts a practice meeting (`英語練習` for English) without
the meeting assistant, reads the end of the last few practice sessions to suggest what to do today, asks how many minutes to
practice (and, a minute before time is up, whether to stop or keep going), and runs
pronunciation, conversation, listening or immersion practice (practice language only; corrections are written to the 💡 AI advice tab instead of spoken). Corrections go to **Advice** and the practice log with example
sentences to **Analysis** in the 💡 AI advice tab; lines starting with 🔊 are read aloud when you click them, in the
language tagged after it (`🔊[en] …`); `🔊[en|sentence] Question 1` shows only "Question 1" and reads the hidden sentence (listening practice). Claude speaks practice-language sentences with the browser's voice (Web Speech API)
instead of VOICEVOX, so keep the dashboard open and click it once (browsers only allow speech after a user action);
without such a tab those sentences fall back to VOICEVOX. The speaker (monitor) is muted while practising, and
Claude ends with a "今日の練習のまとめ: …" line that the next session builds on.

The skill uses these localhost-only endpoints, which you can also call yourself:

| Endpoint | Body / query | Effect |
|---|---|---|
| `POST /api/meeting` | `{"action": "start", "name": "…", "analyze": false}` or `{"action": "end"}` | Start or end a meeting; `analyze: false` skips the AI assistant even with `auto_analyze`; an empty name takes the ongoing calendar event's name, and a name with no usable characters (e.g. `///`) is an error |
| `POST /api/mute` | `{"source": "mic" or "monitor", "muted": true}` | Same as the mute buttons; returns `previous` |
| `POST /api/generated` | `{"kind": "advice" or "analysis", "mode": "replace" or "append", "text": "…"}` | Write the current transcript's advice/analysis (up to 20,000 characters) |
| `GET /api/meeting-history` | `?meeting=…&count=3&tail=15` | `tail` (0–50) adds the last lines of each past transcript |
| `POST /api/say` | `{"text": "…", "lang": "en", "display": "…"}` | A `lang` other than the VOICEVOX language is read by the dashboard's browser voice (ignored when Claude's voice is sent to a meeting app). `display`, if given, is written to the transcript instead of `text` (which is what is spoken), e.g. to hide the spelling in pronunciation drills |

`GET /api/watch` without `file` follows the current transcript: when the day changes or a meeting starts or ends it
sends `<notice>…</notice>` and continues with the new file, so Claude keeps hearing you past midnight.
`interim=1` also streams the not-yet-final text as `<interim>` lines (`interim_min=N` skips texts shorter than N characters),
and `exclude=claude` drops Claude's own `[Claude]` lines, sending nothing when a chunk had only those.

| Key | Default | Description |
|---|---|---|
| `talk_voicevox_url` | `http://localhost:50021` | VOICEVOX engine |
| `talk_speaker_id` | `3` | VOICEVOX style ID |
| `talk_speed` / `talk_pitch` / `talk_intonation` / `talk_volume` | `1.0` / `0.0` / `1.0` / `1.0` | Speed (0.5–2.0), pitch (-0.15–0.15), intonation (0–2), volume (0–2) |
| `talk_output_devices` | `[]` | Output device names (empty = default output). Not used when a route is set; pw-cat plays to the default output |
| `talk_engine` | `console` | `console` (AI Console + clerk-talk skill) or `headless` (`claude -p`) |
| `talk_route_app` | `""` | Last app chosen under "Send Claude's voice to" (start-dialog default) |
| `talk_echo_tail_sec` | `0.3` | A monitor line that starts within this many seconds after Claude stops speaking is treated as Claude's own voice (covers output and capture latency). Lines that start later are kept. If the other side starts talking right after Claude with no silence gap, the VAD merges the two into one segment and it is dropped |
| `talk_workdir` | `""` | Working directory for the talk session (empty = `ai_assistant_workdir`, then home). The start dialog can override it per session |
| `talk_model` | `""` | Model for the talk session (empty = claude default); applies to both engines, picked in the start dialog |
| `talk_allowed_tools` | `WebSearch,WebFetch,Read,Grep,Glob` | Tools Claude may use while talking (`""` = none) (headless only) |
| `talk_language` | `""` | Conversation language (empty = `translate_language`). Falls back to the TTS default when unsupported (VOICEVOX: Japanese only) |
| `talk_personas` | `{}` | Name → personality / how to respond. Edit from the start dialog |
| `talk_default_persona` | `""` | Persona selected by default |
| `talk_filler_sec` | `8` | If Claude has said nothing for this many seconds after you spoke, say a short "hmm" (not written to the transcript). At most once while waiting for a reply, and at least 30 s apart. `0` disables it. Also in Settings |
| `talk_floor_wait_sec` | `10` | With a meeting app chosen, how long a reply waits for the others to stop talking (interim text present) before it is spoken anyway. `0` disables waiting |
| `talk_stop_words` | `["待って", "ストップ", "止めて", "やめて", "stop", "wait", "hold on"]` | Saying one of these stops Claude mid-sentence. Substring match, so avoid short kana that appear inside other words; ASCII words match whole words |

Claude speaks each part of a reply as soon as it is written, in short sentences, so you can cut in
between them. Say a stop word ("wait", "ちょっと待って") and playback stops, the rest of that turn
is dropped, and Claude is told where it was cut off.

### Meeting minutes

Three ways to generate minutes: automatically at meeting end, on demand from the dashboard, or via `clerk-util` from the command line:

```
clerk-util start -d                                # Start daemon (background)
clerk-util stop                                    # Stop daemon
clerk-util recorder-status                         # Show running state
clerk-util gui                                     # Open the status window (start/stop/restart + log)
clerk-util summarize                               # Generate minutes for the current meeting (or today)
clerk-util summarize --mode update                 # Update minutes from the transcript diff
clerk-util summarize 20260425 --mode full          # Specify date
clerk-util command start_meeting                   # Start meeting session
clerk-util command end_meeting                     # End meeting session (auto_summary linked)
clerk-util command translate_start                 # Start translation loop
clerk-util command translate_stop                  # Stop translation loop
```

Meeting start/end is also available via **voice commands** ("sheruku, start meeting" / "sheruku, end meeting") or **dashboard buttons**. The dashboard's **📝 Minutes** tab can generate (or regenerate) the minutes at any time.

When `auto_summary` is on and the [AI Console](#ai-console) is running at meeting end, the minutes are written by the assistant in the console instead (`auto_summary_via_console`, on by default); otherwise the configured LLM writes them.

Generated meeting minutes are saved to `~/.local/share/shadow-clerk/summary-YYYYMMDD.md` (`summary-YYYYMMDDHHMM@Title.md` for a meeting). Put a `summary_template.md` in the data directory to use your own minutes format.

### AI Console

An AI assistant (`claude` or `codex`) can run inside a PTY on the dashboard's **AI assistant screen** (in the bottom drawer, or in the centre via **🪟 View ▾**), watching the meeting transcript and writing generated documents back for the dashboard to display.

- **Installing the skill**: the assistant runs a bundled skill (`clerk-meeting-helper`; `clerk-talk`, `clerk-practice` and `voicevox-sing` for [talk mode](#talk-with-claude) are installed with it) that has to be copied into the agent's own skills directory. The Welcome dialog offers this on a first run, and the command does the same:

  ```bash
  clerk-util install-skill                        # ~/.claude/skills/   (Claude Code)
  clerk-util install-skill --target agents        # ~/.agents/skills/   (Codex and others)
  clerk-util install-skill --target /path/to/dir  # anywhere else; remembered for updates
  clerk-util install-skill --link                 # symlink instead of copy (POSIX only)
  ```

  A destination holding a skill shadow-clerk did not write is left alone unless you pass `--force`.
- **Communication retrospective**: the bundled `clerk-conversation-retrospective` skill reviews your own statements and the surrounding replies after a meeting. Run `/clerk-conversation-retrospective <transcript-path>` in the AI Console, optionally adding a focus such as “answer questions more directly”. It returns evidence with timestamps or line numbers, a successful exchange, a possible improvement with alternative wording, and one action to try next time. Results stay in the conversation unless you request a saved report; they do not automatically appear in the AI Analysis tab.
- **Starting**: automatically when a meeting starts (`auto_analyze: true`), or manually with **Ask the AI assistant to follow** in the **💡 AI advice** tab. Either way, `ai_assistant_init_prompt` (default `/clerk-meeting-helper {transcript} {lang}`) is sent to the PTY once the assistant's TUI is ready — `{transcript}` and `{meeting}` are substituted with the active file paths, and `{lang}` with `translate_language`, so the skill writes in the language you read.
- **Reaching the dashboard**: the assistant calls the dashboard's HTTP API at `$SHADOW_CLERK_URL`, which the console puts in its environment, so it never has to guess the port.
- **Generated documents**: shown in the right pane's **💡 AI advice** tab:
  - `advice-<stem>.md` — open questions and suggestions (overwritten each time)
  - `analysis-<stem>.md` — confirmed facts (appended)

  Both are Markdown and are rendered to HTML server-side. Raw HTML inside them
  is escaped, never rendered.
- **Minutes**: with `auto_summary: true` and `auto_summary_via_console: true` (the default), the minutes at meeting end are written by the assistant while it is running.
- **Session lifecycle**: the PTY session is reused while it stays on the same meeting folder; it is **not** stopped when the meeting ends, so it stays available for writing up minutes afterward. Starting it for a different meeting restarts it in that meeting's folder. Stop it from the Console tab's stop button.
- **Working directory**: the assistant starts in the meeting's folder, so it sees that meeting's past sessions and nothing else by default. Folders it may also read are passed with `--add-dir` (Claude Code only): `ai_reference_dirs` (every meeting), `reference_dirs` per meeting in `DATA_DIR/meeting.yaml`, and the working directory setting — `ai_assistant_workdir`, or `meetings[].workdir` per meeting (for example a Git repository). Relative paths start at the data directory (`sharable`, `private/1on1`). If that working directory has a `.claude/settings.json`, it is passed with `--settings` so its permissions still apply. The skill is told to read nothing outside these folders. This keeps records from mixing by accident; it does not enforce it (a shell command can still read anything).
- **Meeting settings**: **⚙ Per-meeting settings** at the top of the **Meetings** tab (or **More ▾ → Per-meeting settings**) opens a list of meetings. Filter by name, tick meetings, and change whether the AI assistant analyzes them, their publish flag, working directory, destination (Artifact or a Notion URL) and whether the AI analysis Advice is published, all at once. The ⚙ on a row edits that one meeting from its current values. With Advice published, clerk-meeting-helper keeps the published copy up to date during the meeting so everyone can follow it. Values go to `meeting.yaml`; a meeting that matched a rule shared with other meetings gets its own `^name$` rule, copied from that rule and placed just before it. The first time shadow-clerk writes to a `config.yaml` it didn't create itself, it saves a one-time `<path>.bak` copy alongside it, since the YAML writer preserves keys but not your hand-written comments.
- **Analyze: Default / On / Off**: each meeting's `analyze` has three states. **Default** (no `analyze` in `meeting.yaml`) follows `auto_analyze`. **On** launches the assistant when the meeting starts even if `auto_analyze` is off; **Off** never launches it, even with `auto_analyze` on. The AI assistant box in the **▶ Start meeting** dialog overrides this for that one meeting only, and asking the assistant to follow by hand during the meeting always wins over an Off.
  - **Upgrade note**: the previous settings modal saved `analyze: true` for every meeting saved on its own (for example to set a working directory), whether or not you changed it. That value now means **On** and launches the assistant even with `auto_analyze` off. If you don't want that, set those meetings back to **Default** in **⚙ Per-meeting settings**, or remove `analyze: true` from `meeting.yaml`.
- **Permissions**: the assistant runs shell scripts from the meeting skill, so allow them in that working directory's `.claude/settings.json` — otherwise it stops mid-meeting at a permission prompt.
- **Orphaned process on SIGKILL**: the daemon stops the assistant on normal shutdown, but if the daemon itself is killed with SIGKILL (`kill -9`), the assistant process can be left running (it is detached from any terminal). Find and kill it manually with `pgrep -af claude` (or `codex`).
- **Security note if you expose the dashboard**: the AI Console's terminal content (including whatever the assistant reads or prints from your files) is streamed over the same `/api/events` SSE used by the rest of the dashboard, and that SSE fan-out has no per-client filtering. If you bind the dashboard beyond localhost, anyone who can reach it can watch the assistant's terminal live. Console input (`/api/console/input` etc.) itself stays localhost-only and additionally checks the `Origin` header to reject cross-origin requests from your own browser.

### Browser screenshots

A small Chrome extension in [`extension/`](extension/) captures the visible tab — a shared screen in a browser meeting, say — saves the image in the data directory, and adds a `[画面]` line to the transcript being recorded, so the image sits on the same timeline as the speech. Load it unpacked from `chrome://extensions`; see [extension/README.md](extension/README.md) for installation, settings and what gets written.

## Configuration

### Configuration file

Customize defaults and auto-features in `~/.local/share/shadow-clerk/config.yaml`. The main keys, with their defaults:

```yaml
# --- Meeting automation ---
translate_language: en        # Translation target language (ja/en/etc)
auto_translate: false         # Auto-start translation on start meeting
auto_summary: false           # Auto-generate summary on end meeting
auto_summary_via_console: true  # With auto_summary: let the AI Console write the minutes when it is running
auto_analyze: false           # Launch the AI assistant and run the meeting skill when a meeting starts

# --- AI Console ---
ai_assistant_command: claude  # Command to run in the AI Console (claude, codex, ...)
ai_assistant_args: ''         # Arguments for that command, split with shlex
ai_assistant_init_prompt: /clerk-meeting-helper {transcript} {lang}  # Sent to the PTY once the TUI is ready. {transcript}, {meeting} and {lang} are substituted ({lang} = translate_language)
ai_assistant_workdir: ''      # Default working directory (e.g. a Git repository), added as a reference folder. Per-meeting overrides live in meetings[].workdir of DATA_DIR/meeting.yaml
ai_reference_dirs: []         # Folders the assistant may read in every meeting (relative = from the data directory)

# --- Transcription ---
default_language: null        # Default language for clerk-daemon (null=auto-detect)
default_model: small          # Default Whisper model for clerk-daemon
output_directory: null        # Transcript output directory (null=data directory)
transcript_categories: [private, sharable, isolated]  # Record categories; the first is the default
initial_prompt: null          # Whisper initial_prompt (vocabulary hints for recognition)
whisper_beam_size: 5          # Whisper beam size (1=fast, 5=accurate)
whisper_compute_type: int8    # Compute precision (int8/float16/float32)
whisper_device: cpu           # Device (cpu/cuda)
whisper_vad_filter: true      # Silero VAD: drop non-speech inside each segment
whisper_vad_threshold: 0.35   # Silero VAD speech threshold (0.2 / 0.35 / 0.5)
interim_transcription: false  # Interim transcription (real-time display while speaking)
interim_model: base           # Model for interim transcription
japanese_asr_model: default   # Japanese ASR model (default/kotoba-whisper/reazonspeech-k2)
kana_hint: false              # Append "(heard kana?)" to Japanese words that disagree with the audio (kana-hint extra)
kotoba_whisper_model: kotoba-tech/kotoba-whisper-v2.0-faster  # Kotoba-Whisper model
interim_japanese_asr_model: default  # Japanese ASR for interim transcription
english_asr_model: default    # English ASR (default/moonshine/a faster-whisper model name such as medium.en)
asr_engine: whisper           # ASR engine for all languages (whisper/moonshine)
interim_asr_engine: whisper   # ASR engine for interim transcription
reazonspeech_model: ja        # ReazonSpeech k2: ja / ja-en (Japanese-English bilingual)
reazonspeech_precision: fp32  # ReazonSpeech k2: fp32 / int8 / int8-fp32 (fp16 is invalid)

# --- Audio devices ---
mic_device: null              # Microphone device name (null=OS default; not an index — indices shift between runs)
monitor_device: null          # Monitor/speaker device name (null=OS default). Selectable from the dashboard settings panel

# --- LLM, translation and summary ---
llm_provider: claude          # LLM for summary ("claude" or "api")
translation_provider: null    # Translation provider (null=use llm_provider, "claude", "api", "libretranslate")
claude_cli_path: claude       # claude command (full path if not on PATH)
claude_cli_model: haiku       # Model for claude -p (haiku / sonnet / opus / full model id)
translation_claude_model: null # claude -p model for translation only (null = claude_cli_model)
api_endpoint: null            # OpenAI Compatible API base URL
api_model: null               # API model name (gpt-4o, etc.)
api_key_env: SHADOW_CLERK_API_KEY  # Environment variable name for API key
api_disable_thinking: false   # Disable reasoning-model thinking for translation/interim (Qwen3 etc.; sends enable_thinking=false). Summary always keeps thinking.
interim_translation: true     # Translate interim transcription to dashboard's interim panel
interim_translation_provider: null  # null=auto, "api", "libretranslate", or "claude"
translation_hiragana_step: true  # Have the LLM reread Japanese as kana before translating, to catch misrecognized homophones
libretranslate_endpoint: null     # LibreTranslate API URL (e.g. http://localhost:5000)
libretranslate_api_key: null      # LibreTranslate API key (null if not required)
libretranslate_spell_check: false # Spell check before LibreTranslate translation
spell_check_model: mbyhphat/t5-japanese-typo-correction  # Spell check model
summary_source: null          # Summary source (null=auto: prefer translation if exists / "transcript" / "translate")
summary_language: null        # Summary output language (null=fallback to ui_language / ja, en, zh, ...)
summary_length: half          # Minimum length of the minutes (half / 1page / 2pages ... 5pages, in A4 pages)
summary_hiragana_step: true   # Same kana rereading step before summarizing

# --- Voice commands ---
voice_command_key: f23        # Push-to-Talk key (null=disabled)
wake_word: シェルク             # Wake word (trigger word for voice commands)
custom_commands: []           # Custom voice commands (list of pattern + action)

# --- Google Calendar ---
gcal_integration: false       # Start/end meetings from calendar events
gcal_credentials_file: null   # OAuth credentials.json
gcal_token_file: null         # Saved token (null=DATA_DIR/gcal_token.json)
gcal_calendar_id: primary     # Calendar to watch
gcal_buffer_minutes: 2        # Send start_meeting this many minutes before an event starts
gcal_end_buffer_minutes: 1    # Send end_meeting this many minutes after it ends

# --- Dashboard ---
dashboard_port: 8765          # Dashboard port. --dashboard-port overrides it; clerk-util reads this key to reach the daemon
gui_scale: 0                  # Display scale for `clerk-util gui`. 0 asks the compositor; set 0.5-4.0 if the window comes out too small or too large

# --- UI ---
ui_language: ja               # UI language (ja/en) — dashboard, terminal output, LLM prompts
```

Talk mode keys (`talk_*`) are listed under [Talk with Claude](#talk-with-claude). The dashboard also keeps a few keys of its own in this file (`welcome_dismissed`, `skill_update_dismissed_version`, `skill_install_targets`); you don't need to edit them.

Manage configuration from the command line (or from **More ▾ → ⚙ Settings** on the dashboard):

```
clerk-util read-config                                # Show current config
clerk-util write-config-value default_model tiny      # Change a setting
clerk-util write-config-value auto_translate true     # Enable auto-translation
```

With `auto_translate: true`, translation starts automatically when a meeting session begins.
With `auto_summary: true`, meeting minutes are generated automatically when a meeting session ends.
With `auto_analyze: true`, the AI assistant is launched in the AI Console when a meeting session begins.

### Summary source selection

When `summary_source` is unset (null/auto), the summary is generated from the translation file if one exists (falling back to the transcript if not). To pin the behavior explicitly:

```
clerk-util write-config-value summary_source transcript   # always use transcript
clerk-util write-config-value summary_source translate    # always use translation (fallback to transcript if missing)
```

### Summary language

`summary_language` controls the output language of the summary. When unset (null), it falls back to `ui_language`:

```
clerk-util write-config-value summary_language en   # summarize in English
clerk-util write-config-value summary_language ja   # summarize in Japanese
```

## File structure

```
shadow-clerk/                          # Repository
  pyproject.toml                       # Project definition & dependencies
  src/shadow_clerk/                    # Main package
    clerk_daemon.py                    # clerk-daemon entry point (recording, transcription, dashboard: _daemon_*.py)
    clerk_util.py                      # clerk-util: data directory operations & process management
    llm_client.py                      # Translation, summary & LLM queries (_llm_*.py)
    gcal_monitor.py                    # Google Calendar polling
    skill_install.py                   # install-skill
    i18n.py                            # Internationalization (ja/en)
    domain/                            # Domain value objects
    skills/                            # Bundled skills: clerk-meeting-helper, clerk-talk, clerk-practice, voicevox-sing, clerk-conversation-retrospective
    talk_prompts/                      # Talk mode system prompts
  extension/                           # Chrome screenshot extension
  packaging/                           # PyInstaller spec and hooks
  docs/                                # Feature tour, Google Calendar setup
  tests/                               # Tests
  SPEC.md                              # Architecture and module design (Japanese)

~/.local/share/shadow-clerk/           # Runtime data
  <category>/<meeting name>/           # One folder per meeting: every file below that belongs to it
  <first category>/@dates/YYYY-MM/     # Records without a meeting name (daily files, unnamed meetings)
  transcript-YYYYMMDD.txt              # Transcription output (date-based)
  transcript-YYYYMMDDHHMM.txt          # Meeting session transcript
  transcript-YYYYMMDDHHMM@Title.txt    # Meeting session transcript (with event title)
  transcript-YYYYMMDD-<lang>.txt       # Translation output
  transcript-YYYYMMDDHHMM@Title.attendees.json  # Expected attendees from the calendar event
  summary-YYYYMMDD.md                  # Meeting minutes (corresponds to transcript)
  summary-YYYYMMDDHHMM@Title.md        # Meeting minutes (named session)
  advice-YYYYMMDDHHMM@Title.md         # AI Console: open questions and suggestions
  analysis-YYYYMMDDHHMM@Title.md       # AI Console: confirmed facts
  shot-YYYYMMDDHHMM@Title-HHMMSS.png   # Browser screenshots
  meeting.yaml                         # Per-meeting working directories
  glossary.txt                         # Glossary (TSV: translation terms & reading-based text replacement)
  misheard.tsv                         # Misheard word pairs collected by the skills
  config.yaml                          # Configuration file
  .env                                 # API key (SHADOW_CLERK_API_KEY)
  daemon.log / daemon.pid              # Daemon log and PID
  gcal_token.json                      # Google Calendar OAuth token (created by gcal-auth)
  gcal_processed.json                  # Calendar events already started/ended (survives daemon restarts)
  wespeaker_cache/                     # Speaker diarization ONNX model (downloaded on first use)
  speaker_names.json                   # Names, groups and colors of speakers (no voice data)
  voiceprints.json                     # Voiceprints (speaker_enrollment, after naming)
  transcript-YYYYMMDDHHMM@Title.voices.jsonl  # Speaker diarization embeddings (speaker_enrollment)
  transcript-YYYYMMDDHHMM@Title.naming.jsonl  # Naming history of the meeting (for undo)
  transcript-YYYYMMDDHHMM@Title.provisional.json  # Speaker names given during the meeting (applied when it ends)
```

The `transcript-*`, `summary-*`, `advice-*`, `analysis-*`, `minutes-*` and `shot-*` files above live in their
meeting's folder, not directly in the data directory. File names keep `@Title`, so a file copied elsewhere still
says which meeting it came from. Categories (`transcript_categories`, default `private` / `sharable` / `isolated`)
are only places: they carry no behavior. Pick a meeting's category in **⚙ Per-meeting settings** — changing it
moves the meeting's folder. On startup the daemon moves any flat `transcript-*` etc. left in the data directory
into this layout (all into the first category; meeting names are re-sanitized), so it is safe to run repeatedly.

## Troubleshooting

### Device not found

```bash
# List available devices
clerk-daemon --list-devices

# PipeWire: check status
wpctl status

# PulseAudio: list sources
pactl list short sources
```

### Monitor source (system audio) not detected

On PipeWire, check sink (output) devices with `wpctl status`.
On PulseAudio, look for sources containing `.monitor` with `pactl list short sources`.

You can also specify the device number manually:

```bash
clerk-daemon --monitor 5
```

### PortAudio error

Make sure `libportaudio2` is installed:

```bash
dpkg -l | grep portaudio
```

If you see `PortAudioError: Error initializing PortAudio: ... PulseAudio_Initialize: Can't connect to server`, the PulseAudio-compatible service may have crashed. On PipeWire systems, restart `pipewire-pulse`:

```bash
systemctl --user restart pipewire-pulse
```

### Slow transcription

Use a lighter model with `--model tiny`:

```bash
clerk-daemon --model tiny
```

### ASR engine (Moonshine Voice)

`asr_engine: moonshine` (and `interim_asr_engine` for interim transcription) switches
recognition to [Moonshine Voice](https://github.com/moonshine-ai/moonshine) for every
language it supports (ar, de, en, es, ja, ko, tl, uk, vi, zh). It needs the `moonshine`
extra, and the model for the current language is downloaded on first use. Languages it
does not support, and `default_language: auto`, fall back to Whisper. For `ja`, a
`japanese_asr_model` other than `default` still takes precedence.

On CPU it is several times faster than Whisper small with comparable or better Japanese
accuracy. `initial_prompt` is passed as Moonshine's context (term biasing); the Whisper
settings (`default_model`, `whisper_beam_size`, `whisper_compute_type`) are not used.

> **License:** only the English models are MIT. Models for other languages, including
> Japanese, are under the non-commercial [Moonshine Community License](https://www.moonshine.ai/license).

### Japanese ASR models

The `japanese_asr_model` setting selects the ASR backend used when `language=ja`. When the language changes to something other than `ja`, it automatically reverts to standard Whisper.

| Value | Model | Requires | Japanese accuracy | CPU speed |
|---|---|---|---|---|
| `default` | Standard Whisper | — | Depends on model size | Depends on model size |
| `kotoba-whisper` | [Kotoba-Whisper](https://huggingface.co/kotoba-tech/kotoba-whisper-v2.0) | Auto-downloaded on first use | High (rivals large-v3) | ~medium |
| `reazonspeech-k2` | [ReazonSpeech k2](https://github.com/reazon-research/ReazonSpeech) | `uv sync --extra reazonspeech` | High | Fast |

**Kotoba-Whisper** retains the full large-v3 encoder (32 layers) while distilling the decoder down to just 2 layers. Since it has only 2 decoder layers, **beam=5 has almost no speed penalty**.

**ReazonSpeech k2** uses sherpa-onnx for inference. When selected, Whisper-specific settings (`default_model`, `whisper_beam_size`, `whisper_compute_type`, `initial_prompt`) are not used.

`reazonspeech_model: ja-en` switches to the Japanese-English bilingual Zipformer. Japanese
accuracy matches `ja`, and English speech is transcribed instead of collapsing into
`はい`; it is also slightly faster. The upstream repository has been made private, so it is
downloaded from the mirror kept by the sherpa-onnx author (`csukuangfj/reazonspeech-k2-v2-ja-en`).
`reazonspeech_precision: int8` is about twice as fast as `fp32` with near-identical output.

**Selection guide:**

| Use case | Settings |
|---|---|
| **Japanese meetings on CPU (recommended)** | `japanese_asr_model: reazonspeech-k2`, `reazonspeech_model: ja-en`, `reazonspeech_precision: int8` |
| Japanese-focused, accuracy priority (GPU) | `japanese_asr_model: kotoba-whisper`, `whisper_beam_size: 5` |
| Japanese-focused, speed priority (CPU, no extras) | `japanese_asr_model: default`, `default_model: small`, `whisper_beam_size: 3` |
| Non-Japanese languages, fewer hallucinations | `asr_engine: moonshine` (English models are MIT; others non-commercial) |
| Multilingual | `japanese_asr_model: reazonspeech-k2`, `reazonspeech_model: ja-en`, `default_model: small` |

Measured on CPU (int8) with real conversational clips split by the daemon's own VAD:
ReazonSpeech k2 `ja-en` was the fastest (~0.2 s per utterance) with Japanese accuracy equal
to `ja`, and it also transcribes English, which `ja` collapses into `はい`. Whisper small was
about 20× slower and Kotoba-Whisper about twice as slow again, without being more accurate. Moonshine's
Japanese model is close in Japanese but loops on English speech.

**Whisper: Silero VAD (`whisper_vad_filter`, on by default)**

Segments are cut by webrtcvad, which also lets keyboard noise and breathing through.
Without a second VAD, Whisper answers such noise-only segments with its own
`initial_prompt` — the wake word — and only the `no_speech_prob` filter stands between
that and a false voice command. `whisper_vad_filter` runs faster-whisper's built-in Silero
VAD inside each segment: noise-only segments produce nothing and cost almost no CPU, and
normal-volume speech transcribes the same. It applies to Whisper and Kotoba-Whisper, not
to ReazonSpeech or Moonshine.

The trade-off is very quiet speech. With the speech at 1/10–1/20 volume plus noise, the
wake word survived 9/12 times without VAD, 4/12 with threshold 0.35 and 3/12 with the
upstream default 0.5 — hence the default of `whisper_vad_threshold: 0.35`. If voice
commands are missed on a quiet microphone, turn `whisper_vad_filter` off.

`condition_on_previous_text: false`, often recommended alongside it, is deliberately not
used: it only matters across 30-second windows inside one call, and every segment here is
transcribed on its own, so it changed nothing on short segments and caused a repetition
loop on a long mixed-language one.

**Interim transcription:**

`interim_japanese_asr_model` controls which Japanese ASR model is used for interim transcription (real-time display while speaking). On CPU, keeping the default (`default` with a lightweight model like tiny/base) is recommended.

```yaml
# Japanese accuracy priority (GPU recommended)
japanese_asr_model: kotoba-whisper
interim_japanese_asr_model: kotoba-whisper
whisper_beam_size: 5

# Japanese accuracy + fast interim (CPU recommended)
japanese_asr_model: kotoba-whisper
interim_japanese_asr_model: default
interim_model: base
whisper_beam_size: 5        # Kotoba has only 2 decoder layers, beam=5 is fine

# ReazonSpeech (fast & accurate, CPU friendly)
japanese_asr_model: reazonspeech-k2
interim_japanese_asr_model: default
interim_model: base

# Maximum speed (CPU)
japanese_asr_model: default
default_model: small
interim_model: base
whisper_beam_size: 1
```

**Interim translation:**

When `interim_transcription` is on, the daemon also emits a translation of each pre-confirmed line to the dashboard's interim panel. Two knobs control this:

- `interim_translation: true` — toggle the translation panel without disabling interim ASR.
- `interim_translation_provider: null | "api" | "libretranslate" | "claude"` — pick the backend explicitly. `null` falls back to `translation_provider`; if that is `claude` it is auto-routed to `api` then `libretranslate` (claude is too slow for interim, ~5-10s per call). Set to `claude` only if you accept the latency.

The interim panel needs sub-second responses to be useful, so `libretranslate` (local) is recommended; `api` is OK with a fast model. Confirmed-transcript translation is unaffected — it always uses `translation_provider`.

## Building standalone binaries

`packaging/shadow-clerk.spec` produces a one-directory PyInstaller bundle containing both `clerk-daemon` and `clerk-util`.

**Without a Windows machine**, `.github/workflows/build-binary.yml` builds the bundle on `windows-latest` and `ubuntu-latest` with ReazonSpeech and Google Calendar included. Push a `v*` tag and both
binaries are attached to the release, or start it by hand from the Actions tab and
download them as artifacts.

**PyInstaller does not cross-compile.** It only builds for the OS it runs on: the bootloader is a native binary, and the analysis step imports every module to trace dependencies. A Windows `.exe` therefore has to be built on Windows — a machine, a VM, or a `windows-latest` GitHub Actions runner. Wine with a Windows Python is the usual workaround, but analysis imports `pywinpty` (ConPTY), `PyAudioWPatch` (WASAPI) and `ctranslate2`, which are exactly the pieces Wine emulates poorly.

To build locally:

```powershell
# Windows (PowerShell)
uv sync
uv run --with pyinstaller pyinstaller packaging/shadow-clerk.spec
dist\shadow-clerk\clerk-daemon.exe --list-devices   # smoke test
```

```bash
# Linux / macOS
uv sync
uv run --with pyinstaller pyinstaller packaging/shadow-clerk.spec
./dist/shadow-clerk/clerk-daemon --list-devices      # smoke test
```

`--list-devices` is a good smoke test: it exercises the bundled PortAudio and the native extensions without recording anything.

To bundle ReazonSpeech and Google Calendar as well, replace the `uv sync` line — in
this order, because `reazonspeech-k2-asr` is not declared anywhere and a later sync
would remove it:

```powershell
uv sync --extra reazonspeech --extra gcal
uv pip install "reazonspeech-k2-asr @ git+https://github.com/reazon-research/ReazonSpeech.git#subdirectory=pkg/k2-asr"
uv run python -c "import sherpa_onnx, reazonspeech.k2.asr; print('ok')"   # check before building
uv run --with pyinstaller pyinstaller packaging/shadow-clerk.spec
```

The `spell-check` extra is **not** bundled even if you install it: the spec drops
`torch`, `transformers` and `sentencepiece` in `_EXCLUDES`. Remove them from that
list to include it, and expect the bundle to grow by several GB.

**Do not `uv pip install pyinstaller`.** `uv sync` makes the environment match what the project declares and removes everything else, so the next `uv sync --extra ...` would drop it again. `--with` puts PyInstaller in a temporary layer over the project environment instead: it can still see the project's packages, and nothing is left behind to be removed.

Notes:

- **Output**: `dist/shadow-clerk/`, roughly 460 MB with the default dependencies and no extras.
- **Extras are collected only if installed** in the environment you build from, so install them before building — every extra you want named in one `uv sync`, and any `uv pip install` last. See [Setup](#2-install) for why the order matters. The spec collects `sherpa_onnx` (which carries its own `onnxruntime` DLL in `lib/`) and `reazonspeech.k2.asr` when they are there; the ASR weights themselves are fetched on first use, like the Whisper models.
- **Whisper models are not bundled.** `small` is around 500 MB and is fetched from Hugging Face on first run, then cached (`%USERPROFILE%\.cache\huggingface` on Windows, `~/.cache/huggingface` elsewhere). For an offline bundle, add that cache to `datas` in the spec, or ship a converted CT2 model and point `--model` at it.
- **`packaging/hooks/` overrides PyInstaller's bundled hooks.** There is one today: the bundled `hook-webrtcvad.py` calls `copy_metadata('webrtcvad')`, but this project depends on `webrtcvad-wheels`, so without the override the build aborts with `ImportErrorWhenRunningHook`.
- Add a dependency that ships DLLs or data files? Add it to `_PACKAGES` in the spec. PyInstaller only follows `import` statements, so anything else is silently left out and fails at runtime rather than at build time.
