Metadata-Version: 2.4
Name: vaani-voice
Version: 0.1.1
Summary: Vaani: talk to your coding agent and hear it answer. Local speech, English and Hindi, macOS and Windows.
Author-email: Kirtika <impexinfodev@gmail.com>
Maintainer-email: Kirtika <impexinfodev@gmail.com>
License: LicenseRef-PolyForm-Shield-1.0.0
Project-URL: Homepage, https://github.com/kirtika01/vaani
Project-URL: Source, https://github.com/kirtika01/vaani
Project-URL: Issues, https://github.com/kirtika01/vaani/issues
Keywords: voice,speech,hindi,hinglish,claude-code,developer-tools
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: Other/Proprietary License
Classifier: Operating System :: MacOS :: MacOS X
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE.md
License-File: NOTICE
Requires-Dist: pyyaml>=6.0
Requires-Dist: platformdirs>=4.2
Requires-Dist: rich>=13.7
Requires-Dist: pyobjc-framework-Speech>=10.0; sys_platform == "darwin"
Requires-Dist: pyobjc-framework-AVFoundation>=10.0; sys_platform == "darwin"
Requires-Dist: sounddevice>=0.4; sys_platform != "darwin"
Requires-Dist: faster-whisper>=1.0; sys_platform != "darwin"
Requires-Dist: numpy>=1.26; sys_platform != "darwin"
Provides-Extra: portable
Requires-Dist: sounddevice>=0.4; extra == "portable"
Requires-Dist: faster-whisper>=1.0; extra == "portable"
Requires-Dist: numpy>=1.26; extra == "portable"
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: ruff>=0.6; extra == "dev"
Requires-Dist: mypy>=1.11; extra == "dev"
Requires-Dist: types-PyYAML>=6.0; extra == "dev"
Dynamic: license-file

<div align="center">

# Vaani

**Talk to your coding agent. Hear it answer.**

Local speech in and out, in English *and* Hindi, on macOS and Windows.
No API key, no speech service, nothing leaves your machine.

Built by **Kirtika** · [Report an issue](https://github.com/kirtika01/vaani/issues)

</div>

---

> **वाणी** *(vaani)* — speech, voice. Say what you want built; hear what was done.

```
you      is folder mein ek hello python file banao aur chalao
vaani    Ek second, soch raha hoon.
         Main ek command chala raha hoon.
claude   Ho gaya. Maine hello folder banaya, usme namaste.py likhi,
         aur chalane par output aaya: Namaste.
         [2.2s to the first spoken word]
```

## Why it is different

* **It answers while it is still thinking.** The reply is spoken sentence by sentence
  as the agent writes it — about **2 seconds** from the end of your question to the
  first word, instead of waiting for the whole answer.
* **Talk over it.** Interrupt a reply mid-sentence and it stops and listens, through
  Apple's echo canceller — no headphones needed on a Mac.
* **Hindi is not an afterthought.** Hinglish is understood as spoken, replies come
  back in Hinglish, and Devanagari is routed to a Hindi voice automatically. Commands
  work in both languages: *"targets batao"*, *"listurad pe jao"*, *"band karo"*.
* **It says what it is doing.** "Ek second…", "Main files dekh raha hoon", then a
  spoken summary of what changed — so a long task never sounds like a hang.
* **Your machine only.** On-device recognition, system voices, your own agent login.
  Audio is never written to disk; transcripts stay out of the log.
* **Two ways to use it**: a terminal loop, or an editor panel (VS Code, Cursor,
  Windsurf, Antigravity).

## Proof it works

| | |
|---|---|
| Automated tests | **885**, with lint and strict type checking |
| Live checks through a real microphone and speakers | `python scripts/live_check.py` |
| Platforms tested in CI | macOS, Windows, Linux |
| First spoken word after a question | ~2.2s, measured |

## Requirements

- macOS
- Python 3.11 or newer

## Setup

```bash
./scripts/setup.sh
source .venv/bin/activate
```

The script creates `.venv`, installs the package in editable mode with the dev
extras, and runs a smoke check.

## Usage

```bash
voice-claude config              # print the effective configuration
voice-claude config --sources --paths
voice-claude doctor              # one-command self-test: env, mic, speech, Claude Code
voice-claude doctor --strict     # also exit non-zero if a readiness check fails
voice-claude --version

voice-claude stt setup           # build the speech helper, grant permissions (once)
voice-claude stt check           # is speech recognition ready?
voice-claude stt doctor          # diagnose permission / bundle-identity problems
voice-claude stt diagnose        # record a sample, report mic levels + recogniser health
voice-claude stt check --locales # every locale this Mac supports
voice-claude stt listen          # record a phrase, print the transcript + latency

voice-claude claude check        # can prompts reach Claude Code?
voice-claude claude send "..."   # submit a typed prompt

voice-claude tts check           # is local speech output ready?
voice-claude tts voices          # what voices this Mac has
voice-claude tts speak "..."     # speak through the Mac speakers
voice-claude tts stop            # stop speech that is playing

voice-claude voice               # the full loop: speak, hear Claude's reply
voice-claude voice --continuous  # hands-free: no keypress between turns
voice-claude targets             # the workspaces a turn can be sent to
voice-claude voice --wake        # stay idle until "hey claude"
voice-claude wake-word check     # what the wake phrase accepts
```

`python -m voice_claude` works identically.

## Speech-to-text

### First run

```bash
voice-claude stt setup
```

macOS will ask twice — **Microphone** and **Speech Recognition** — attributed to
"voice-claude speech helper". Approve both. This is needed once; re-run the command
if you dismiss a prompt. You can also grant access later under
**System Settings → Privacy & Security → Microphone / Speech Recognition**.

If System Settings shows **VoiceClaudeSpeech** enabled but the CLI still reports it
as not granted, run:

```bash
voice-claude stt doctor
```

It prints the bundle identity macOS actually sees and the authorization status under
both spawn modes, which is enough to tell a real refusal from a misattribution.

### Dictating

```bash
voice-claude stt listen
```

`space` starts and stops the recording, `esc` cancels it, `q` quits. All three are
configuration, not constants — see `stt.push_to_talk` below. Each capture prints the
transcript followed by the audio duration, the transcription latency and the
end-to-end wall time.

Keys buffered while the microphone is opening are discarded, and a stop arriving
within `audio.min_recording_s` is ignored — otherwise auto-repeat or an impatient
double-tap ends the recording before any audio exists, which reads as
"no speech detected". Cancel is never delayed.

### If a recording comes back empty

```bash
voice-claude stt diagnose --seconds 5
```

It records a short sample and reports, without ever printing your audio or
transcript: whether the microphone opened, how many samples arrived, peak and RMS
level in dBFS, whether the buffer was silent, the recogniser's authorization and
availability, and the transcription latency. It then names the stage that lost the
audio — no audio at all, silence, or audio that the recogniser rejected.

`stt listen` uses the same metering to annotate an empty result, e.g.
`(no speech detected) - the audio was silent (peak -inf dBFS); check your input device`.

> **The audio format matters more than it looks.**
> `SFSpeechAudioBufferRecognitionRequest` accepts interleaved Int16 PCM and then
> reports "no speech detected" for audio that is plainly not silent. It is fed the
> input node's **native** float32 format instead. The configured
> `audio.sample_rate`/`channels`/`sample_format` govern metering and the optional
> saved recording, which is where they actually matter.

### Language support

The default locale is **`en-IN`**. On this machine it is the only locale that runs
fully on-device *and* handles English, Hindi and Hinglish code-switching in a single
pass, romanising the Hindi words:

| You say | `en-IN` gives you |
| --- | --- |
| "Refactor the login handler" | `Refactor the login handler` |
| "login handler ko refactor karo" | `Login handler ko refactor karo` |

For Devanagari output set `stt.locale: hi-IN`, or pass `--locale hi-IN` for one run.
**Hindi locales have no on-device asset**, so they go to Apple's (free) speech
service and need network access; expect roughly 1.5–2s instead of ~0.2s.

`hi-Latn` and `hi-IN-translit` appear in the supported-locale list but fail in
practice — the recogniser returns `kAFAssistantErrorDomain error 1`. Use `en-IN`.

### How it works, and why it looks unusual

macOS will not let a process use speech recognition unless its main bundle declares
`NSSpeechRecognitionUsageDescription`. A plain `python` binary has no such
Info.plist, and macOS does not return an error — it kills the process with SIGABRT.

So `voice-claude stt setup` builds a small `.app` under
`~/Library/Application Support/voice-claude/helper/`, whose executable is a copy of
the framework Python binary, and runs the capture in there. Two details are
load-bearing and were established by experiment:

* the copy must be the **`Python.app` stub inside the framework**, not `bin/python3.x`
  — the latter re-execs into Homebrew's own `Python.app`, so `NSBundle.mainBundle()`
  resolves to `org.python.python` and our Info.plist is never read
* the right Info.plist is necessary but **not sufficient**. TCC judges the process
  *responsible* for the helper, and an ordinarily-spawned child inherits the
  terminal's responsibility. So the helper is spawned with
  `responsibility_spawnattrs_setdisclaim`, making it responsible for itself. The
  one-time permission prompt goes through LaunchServices (`open`), which does the
  same thing implicitly and also registers the app in System Settings.

Get that second point wrong and macOS shows the helper switched **on** while
`authorizationStatus()` answers `notDetermined` — the two are asking about different
processes. `voice-claude stt doctor` shows both answers side by side.

Rebuilding the bundle changes its code signature and resets the grant, so an
up-to-date bundle is deliberately left untouched.

### Privacy

Two defaults are promises, and there are tests pinning both:

* **audio is never written to disk.** It is streamed straight into the recogniser.
  Set `audio.save_recordings: true` to keep recordings; the log warns when you do.
* **transcripts stay out of the log.** Records carry timings and counts — character
  count, word count, latency — but not what you said. Set `stt.log_transcripts: true`
  to include the text.

With `en-IN` and `prefer_on_device: true` the audio never leaves the machine at all.
Hindi locales are the exception noted above.

## The voice loop

```bash
voice-claude voice                    # push-to-talk, Claude Code, spoken reply
voice-claude voice --continuous       # hands-free; no keypress between turns
voice-claude voice --once             # a single turn, then exit
voice-claude voice --no-speak         # print the reply instead of speaking it
voice-claude voice --max-spoken 400   # speak the first ~25s, print the rest
voice-claude voice --target listurad  # start on a named workspace
```

`space` starts and finishes a recording, `esc` cancels, `q` quits — the same
bindings as `stt listen`, from the same `stt.push_to_talk` configuration.

One turn runs `LISTENING -> SENDING -> SPEAKING`, printing each stage as it goes.
All three subsystems are checked before the microphone is ever opened, so a missing
permission or a broken voice is reported up front rather than mid-sentence.

### Speed

Measured on this Mac before this work, a two-word question took 12.8s from the start
of speaking to the end of the reply, and most of that was not Claude thinking:

| where the time went | before | now |
|---|---|---|
| waiting to be sure you had finished | 1.5s of silence | 1.0s (`voice.utterance_silence_s`) |
| starting `claude` for the turn | ~3.5s every turn | none: one process per target stays running |
| loading claude.ai MCP connectors | ~2.2s of that start-up | skipped (`claude.load_mcp_servers: false`) |
| waiting for the whole reply before speaking | the full reply | speech starts with the first sentence |
| pause before listening again | 0.5s | 0.2s (`voice.settle_after_speech_s`) |

**One Claude Code process per target** stays running between turns
(`claude.session: persistent`), started as soon as the loop starts, using Claude Code's
documented streaming mode (`--input-format stream-json`). Its reply arrives as it is
written, and each complete sentence is spoken straight away; sentences that arrive
while one is playing are spoken together next. If the process dies, the next turn
starts another that resumes the same conversation. `claude.session: per-turn` restores
a fresh `claude -p` per prompt.

**While Claude works, the status line says what it is doing** — `Claude is reading
files...`, `Claude is running a command...` — so a long turn visibly makes progress.

**Replies are shaped for listening.** `claude.append_system_prompt` asks for short spoken
sentences without markdown, a one-line heads-up before multi-step work, and a
clarifying question instead of a long exploration when a request is too vague. Set it
to `""` to send nothing extra.

A request that makes Claude read files and run commands still takes as long as that
work takes; this removes the overhead around it, not the work.

### Talking like a conversation

* **Everything is spoken** when `voice.max_spoken_chars` is `0`; talk over a reply to stop it.
* **Progress is spoken while Claude works** (`voice.narrate`): "One moment" if nothing has
  been said after `voice.thinking_filler_s`, then "Let me look at the files", "Running a
  command" as tools start, in Hinglish when you spoke Hindi. A phrase is never said over
  speech or within 3s of Claude's own words. Claude is also asked to say what it is about
  to do, and to finish with a short summary of what it did.
* **Hindi.** Measured on this Mac by synthesizing replies and transcribing them back:
  an English voice reads Devanagari as *silence*; romanised Hinglish read by an Indian
  English voice (Rishi/Aman/Tara, word error ~0.32) beat the system default (0.42) and
  the Hindi voice Lekha reading Devanagari (0.47, worst on English technical words).
  So Claude is asked for Hinglish in Latin script, and any Devanagari sentence that still
  appears is routed to `tts.hindi_voice` (Lekha) automatically. Input stays `en-IN`: the
  on-device `hi-IN` recogniser garbled English words ("snakes and ladders").
* **Permissions.** `claude.permission_mode: auto` is the mode Claude Code's IDE session
  uses: its own safety check lets ordinary work (folders, files, running the project)
  through without asking and still blocks risky actions. Whatever it blocks is asked out
  loud in hands-free mode (`voice.voice_approval`) — say "yes"/"haan" or "no"/"nahi"; a
  refusal anywhere in the answer wins. voice-claude never passes `bypassPermissions`.

### Hands-free mode

`--continuous` takes turns without a keypress. An utterance ends on the pause after
it, so you simply speak when the line says `listening...`.

This needs a different question answered than "did any audio arrive". Room noise sits
comfortably above the dead-microphone floor (`audio.silence_rms_threshold`, ~-48 dBFS),
so using that to detect the end of speech never fires. Instead the first 0.6s of each
recording learns the room's noise level, and speech must exceed it by 3x (with an
absolute floor) to count. The pause is then measured from the last buffer that cleared
*that* bar.

A turn also gives up by itself if nobody starts speaking within `listen_timeout_s`,
and those silent cycles are not announced — otherwise a quiet spell would bury the
turns that actually happened. After speaking, the loop waits
`settle_after_speech_s` before reopening the microphone, so the tail of Claude's reply
is not heard as the next instruction.

Press `q` to stop, or say **"stop listening"**.

### Wake phrase

```bash
voice-claude voice --continuous --wake
voice-claude wake-word check
voice-claude wake-word test "Hey Claude, switch to listurad"
```

With `--wake` the loop stays idle until it is addressed. Say it **as one sentence with
the command**:

```
Hey Claude, switch to listurad
Hey Claude, create a hello world function
```

> **A bare "Hey Claude" is unreliable.** On-device `en-IN` hears those two words alone
> as **"Hey Lord"** — measured, not guessed. Fuzzy matching accepts that mishearing, and
> a bare phrase does work: it then waits `timeout_seconds` for the command. But the
> one-sentence form is the supported way, and it is recognised correctly.

**You only say it once.** After waking, the loop keeps listening without the phrase
for `stay_awake_seconds` (45 by default), and every reply restarts that window — so a
back-and-forth needs the phrase only at the start. The window measures how long *you*
have been quiet, not how long Claude spent talking, so a long answer cannot close the
conversation out from under you. When it does lapse it says so.

Matching is fuzzy with a floor, and the phrase must appear within the first
**2 words** — so "please could you hey claude do this" is someone talking *about* the
assistant, not *to* it, and is ignored. `wake-word check` prints exactly what the
current setting accepts:

```
  wake  1.00  'hey claude, switch to a target'
  wake  0.84  'hey cloud list targets'
  wake  0.67  'Hey Lord'
    -   0.53  'hey there what is this'
    -   0.22  'Create a login endpoint'
```

`sensitivity` is the **minimum match ratio — higher is stricter**. The default `0.72`
admits the common mishearings; raise it to ~0.75 to require a clean match and accept
more misses. Measured ambient speech has scored as high as 0.57, so the margin is not
enormous: raise it if you get false activations.

Nothing reaches Claude unless the phrase matched. `cooldown_seconds` stops one
utterance triggering twice. Echo from the assistant's own voice needs no special
handling — the capture microphone is closed while speech synthesis runs (barge-in, below,
listens during speech through a separate, echo-cancelled monitor that never transcribes).

**Wake mode records in bursts.** While idle it repeatedly records short utterances and
transcribes them on-device to listen for the phrase. Nothing is stored and nothing
leaves the machine, but it is recording; that is why `wake_word.enabled` defaults to
`false`.

### Barge-in

Talk over a reply to stop it: speech stops, the full reply is still printed, and the
loop listens to what you are saying. Works on laptop speakers, no headphones needed.

```sh
voice-claude voice --continuous --barge-in   # or set voice.barge_in: true
voice-claude barge-in check                  # stay quiet: does a reply interrupt itself?
voice-claude barge-in check --talk           # talk over the sample: is your voice enough?
```

**How.** When speaking starts, a separate helper process opens the microphone with
Apple's voice processing — the echo canceller FaceTime uses, which subtracts whatever the
Mac is playing. Measured on this Mac's speakers, it cuts the assistant's own voice from
rms 0.041 to about 0.002. When the level stays above `voice.barge_in_threshold` (0.03)
for `voice.barge_in_min_speech_ms` (300ms), the monitor reports it, speech is stopped
and the monitor exits, releasing the microphone for the ordinary capture. The first 0.4s
is ignored because voice processing emits a loud transient as it starts.

**Limits.**

- Hands-free mode only; push-to-talk already has a key for this.
- The first ~0.5s of what you say goes to stopping the reply, so start with a short
  lead-in ("wait —", "stop —") rather than the important words.
- If echo cancellation is unavailable the monitor refuses to run and replies simply play
  to the end — without cancellation, the reply would interrupt itself every time.
- Voice processing normally turns down ("ducks") everything else the Mac plays, which
  made replies go quiet. The monitor asks for the least ducking; replies may still be
  slightly quieter while they play.
- Loud background speech (a TV, other people) above the threshold will interrupt too.
  Raise the threshold, or turn barge-in off with `--no-barge-in`.

**Privacy.** The monitor computes loudness only. It never transcribes, never records,
and its reports contain numbers alone.

### Being understood

Three things measured through the microphone on this Mac:

* **The recogniser is told your target names and the wake phrase**
  (`stt.contextual_strings`, plus whatever the loop adds). "Switch to listurad" came
  back as "Switch to Instagram" without it.
* **Short commands are matched loosely.** "list targets" arrives as "please target",
  "targets batao" as "target bat". Anything within 0.82 of a command counts as one,
  but only for four words or fewer: measured, mishearings of a command score 0.87-1.00
  while ordinary short instructions reach at most 0.78.
* **The wake phrase has aliases.** "hey claude" comes back as "hey lord" or "hey
  cloud"; those are listed in `wake_word.aliases` and matched near-exactly, which is
  safer than a loose threshold — at `0.65`, "the code looks fine" scored 0.67 and woke it.

Commands work in both languages: *"list targets"* / *"targets batao"*, *"switch to
listurad"* / *"listurad pe jao"*, *"stop listening"* / *"band karo"*. A bare **"stop"**
is deliberately **not** a command: with barge-in it is how you interrupt a reply.

### Targets

```bash
voice-claude targets --conversations
```

A target is a name and a workspace directory. Claude Code keeps conversations per
directory, so which target a turn goes to decides which conversation it joins.

```yaml
voice:
  targets:
    voice: ~/Documents/claude-voice
    listurad: ~/Documents/listurad/listurad
  default_target: voice
```

Say **"switch to listurad"** to change target mid-session, or **"list targets"** to
hear what is configured. Switching and stopping never reach Claude — they are local.

A misheard target name is reported rather than guessed at. `"switch to …"` with a name
that matches nothing prints the available targets instead of sending your words to
whichever project happened to be current. Ordinary instructions are never mistaken for
commands: `"use a dictionary here"` and `"open the README"` go to Claude untouched,
because `use` and `open` only switch target when the name really matches one.

> **These are not the chats open in your IDE panel.** Antigravity runs each panel chat
> as its own `claude` process over pipes the extension owns, and the extension exposes
> no command, URI handler or API that accepts prompt text — I checked version 2.1.267.
> A target is a *parallel* voice channel into the same project on the same account, not
> a remote control for a panel.

### Architecture

[`pipeline.py`](src/voice_claude/pipeline.py) is the only module that knows about all
three subsystems. The dependency direction is one-way:

```
VoicePipeline
   ├── stt     (Dictation, PushToTalkLoop)
   ├── bridge  (PromptSender)
   └── tts     (Speaker)
```

`stt`, `bridge` and `tts` import none of each other — there is a test asserting that
by parsing imports, so the check cannot be fooled by a docstring cross-reference. The
pipeline reuses the existing push-to-talk capture rather than opening the microphone
itself: it takes anything with a `capture()` method, which `PushToTalkLoop` already
satisfies.

### Behaviour at the edges

* **Nothing heard** — Claude is never called.
* **Claude returns nothing** — TTS is never called.
* **Speaking fails** — the turn still counts as successful: Claude answered, the reply
  is printed, and the reason it was not spoken is shown underneath.
* **Errors are never spoken.** Only the user-facing reply reaches TTS — no session id,
  timings, command line or traceback.
* **Cancellation** (`esc`) propagates to whichever stage is running: the recording, the
  `claude` process, or `say`. A watcher thread handles the cancel key while the main
  thread is blocked in Claude or speech.
* **Duplicate protection** is the bridge's existing guard, reused — saying the same
  thing twice does not submit twice.

### Privacy

Unchanged from earlier milestones and enforced by tests: neither the transcript nor
Claude's reply reaches the log file unless the relevant `log_*` option is enabled.
Printing to the terminal is not logging. Recordings are still never written to disk.

| Key | Default | Notes |
| --- | --- | --- |
| `voice.speak_responses` | `true` | speak Claude's reply |
| `voice.show_transcript` | `true` | print the transcript (terminal only) |
| `voice.show_response` | `true` | print the reply; this survives a TTS failure |
| `voice.max_spoken_chars` | `0` | speak at most N chars, cut at a sentence; 0 = all |
| `voice.continuous` | `false` | hands-free; no keypress between turns |
| `voice.utterance_silence_s` | `1.0` | pause that ends an utterance |
| `voice.listen_timeout_s` | `12.0` | give up if nobody speaks |
| `voice.settle_after_speech_s` | `0.2` | pause before reopening the mic |
| `voice.targets` | `{}` | `name: /path` workspaces |
| `voice.default_target` | `""` | which target to start on |
| `wake_word.enabled` | `false` | stay idle until addressed |
| `wake_word.phrase` | `hey claude` | |
| `wake_word.sensitivity` | `0.72` | minimum match ratio; higher is stricter |
| `wake_word.aliases` | `hey lord`, `hey cloud`, `hey clod` | known mishearings, matched near-exactly |
| `wake_word.timeout_seconds` | `8.0` | wait for a command after a bare phrase |
| `wake_word.cooldown_seconds` | `1.5` | ignore the phrase for this long after a turn |
| `wake_word.stay_awake_seconds` | `45.0` | keep listening without the phrase; 0 = every time |

## The Claude Code bridge

```bash
voice-claude claude check
voice-claude claude send "Create a simple Python hello-world function."
voice-claude claude send --dry-run "..."   # print the command, run nothing
voice-claude claude send --force "..."     # resend an identical prompt
```

`send` prints an unambiguous verdict — `SUBMITTED`, `NOT SUBMITTED`, or `DRY RUN` —
and the exit code matches it (0 submitted or dry run, 1 failed, 2 duplicate).

### What it connects to, and what it does not

Submissions run the **existing `claude` binary** with the documented `-p/--print`
flag, from the workspace directory. That reuses the installation and the
`claude.ai` login already on this machine: no API key, no second agent, no bypassed
authentication.

It is a **separate CLI invocation, not the conversation open in the IDE panel.**
Antigravity runs Claude Code through the `anthropic.claude-code` extension, which
spawns `claude --output-format stream-json` as its own child and talks to it over
pipes the extension owns. Nothing outside the IDE can write to those pipes, and the
extension declares no command, URI handler or exported API that accepts prompt text —
its nearest commands only *focus* the input box or insert an @-file reference. The
`~/.claude/ide/<port>.lock` WebSocket runs the other way: the CLI connects to the IDE
to fetch selections and show diffs. Driving the panel would mean reverse-engineering
a private protocol, so this bridge does not attempt it.

**The voice loop keeps that process running.** Instead of one `claude -p` per prompt,
`voice-claude voice` starts one `claude -p --input-format stream-json` per target and
writes each prompt to it — the same binary, login, flags and approval rules, without
paying Claude Code's start-up on every turn. Cancelling interrupts the reply but keeps
the conversation. One-shot commands such as `claude send` still run once per prompt.

**Turns within a run share a conversation.** Each one resumes the previous turn's
session, so a follow-up like "now give me an example" knows what was just discussed.
A new run starts fresh. Set `claude.remember_context: false` to make every turn
stateless.

Set `claude.continue_conversation: true` to *also* pick up the workspace's most recent
existing conversation on the first turn. It is always paired with
`--fork-session`, so a conversation the IDE panel still has open cannot be written to
by two processes at once.

### When Claude needs permission

A non-interactive Claude Code run has nobody to show a permission prompt to, so a tool that needs approval is
refused and Claude can only *describe* what it would have done. Rather than leaving you
with "the prompt was declined", the refusal comes back as data and is offered to you:

```
  claude needs permission to:
      Write  /Users/you/project/tictactoe.py  (7362 chars)
  [y] allow and finish     [any other key] skip
```

Pressing `y` **resumes the same conversation** with file edits allowed, so Claude
carries on from where it stopped instead of reasoning the whole task out again. For the
typed path, `voice-claude claude send --approve-edits "..."` does the same without the
keypress.

Approval grants **file edits only** — shell commands stay blocked, and there is a test
asserting the approval never passes `bypassPermissions` or
`--dangerously-skip-permissions`. Nothing is approved automatically.

To skip the prompt entirely and always allow edits, set `claude.permission_mode:
acceptEdits`.

### Safety

* The normal Claude Code approval flow is preserved. `permission_mode` defaults to
  empty (Claude Code's own default) and the bridge never passes
  `--dangerously-skip-permissions`. A prompt asking for a file write comes back
  saying it needs permission, rather than writing it.
* Prompts are logged by shape — length, word count, a 12-character digest — never by
  content, unless `claude.log_prompts` is explicitly enabled.
* An identical prompt resubmitted within `claude.duplicate_window_s` (120s) is
  suppressed. The record lives in the state directory and holds only a digest and a
  timestamp, so a double-tap is caught even across separate CLI invocations.
* Credentials never pass through the bridge. `claude check` reports the account label
  and auth method; no token is read, stored or printed.

### Bridge configuration

| Key | Default | Notes |
| --- | --- | --- |
| `claude.mode` | `cli` | `cli` or `fake` (tests) |
| `claude.cli_path` | `claude` | resolved on PATH, then `~/.local/bin`, `~/.claude/local`, … |
| `claude.workspace` | `null` | `null` = current directory |
| `claude.timeout_s` | `120.0` | |
| `claude.continue_conversation` | `false` | adds `--continue --fork-session` |
| `claude.remember_context` | `true` | one run's turns share a conversation |
| `claude.permission_mode` | `""` | empty = Claude Code's default approval flow |
| `claude.model` | `""` | empty = whatever Claude Code is configured to use |
| `claude.session` | `persistent` | keep one `claude` running per target (voice loop); `per-turn` runs one per prompt |
| `claude.load_mcp_servers` | `false` | load MCP servers / claude.ai connectors (~2.2s per start) |
| `claude.append_system_prompt` | voice style | instructions for spoken replies; `""` sends none |
| `claude.dry_run` | `false` | also available as `--dry-run` |
| `claude.log_prompts` | `false` | privacy |
| `claude.max_prompt_chars` | `8000` | |
| `claude.duplicate_window_s` | `120.0` | `0` disables |

## Text-to-speech

```bash
voice-claude tts check
voice-claude tts voices --language en
voice-claude tts speak "Hello, this is Voice Claude."
voice-claude tts speak --voice Rishi --rate 200 "Speaking a little faster."
voice-claude tts stop
```

`speak` prints `COMPLETED`, `CANCELLED`, `NOT SPOKEN` or `FAILED`, and the exit code
matches (0 completed, 1 failed or skipped, 2 cancelled).

### Mechanism

The macOS **`say`** command, run as an argument array through `subprocess` — never a
shell, and `shell=True` appears nowhere in the package (there is a test asserting
that). Entirely local: no network, no account, no key, no cost.

Text is written to `say`'s **stdin** (`say -f -`) rather than passed as an argument.
That removes the `ARG_MAX` ceiling on a long Claude response, and means text
beginning with `-` can never be read as an option.

`say` also interprets `[[...]]` as embedded synthesiser commands. A Claude response
is arbitrary text, so those sequences are defused before they reach it.

### Voices

Nothing is hard-coded: the voice list comes from `say -v '?'` at runtime. `tts.voice`
defaults to **empty**, meaning the system default — voice names differ between Macs,
so assuming one would be wrong. If a configured voice is not installed, `tts check`
and `tts speak` fail clearly, name the missing voice, and suggest alternatives.

### Long responses

Text longer than `tts.max_chunk_chars` (1200) is split on sentence boundaries —
including the Devanagari danda `।` — falling back to clause, then word, then a hard
cut. Chunks are spoken in order. This also gives cancellation somewhere to take
effect: `tts stop` ends the chunk currently speaking and abandons those queued behind
it. This is deliberately not streaming.

`tts stop` works across processes: the running `say` pid is recorded in the state
directory. The pid is verified to still *be* a `say` process before any signal is
sent, because pids get recycled and signalling a stranger would be worse than failing
to stop.

### Configuration

| Key | Default | Notes |
| --- | --- | --- |
| `tts.enabled` | `true` | |
| `tts.engine` | `apple-say` | `apple-say` or `fake` (tests) |
| `tts.voice` | `""` | empty = system default; see `tts voices` |
| `tts.rate_wpm` | `0` | `say -r`; 0 = the voice's own rate |
| `tts.volume` | `1.0` | `(0, 1]`, via the `[[volm]]` command |
| `tts.max_chunk_chars` | `1200` | sentence-aware split point |
| `tts.timeout_s` | `300.0` | per-chunk ceiling |
| `tts.log_text` | `false` | privacy |

### Privacy

Spoken text is a Claude response, so it is logged by shape — character count, word
count, chunk count, duration — and never by content unless `tts.log_text` is
explicitly enabled. No audio is written to disk, nothing is sent anywhere, and no
credential is read.

## Layout

```
voice-claude/
├── config/
│   ├── default.yaml           # checked-in project defaults
│   └── local.yaml.example     # template for untracked machine overrides
├── scripts/
│   ├── setup.sh               # create .venv and install
│   └── check.sh               # ruff + mypy + pytest
├── src/voice_claude/
│   ├── cli.py                 # argparse entry point
│   ├── config/
│   │   ├── schema.py          # typed dataclasses — source of truth for defaults
│   │   ├── loader.py          # layered merge: files -> env -> CLI
│   │   └── paths.py           # macOS directory resolution
│   ├── logging_setup/
│   │   ├── setup.py           # handler wiring, idempotent configure()
│   │   ├── formatters.py      # JSON and console formatters
│   │   └── context.py         # per-interaction session ids
│   ├── audio/format.py        # the PCM format capture delivers
│   ├── stt/
│   │   ├── base.py            # SpeechEngine / SpeechSession protocols
│   │   ├── service.py         # Dictation: logging + privacy policy
│   │   ├── interactive.py     # push-to-talk state machine
│   │   ├── fake.py            # scripted engine used by the tests
│   │   └── apple/
│   │       ├── bundle.py      # builds the .app macOS privacy requires
│   │       ├── helper.py      # runs inside it; the only AVFoundation code
│   │       ├── engine.py      # drives the helper from the CLI process
│   │       └── protocol.py    # the JSON-lines IPC both sides share
│   ├── bridge/
│   │   ├── base.py            # ClaudeCodeBridge protocol, DuplicateGuard
│   │   ├── cli_bridge.py      # runs the documented `claude -p`
│   │   ├── service.py         # PromptSender: logging + privacy policy
│   │   └── fake.py            # scripted bridge used by the tests
│   ├── tts/
│   │   ├── base.py            # TTSService protocol, outcomes, Voice
│   │   ├── apple_say.py       # the macOS `say` backend
│   │   ├── chunking.py        # sentence-aware splitting
│   │   ├── service.py         # Speaker: logging + privacy policy
│   │   └── fake.py            # scripted backend used by the tests
│   ├── pipeline.py            # VoicePipeline: STT -> Claude -> TTS
│   ├── targets.py             # named workspaces a turn can go to
│   ├── intent.py              # instruction, or a command to the loop?
│   ├── wake.py                # wake-phrase matching and the capture gate
│   ├── ptt.py                 # key sources and bindings
│   └── utils/errors.py        # exception hierarchy
└── tests/
```

## Configuration

Layers are merged key by key, lowest precedence first:

1. dataclass defaults in `src/voice_claude/config/schema.py`
2. `config/default.yaml`
3. `~/Library/Application Support/voice-claude/config.yaml`
4. `config/local.yaml` (untracked)
5. `VOICE_CLAUDE__<SECTION>__<KEY>` environment variables
6. command-line flags

So a nested key is set from the environment like this:

```bash
VOICE_CLAUDE__LOGGING__CONSOLE__LEVEL=DEBUG voice-claude doctor
```

Values are parsed as YAML scalars, so `true`, `5` and `null` arrive with the right
type. Unknown keys are rejected rather than ignored, so a typo fails loudly.

`config/default.yaml` mirrors the dataclass defaults; a test asserts the two stay
in sync. The `tts`, `wake_word`, `claude` and `ide` sections are declared so the file
layout is stable — nothing reads them yet.

### Keys that matter for speech

| Key | Default | Notes |
| --- | --- | --- |
| `stt.engine` | `apple` | `apple` or `fake` (tests) |
| `stt.locale` | `en-IN` | on-device; handles English + Hinglish |
| `stt.fallback_locales` | `[en-US]` | tried when `locale` is unavailable |
| `stt.prefer_on_device` | `true` | keeps audio on the machine |
| `stt.require_on_device` | `false` | fail rather than use Apple's servers |
| `stt.partial_results` | `true` | stream interim hypotheses |
| `stt.log_transcripts` | `false` | privacy — see below |
| `stt.push_to_talk.start_stop_key` | `space` | any single char, or `enter`/`esc`/`tab` |
| `stt.push_to_talk.cancel_key` | `esc` | |
| `stt.push_to_talk.quit_key` | `q` | Ctrl-C and Ctrl-D always quit too |
| `audio.sample_rate` | `16000` | converted from the device's native rate |
| `audio.channels` | `1` | |
| `audio.sample_format` | `int16` | or `float32` |
| `audio.max_duration_s` | `60.0` | capture stops itself at this ceiling |
| `audio.silence_rms_threshold` | `0.004` | dead-microphone floor (~ -48 dBFS), not a VAD |
| `audio.min_recording_s` | `0.35` | ignores a stop arriving before this much audio exists |
| `audio.save_recordings` | `false` | privacy — see below |

Rebinding push-to-talk needs no code change:

```bash
VOICE_CLAUDE__STT__PUSH_TO_TALK__START_STOP_KEY=enter voice-claude stt listen
```

### Directories

Standard macOS locations, via `platformdirs`:

| Purpose | Path |
| --- | --- |
| config | `~/Library/Application Support/voice-claude/` |
| data / state | `~/Library/Application Support/voice-claude/` |
| logs | `~/Library/Logs/voice-claude/` |

Set `VOICE_CLAUDE_HOME=/some/dir` to point all of them at one sandbox root — the
test suite uses this so it never touches your real directories.

## Logging

`logging_setup.configure()` installs two handlers on the root logger:

- **console** (stderr) — human-readable, colourised through `rich` when attached to
  a TTY, `INFO` by default
- **file** — rotating, `DEBUG` by default, one JSON object per line at
  `~/Library/Logs/voice-claude/voice-claude.log`

Use it from any module:

```python
from voice_claude.logging_setup import get_logger, session

log = get_logger(__name__)

with session() as session_id:
    log.info("transcribed utterance", extra={"words": 12})
```

Every record carries a `session_id`, so one voice interaction can be pulled out of
an interleaved log later. `extra=` fields are preserved as top-level JSON keys.
`configure()` is idempotent and only removes handlers it installed itself.

Dictation emits `recording started`, `recording stopped`, `transcription started`,
`transcription completed` (with `transcription_latency_s`, `total_latency_s`,
`audio_duration_s`, `words`, `chars`) and errors — never the transcript itself
unless `stt.log_transcripts` is on.

## In your editor

```bash
cd extension && npm install && npm run compile   # then press F5 in VS Code to try it
npx vsce package                                 # a .vsix anyone can install
```

`extension/` is a VS Code extension that works in **VS Code, Cursor, Windsurf and
Antigravity**: a panel with a microphone button, your words as text, Claude's reply
shown and spoken, and a project picker. It drives the same voice turns as the terminal
through `voice-claude serve`, a JSON-lines service:

```bash
echo '{"command":"ask","text":"what is a list","speak":false}' | voice-claude serve
```

Commands are `listen`, `ask`, `cancel`, `stop_speaking`, `target`, `targets`, `status`
and `quit`; events carry the transcript, each tool Claude starts using, the reply, and
what a turn cost. See `src/voice_claude/serve.py`.

The extension cannot type into *another* extension's chat box — Antigravity's Claude
panel included — which is why it brings its own panel rather than driving that one.

## Checking it live

```bash
python scripts/live_check.py            # all groups, ~4 minutes
python scripts/live_check.py --only turn
```

Everything else here runs against fakes. This one speaks each phrase **through the
speakers**, hears it with the **microphone**, recognises it on-device and — for the
conversation checks — sends it to Claude Code and reads the answer back: speech,
spoken commands in English and Hindi, the wake phrase, whole turns, and a permission
request answered out loud. It restores your volume afterwards.

Two things it cannot do: a synthesised voice is not your voice (a poor transcript here
is not proof that your speech fails), and barge-in is invisible to it, because the echo
canceller removes exactly what this script plays. See `docs/live-test-checklist.md`.

## Development

```bash
./scripts/check.sh    # ruff, ruff format, mypy --strict, pytest
```
