Metadata-Version: 2.4
Name: avrvx
Version: 0.4.0
Summary: Give your LLMs a voice. Dictate, speak, and converse with any OpenAI-compatible LLM from a global hotkey on Linux.
Author: Carl E. Hartung
License-Expression: MIT
Project-URL: Homepage, https://github.com/avrvx/AverVOX-OSS
Project-URL: Repository, https://github.com/avrvx/AverVOX-OSS
Project-URL: Issues, https://github.com/avrvx/AverVOX-OSS/issues
Keywords: speech,dictation,tts,stt,voice,whisper,piper,desktop
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: X11 Applications :: GTK
Classifier: Intended Audience :: End Users/Desktop
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: sounddevice>=0.4.6
Requires-Dist: webrtcvad>=2.0.10
Requires-Dist: faster-whisper>=1.0.0
Requires-Dist: piper-tts>=1.2.0
Requires-Dist: httpx>=0.27
Requires-Dist: pynput>=1.7.6
Requires-Dist: PyYAML>=6.0
Requires-Dist: numpy>=1.24.0
Dynamic: license-file

# AverVOX OSS - Give your LLMs a voice.
Website Edition
version: 0.4.0

Add voice to any OpenAI-compatible endpoint, local or remote. Any app with focus can receive your speech as text. Select text to have it read aloud. Hold free-form voice conversations with Large Language Models (LLMs) on Linux.

## Key Features

- **Local-only voice processing** - Both Speech-to-Text (STT) and Text-to-Speech (TTS) run on your own machine. The only network traffic is to the LLM endpoints you configure, which may be:
  - Hosted locally (on the same computer or another device on your LAN), **or**
  - Accessed remotely via any inference provider you choose (OpenAI, LocalAI, vLLM, SGLang, LM Studio, Ollama, Hermes Agent, OpenClaw, etc.).
- **Tray app + command-line interface** - A system-tray GUI for everyday use and a CLI tool (`avrvx`) for scripting and automation.
- **Speech engine for other apps** - `avrvx --synthesize`, `--transcribe`, and `--capabilities` let tools such as Hermes Agent and OpenClaw hand their text and recordings to AverVOX instead of a cloud speech API.
- **Piper TTS** - Fast, low-resource local speech synthesis.
- **Multiple LLM profiles** - Define named endpoints and switch between them instantly via the tray menu.
- **XDG-compliant configuration** - All settings are stored in YAML files under `~/.config/avervox/`; no extra dotfiles clutter your home directory.

## Free vs Pro

| Feature | Free | Pro |
|---------|:----:|:---:|
| Speech-to-Text (`Ctrl+Alt+Space`) | Yes | Yes |
| Text-to-Speech (`Ctrl+Alt+S`) | Yes | Yes |
| Converse (`Ctrl+Alt+C`) | Yes | Yes |
| CLI (`avrvx --listen`, `--speak`) | Yes | Yes |
| Bridge CLI (`--synthesize`, `--transcribe`, `--capabilities`) | Yes | Yes |
| Piper TTS | Yes | Yes |
| faster-whisper STT | Yes | Yes |
| Voice interrupt | Yes | Yes |
| Conversation HUD | Yes | Yes |
| Streaming TTS | Yes | Yes |
| Kokoro TTS | | Yes |
| TTS speed control | | Yes |
| [Custom wake word](https://openwakeword.com/) | | Yes |
| System prompts (per profile) | | Yes |
| Session memory (survives restart) | | Yes |
| LAN client/server (`avrvx --serve`) | | Yes |
| Dashboard (endpoints, models, profiles) | | Yes |

AverVOX OSS is free and open-source. For issues and questions, please open an issue on the project repository. If you want the Pro-only features above (Kokoro TTS, wake word, session memory, LAN, and more), see [AverVOX Pro](https://avervoxpro.com/) - a one-time, no-subscription companion app, distributed separately (not on GitHub or PyPI).

---

## Installation

```bash
git clone https://github.com/avrvx/AverVOX-OSS.git
cd AverVOX-OSS
bash install.sh
```

This will:

- Install the required system packages (PyGObject, PortAudio, xdotool, xclip)
- Create a Python virtual environment under `~/.local/share/avervox/`
- Install all required Python packages
- Download the Piper voice model
- Write an `avrvx` launcher to `~/.local/bin/`
- Add desktop menu and autostart entries

Alternatively, `pip install avervox` installs the package; you still need the system dependencies (GTK, xdotool, xclip, portaudio) - see `install.sh` for the full list.

**New to AverVOX OSS?** See [QUICK_START-OSS.md](QUICK_START-OSS.md) for a guided tour of the tray app and every Settings tab.

---

## Interaction Modes

| Mode | Shortcut / Command | What it does |
|------|-------------------|--------------|
| **Speech-to-Text** | `Ctrl+Alt+Space` | Press once to start dictation, speak naturally, press again to stop. Text is inserted as you pause. |
| **Text-to-Speech** | `Ctrl+Alt+S` | Select any text, hit the shortcut, and AverVOX OSS reads it aloud using Piper. |
| **Converse** | `Ctrl+Alt+C` | Hold a conversation with your chosen LLM. |
| **CLI** | `avrvx ...` | Headless commands for scripting (see below). |

---

## Setup

1. Local: Start an OpenAI-compatible LLM server such as LM Studio or Ollama and confirm its API base URL.
2. Remote: Can be used with any inference provider using an OpenAI-compatible API.
3. Right-click the AverVOX OSS tray icon and open **Settings...**
4. Add a profile with a **Label** and **API base URL**; click the circular Test Connection button and select a **Model**.
5. Optionally set a **Session header** if the endpoint supports session-aware conversations.
6. Save, then press `Ctrl+Alt+C`.

---

## Usage

### GUI (system-tray) mode
```bash
avrvx
```
- Starts a tray icon with hotkeys active.
- A desktop notification confirms the app is ready ("AverVOX OSS - Ready - hotkeys active").
- No main window appears; all interaction is via shortcuts or the tray menu.

**Tray-icon right-click menu**

- **LLM:** Shows the active profile; click to switch profiles.
- **Reload config:** Re-reads `config.yaml` without restarting.
- **Settings...** Opens the full settings dialog.
- **Copy Last Response** Copies the most recent LLM reply to the clipboard.
- **Open Log** Opens `avervox.log` in your default viewer.
- **About AverVOX OSS** Shows version, tagline, and links.
- **Quit AverVOX OSS** Exits the program.

### CLI
```bash
# Record speech (VAD auto-stop) and print transcript
avrvx --listen

# Speak literal text
avrvx --speak "Hello world"

# Pipe text from another command
echo "The quick brown fox" | avrvx --speak

# Show version and exit
avrvx --version

# Combine with any tool
avrvx --listen | my-llm-tool | avrvx --speak
```

### Bridge CLI (for other applications)

Three commands let other programs use AverVOX as their local speech engine without embedding anything. Unlike `--speak` and `--listen`, they exchange files and JSON instead of driving your speakers and microphone directly.

```bash
# Synthesize to a WAV file instead of playing it
avrvx --synthesize --text "Deployment complete" --output /tmp/reply.wav
avrvx --synthesize --text-file /tmp/reply.txt --output /tmp/reply.wav
echo "Deployment complete" | avrvx --synthesize --text - --output /tmp/reply.wav

# Transcribe an existing recording or voice message
avrvx --transcribe /tmp/voice-message.ogg

# Ask what this install can do (JSON on stdout)
avrvx --capabilities
```

Notes:

- `--synthesize` writes a mono 16-bit PCM WAV at the engine's sample rate, prints the path on stdout, and creates the file mode `0600`. The output directory must already exist.
- Prefer `--text-file` or `--text -` (stdin) over `--text "…"` for anything sensitive: arguments are visible to other users in the process list.
- `--transcribe` prints the transcript on stdout and exits non-zero when no speech was found. Anything ffmpeg can decode works.
- `--capabilities` prints a JSON object on stdout (diagnostics stay on stderr, so it is always safe to pipe into a parser) reporting `edition`, `licensed`, `version`, the available TTS engines, the STT model, and a `features` map. Host integrations use it to detect OSS versus Pro instead of shipping separate builds.

Ready-made integration packages for Hermes Agent, OpenClaw, and Odysseus live in `integrations/` in the source tree; each has its own README.

### Scripting your assistant
AverVOX OSS is built for terminal pipelines. Example workflows:

```bash
echo "Build succeeded" | avrvx --speak
avrvx --speak "Deployment complete"
avrvx --listen                             # capture speech, output transcript
avrvx --listen | my-tool | avrvx --speak   # pipe through an LLM or other processor
```
Running `avrvx` with no flags launches the system-tray app described above.

---

## Converse

- **How to start** - Press **Ctrl+Alt+C**.
- **HUD pill** - A colour-coded pill indicator shows the current state:
  - **Red** = Listening
  - **Orange** = Processing
  - **Green** = Streaming response
- **Interaction flow** - Speech -> STT -> LLM -> streaming TTS. Playback begins as soon as the first sentence is ready and continues with minimal gaps while the model finishes generating. After each reply AverVOX OSS automatically listens for the next turn.
- **Ending a conversation** - any of these will stop the loop:
  1. Say a goodbye phrase (e.g., "talk to you later", "goodbye", "that's all"). Customisable in Settings or `converse.goodbye_phrases`.
  2. Remain silent for the configured *silence timeout* (default 7 s, `converse.silence_timeout_ms`).
  3. Press **Ctrl+Alt+C** again.
- **Voice interrupt (barge-in)** - When enabled in Settings, you can interrupt the assistant mid-response simply by speaking; AverVOX OSS stops playback immediately and listens for your next turn. This requires headphones - without them, the TTS audio feeds back into the microphone and triggers false interrupts.
- **Markdown stripping** - LLM output is cleaned of markup (headings, bold, code blocks, links, etc.) before it is spoken, so the audio sounds natural rather than reading out syntax.
- **Echo prevention** - The microphone is muted while TTS is playing, and a configurable re-arm delay (default 250 ms, `converse.rearm_delay_ms`) is applied before the mic reopens. Headphones are recommended.
- **Session tracking** - If your LLM endpoint supports session-aware conversations, set the *Session header* field in the LLM profile (e.g., `X-Hermes-Session-Id`). AverVOX OSS sends a UUID with each request; if the server returns a different ID it adopts that for the rest of the app session. OSS supports session headers for the current app session; persistent sessions that survive restarts are a Pro feature.
- **State machine**

```
IDLE -> LISTENING <-> TRANSCRIBING -> CONVERSING -> SPEAKING -> (rearm delay) (loop)
       `- silence timeout or goodbye phrase -> IDLE
```

The microphone stays off during the entire STT -> LLM -> TTS pipeline; the configurable re-arm delay reduces speaker-to-mic feedback before listening resumes.

---

## How It Works

AverVOX OSS is built around a simple pipeline: your voice goes in, text comes out (or vice versa), with an LLM in the middle if you want one. Here's what's doing the work under the hood.

**Voice I/O** uses `parec` + `sounddevice` for audio capture and playback, with `webrtcvad` handling voice activity detection. STT is handled by `faster-whisper`; TTS by Piper (an ONNX model, ~16 MB, very fast). Sentences are streamed to the TTS engine as they arrive from the LLM, so playback starts almost immediately rather than waiting for a full response.

**LLM connectivity** goes through `httpx` to any `/v1`-compatible endpoint - LM Studio, Ollama, Hermes Agent, OpenClaw, OpenAI, whatever you're running. Responses stream via SSE. Markdown is stripped before speech so you don't hear "asterisk asterisk bold asterisk asterisk."

**Activation** is handled by `pynput` for global hotkeys. Both X11 and Wayland (via XWayland) are supported. Text selection uses `xclip`; text injection uses `xdotool` or the clipboard depending on the target app.

### Architecture

```
Activation layer
  `- Hotkeys (pynput)   -> Ctrl+Alt+Space  = Speech-to-Text
                        -> Ctrl+Alt+S      = Text-to-Speech
                        -> Ctrl+Alt+C      = Converse

Services layer
  - SpeechService   - STT (faster-whisper)
  - InsertService   - xdotool / clipboard text injection
  - LLMService      - httpx -> any OpenAI-compatible API (streaming SSE)

Core components
  - TTS engine: Piper (ONNX, ~16 MB)
  - Audio I/O: parec + sounddevice + webrtcvad
  - Selection provider: xclip
```

### Source layout

```
src/avervox/
|-- __init__.py          # package metadata
|-- __main__.py          # CLI entry point (--listen, --speak, --synthesize, --transcribe, --capabilities, --version, or GUI)
|-- main.py              # GUI controller (state machine, hotkey handlers, notifications)
|-- config.py            # configuration loading, LLM profiles, dataclasses
|-- audio.py             # microphone capture + VAD/recorder + interrupt monitor
|-- stt.py               # speech-to-text (faster-whisper)
|-- tts.py               # text-to-speech engine (Piper), markdown stripping
|-- inserter.py          # text insertion + selection grabbing
|-- hotkeys.py           # global hotkey manager (pynput)
|-- tray.py              # system tray icon, profile submenu, copy/log/about menu items
|-- logger.py            # logging setup
|-- settings.py          # GTK settings dialog (tabbed: hotkeys, LLM, TTS, dictate, converse, advanced)
|-- hud.py               # conversation-mode HUD overlay
|-- llm.py               # OpenAI-compatible HTTP client (streaming SSE)
`-- services/
    |-- __init__.py      # service factory (create_services)
    |-- base.py          # Protocol definitions (SpeechService, InsertService, LLMService)
    |-- local.py         # local implementations wrapping existing modules
    `-- direct.py        # DirectLLMService (calls LLM API directly, with streaming)
```

---

## Configuration

`~/.config/avervox/config.yaml` (created on first run):

```yaml
hotkeys:
  listen: "<ctrl>+<alt>+space"       # Settings -> Hotkeys
  speak_selection: "<ctrl>+<alt>+s"
  converse: "<ctrl>+<alt>+c"

stt:
  model: base        # tiny, base, small, medium, large-v3
  language: en
  device: auto       # auto | cpu | cuda

tts:
  voice_model: ~/.local/share/piper-tts/voices/en_US-lessac-high.onnx

audio:
  vad_aggressiveness: 2       # 0-3 (higher = more aggressive silence detection)

dictate:
  interim_pause_ms: 1000      # pause before inserting dictation text (Settings -> Dictate)

backends:
  text_inserter: xdotool      # xdotool | ydotool - Settings -> Advanced
  selection_provider: xclip   # xclip | xsel | wl-paste

converse:
  end_of_turn_ms: 1100              # pause after you stop speaking (Settings -> Converse)
  silence_timeout_ms: 7000          # silence before ending conversation (ms)
  rearm_delay_ms: 250               # pause after TTS before mic reopens (ms; skipped with headphones)
  early_listen_ms: 300              # pre-open mic before TTS ends (headphones only)
  goodbye_phrases:                  # phrases that end the conversation
    - "talk to you later"
    - "goodbye"
    - "bye bye"
    - "see you later"
    - "that's all"
    - "good night"
    - "i'm done"
  interrupt_enabled: false          # voice interrupt (barge-in) - requires headphones
  interrupt_headphones_confirmed: false

llm:
  active: my-server
  profiles:
    my-server:
      label: "LM Studio (local)"
      api_base: "http://localhost:1234/v1"
      api_key: ""              # leave blank for local models
      default_model: ""        # model name returned by /v1/models
      session_header: ""       # HTTP header for session tracking (e.g. X-Hermes-Session-Id)
```

Most users only need to edit the **llm** section, and the Settings dialog handles that. Everything else has sensible defaults. You can add multiple profiles and switch between them from the tray menu.

**Backward compatibility:** Old flat-format `llm:` configs (with `api_base` directly under `llm:`) are automatically migrated to a single profile on first load.

**Secrets:** API keys are masked in Settings (show/hide toggle) and stored encrypted in `config.yaml` as `enc:...` values, bound to your machine. Existing plaintext keys still load until the next save.

---

## System Requirements

- Linux Mint 21/22 or Ubuntu 22.04/24.04 (X11 or XWayland)
- PulseAudio **or** PipeWire
- Python 3.12+

---

## Audio Hardware & Environment Tips

| Factor | Recommendation |
|--------|----------------|
| Microphone | USB mic or boom headset for best accuracy |
| Background noise | Reduce fans, traffic, conversations; use a quieter room |
| Room acoustics | Avoid echoey spaces; consider soft furnishings |
| Mic distance/level | Keep a steady distance and avoid clipping |

Adjust **Settings -> Dictate** and **Settings -> Converse** to match your conditions. For noisy environments, try a larger STT model (e.g., `small`).

---

## Performance Tuning

| Setting | Default | Conservative | Where | Effect |
|---------|---------|-------------|-------|--------|
| `dictate.interim_pause_ms` | **1000** | 1500 | Settings -> Dictate | Dictate: pause before typing an interim chunk. |
| `converse.end_of_turn_ms` | **1100** | 1500 | Settings -> Converse | Converse / `avrvx --listen`: end-of-turn delay. |
| `rearm_delay_ms` | **250** | 500 | `config.yaml` -> `converse` | Pause after TTS before mic reopens (skipped when headphones confirmed). |
| `silence_timeout_ms` | **7000** | 10000 | `config.yaml` -> `converse` | How long to wait with no speech before ending the conversation. |
| STT `beam_size` | **1** | 5 | `stt.py` | Greedy (1) is faster; beam search (5) is more accurate for mumbled or technical speech. |
| STT model | **base** | tiny / small | `config.yaml` -> `stt.model` | `tiny` is fastest, `small`/`medium` more accurate. `base` is a good middle ground. |

**Tips:**

- If Converse turns get clipped (cut off mid-sentence), increase `converse.end_of_turn_ms`.
- If you hear echo (AverVOX OSS responding to its own TTS), increase `rearm_delay_ms`.
- For the fastest possible turns at the cost of some accuracy, use `stt.model: tiny` with `beam_size: 1`.

---

## LLM model health & failure detection

During Converse, AverVOX monitors each LLM stream and reacts when a model misbehaves, so you are not left waiting in silence for a minute or more. These thresholds are **critical for voice UX** — they define how long AverVOX waits before giving up, unloading the model, and switching you to something else.

### What gets checked

| Threshold | Default | Defined in | Triggers when |
|-----------|---------|------------|---------------|
| **First-token timeout** | **30 s** | `src/avervox/llm_control.py` → `FIRST_TOKEN_TIMEOUT_S` | No **content** token arrives within this time after the stream request is sent. Empty SSE keepalive chunks (common with LM Studio) do **not** count as output. |
| **Empty response** | **30 s** | `EMPTY_RESPONSE_MIN_S` | The stream finishes with zero usable content after at least this long. |
| **Stream read stall** | **90 s** | `STREAM_READ_TIMEOUT_S` | No SSE bytes at all arrive for this long (connection hung). |

The first-token timer starts when AverVOX sends the chat/completions request. That elapsed time includes **model load on the server** (e.g. ~11 s for a cold `gpt-oss-120b` on LM Studio before it can respond) **plus** generation time until the first word of the reply. Set the threshold high enough to tolerate your slowest *healthy* cold start, but low enough to catch models that stall (e.g. 70+ s with no speech).

**Example:** A cold `gpt-oss-120b` that loads in ~11 s and speaks its first word at ~16 s will pass a 30 s threshold. A model that sends keepalive SSE for 70 s before any content will fail at 30 s.

### What happens on failure

1. The in-flight request is aborted and the model is unloaded (LM Studio `/api/v1/models/unload`, or Ollama `keep_alive: 0`).
2. The model is added to that endpoint's `disabled_models` in `config.yaml`, with the failure reason recorded.
3. AverVOX speaks a brief notice and suggests other enabled models on the same endpoint.
4. The dashboard greys out the disabled model.

### Re-enabling a disabled model

Disabled models stay off for the **rest of the current app session** (so a bad model is not retried immediately). **The next time you start AverVOX**, all `disabled_models` entries are cleared automatically and every model is available again.

To re-enable during the same session without restarting, edit `~/.config/avervox/config.yaml` and remove the model from the endpoint's `disabled_models` map, then reload config from the tray menu:

```yaml
llm:
  endpoints:
    my-spark:
      disabled_models:
        omnicoder-9b: no usable output within 30s   # delete this entry
```

Only re-enable a model during the same session if you have fixed the underlying issue (VRAM, quantisation, server hang, etc.).

### Tuning the thresholds

These constants are compile-time defaults in `llm_control.py`. If your largest healthy model needs more than 30 s on a cold start, increase `FIRST_TOKEN_TIMEOUT_S` and rebuild/reinstall. If false positives occur on cold loads, raise the value; if bad models leave you waiting too long, lower it.

---

## Troubleshooting

| Symptom | Check / Fix |
|---------|-------------|
| No audio output | Run `avrvx --listen` to verify capture and playback. |
| Hotkeys don't work | Check `journalctl --user -f` for pynput/X11 errors. |
| ALSA/Pulse errors | Make sure PulseAudio or PipeWire is running. |
| Converse fails | Open Settings, verify the test button shows a green checkmark and models are listed; check the log for `LLM error` entries. |
| Model greyed out / "disabled" | A failure threshold tripped — see **LLM model health & failure detection** above; check `disabled_models` in config and the log for `LLM model failure`. |
| Bad transcription | Reduce background noise, move closer to the mic, or switch to a larger STT model (`small`/`medium`). |
| Conversation ends too early | Increase `converse.end_of_turn_ms` or `silence_timeout_ms`; use headphones to limit speaker bleed in loud environments. |

Log file: `~/.local/share/avervox/avervox.log`

---

## Commit attribution

Past commits may show "Cursor Agent" as a co-author due to an unanticipated IDE feature. The code is authored and maintained by Carl E. Hartung. That attribution setting has been turned off going forward.

---

## License

AverVOX OSS is free and open-source software. See the `LICENSE` file for full terms.
