Metadata-Version: 2.4
Name: wyoming_openai
Version: 0.9.0
Summary: OpenAI-Compatible Proxy Middleware for the Wyoming Protocol
Author: Rory Eckel
License-Expression: Apache-2.0
Project-URL: Homepage, https://github.com/roryeckel/wyoming_openai
Project-URL: Issues, https://github.com/roryeckel/wyoming_openai/issues
Keywords: proxy,tts,openai,home-assistant,stt,asr,wyoming
Classifier: Intended Audience :: Developers
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Operating System :: OS Independent
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: openai[realtime]==3.44.0
Requires-Dist: websockets<16,>=15.0.1
Requires-Dist: wyoming==1.11.0
Requires-Dist: yasbd-lib<2.0,>=1.0.5
Provides-Extra: dev
Requires-Dist: ruff==0.17.3; extra == "dev"
Requires-Dist: pytest==9.2.0; extra == "dev"
Requires-Dist: pytest-asyncio==1.4.1.post1; extra == "dev"
Requires-Dist: pytest-mock==3.17.0; extra == "dev"
Requires-Dist: pytest-cov==7.1.0; extra == "dev"
Requires-Dist: pytest-timeout==2.4.0; extra == "dev"
Requires-Dist: pyright==1.1.414; extra == "dev"
Requires-Dist: rapidfuzz==3.14.6; extra == "dev"
Requires-Dist: httpx==0.28.1; extra == "dev"
Dynamic: license-file

# Wyoming OpenAI

OpenAI-Compatible Proxy Middleware for the Wyoming Protocol

[![License](https://img.shields.io/github/license/roryeckel/wyoming_openai.svg)](https://github.com/roryeckel/wyoming_openai/blob/main/LICENSE) [![Python version](https://img.shields.io/pypi/pyversions/wyoming-openai.svg)](https://pypi.org/project/wyoming-openai/) [![GitHub issues](https://img.shields.io/github/issues/roryeckel/wyoming_openai.svg)](https://github.com/roryeckel/wyoming_openai/issues) [![Docker](https://img.shields.io/github/v/release/roryeckel/wyoming_openai?label=ghcr.io&logo=docker&logoColor=white&color=2496ED)](https://github.com/roryeckel/wyoming_openai/pkgs/container/wyoming_openai) [![PyPI version](https://badge.fury.io/py/wyoming-openai.svg)](https://pypi.org/project/wyoming-openai/)

**Author:** Rory Eckel

Note: This project is not affiliated with OpenAI or the Wyoming project.

## Overview

This project introduces a [Wyoming](https://github.com/OHF-Voice/wyoming) server that connects to OpenAI-compatible endpoints of your choice. Like a proxy, it enables Wyoming clients such as the [Home Assistant Wyoming Integration](https://www.home-assistant.io/integrations/wyoming/) to use the transcription (Automatic Speech Recognition - ASR) and text-to-speech synthesis (TTS) capabilities of various OpenAI-compatible projects. By acting as a bridge between the Wyoming protocol and OpenAI, you can consolidate the resource usage on your server and extend the capabilities of Home Assistant. The proxy now provides incremental TTS streaming compatibility by intelligently chunking text at sentence boundaries with [yasbd](https://github.com/speedyk-005/yasbd-lib) for responsive audio delivery. When streaming is enabled, Wyoming OpenAI prefetches several OpenAI synthesis requests in parallel while playing the audio sequentially, keeping latency low without breaking event order.

## Featured Models

This project features a variety of examples for using cutting-edge models in both Speech-to-Text (STT) and Text-to-Speech (TTS) scenarios:

- **`gpt-transcribe`**: OpenAI's recommended model for accurate transcription of recorded speech, with optional response streaming.
- **`gpt-live-transcribe`**: OpenAI's recommended low-latency model for Realtime transcription sessions with live transcript deltas.
- **`gpt-4o-mini-tts`**: A compact and efficient text-to-speech model from OpenAI, perfect for responsive vocalization.
- **`gpt-realtime-2.1-mini`**: OpenAI's stated replacement for its text-to-speech models, used here for [speech synthesis over the Realtime API](#openai-model-deprecations).
- **`voxtral-mini-latest`**: Mistral AI's multilingual Voxtral ASR, built for long-form audio (32k token context) and tested on up to ~30 minutes per file, available via [Mistral AI](#5-deploying-with-mistral-ai-voxtral) or self-hosted open weights.
- **`kokoro`**: A high-quality, open-source text-to-speech model, available for local deployment via [Speaches](#2-deploying-with-speaches-local-service) and [Kokoro-FastAPI](#4-deploying-with-kokoro-fastapi-and-speaches-local-services).
- **`piper`**: Fast, local neural text-to-speech system with multiple high-quality voices, available for local deployment via [LocalAI](#3-deploying-with-localai-local-service).
- **`whisper`**: The original renowned open-source transcription model from OpenAI, widely used for its accuracy and versatility.
- **`Microsoft Edge TTS`**: High-quality neural voices from Microsoft's free cloud TTS API, no API key required, available via [OpenAI Edge TTS](#6-deploying-with-microsoft-openai-edge-tts).
- **`Chatterbox TTS`**: Self-hosted neural speech synthesis with voice cloning, easily deployable via included Docker Compose. See [Chatterbox TTS deployment guide](#7-deploying-with-chatterbox-tts) for details.
- **`orpheus`**: Canopy Labs' open-source LLM-based TTS built on Llama 3B, trained on 100,000+ hours of English speech with human-like intonation and emotion control via tags like `[cheerful]` or `[whisper]`, available via [Groq](#8-deploying-with-groq) for ultra-fast inference.

## Objectives

1. **Wyoming Server, OpenAI-compatible Client**: Function as an intermediary between the Wyoming protocol and OpenAI's ASR and TTS services.
2. **Service Consolidation**: Allow users of various programs to run inference on a single server without needing separate instances for each service.
Example: Sharing TTS/STT services between [Open WebUI](#open-webui) and [Home Assistant](#usage-in-home-assistant).
3. **Asynchronous Processing**: Enable efficient handling of multiple requests by supporting asynchronous processing of audio streams.
4. **Streaming Compatibility**: Bridge Wyoming's streaming TTS protocol with OpenAI-compatible APIs through intelligent sentence boundary chunking powered by [yasbd](https://github.com/speedyk-005/yasbd-lib), enabling responsive incremental audio delivery even when the underlying API doesn't support streaming text input. Concurrent pipelining (default limit of three in-flight requests) keeps playback smooth while ensuring events remain ordered.
5. **Simple Setup with Docker**: Provide a straightforward deployment process using [Docker and Docker Compose](#docker-recommended) for OpenAI and various popular open source projects.

## Terminology

- **TTS (Text-to-Speech)**: The process of converting text into audible speech output.
- **ASR (Automatic Speech Recognition) / STT (Speech-to-Text)**: Technologies that convert spoken language into written text. ASR and STT are often used interchangeably to describe this function.

## Installation (Local Development)

### Prerequisites

- Tested with Python 3.12 and 3.13
- Optional: OpenAI API key(s) if using proprietary models

### Instructions

1. **Clone the Repository**

   ```bash
   git clone https://github.com/roryeckel/wyoming_openai.git
   cd wyoming_openai
   ```

2. **Create a Virtual Environment** (optional but recommended)

   ```bash
   python3 -m venv venv
   source venv/bin/activate  # On Windows use `venv\Scripts\activate`
   ```

3. **Install as a Development Package**

    ```bash
    pip install -e .
    ```
    Assuming you have activated a virtual environment, the wyoming_openai package will be installed into it. This will build and install the package in editable mode, allowing you to make changes to the code without needing to reinstall it each time.

    Or, if you prefer to install it as a regular (production) package:

    ```bash
    pip install .
    ```

    This is more suitable for a global installation.

4. **Configure Environment Variables or Command Line Arguments**

### Note on Symlinks

This repository uses symbolic links to keep AI agent configuration files tool-agnostic:

- `CLAUDE.md` → `AGENTS.md`
- `.claude/skills` → `.agents/skills`

On **Linux and macOS**, symlinks work out of the box. On **Windows**, git may check out symlinks as plain text files containing the target path instead of actual symlinks. If this happens, the AI agent configuration files will not function correctly.

To enable symlink support on Windows:

1. Enable **Developer Mode** in Windows Settings (Settings → System → For developers)
2. Either clone with symlinks enabled:
   ```bash
   git clone -c core.symlinks=true https://github.com/roryeckel/wyoming_openai.git
   ```
   Or, if you've already cloned, enable symlinks and re-checkout:
   ```bash
   git config core.symlinks true
   git checkout -- .
   ```
   > **Warning:** `git checkout -- .` will reset all working tree files — commit or stash any uncommitted changes first.

You can verify symlinks are working by checking that `CLAUDE.md` contains the full project instructions rather than just the text `AGENTS.md`.

## Installation from PyPI [![Publish to PyPI](https://github.com/roryeckel/wyoming_openai/actions/workflows/publish-to-pypi.yml/badge.svg)](https://github.com/roryeckel/wyoming_openai/actions/workflows/publish-to-pypi.yml)

Since v0.3.2, `wyoming-openai` is now available on [PyPI](https://pypi.org/project/wyoming-openai/). To install the latest release, run:

```bash
pip install wyoming-openai
```

This is useful for local deployment when you don't want to clone the repository or if you want to use the library components in your own projects.

To upgrade to the latest version, run:

```bash
pip install --upgrade wyoming-openai
```

## Command Line Arguments

The proxy server can be configured using several command line arguments to tailor its behavior to your specific needs.

### Example Usage

Assuming you have installed the package in your current environment, you can run the server with the following command:

```bash
python -m wyoming_openai \
  --uri tcp://0.0.0.0:10300 \
  --log-level INFO \
  --log-file /tmp/logs/wyoming_openai.log \
  --languages en \
  --stt-openai-key YOUR_STT_API_KEY_HERE \
  --stt-openai-url https://api.openai.com/v1 \
  --stt-streaming-models gpt-transcribe \
  --stt-realtime-models gpt-live-transcribe gpt-realtime-whisper \
  --stt-backend OPENAI \
  --tts-openai-key YOUR_TTS_API_KEY_HERE \
  --tts-openai-url https://api.openai.com/v1 \
  --tts-models gpt-4o-mini-tts \
  --tts-streaming-models gpt-4o-mini-tts \
  --tts-voices alloy ash coral echo fable onyx nova sage shimmer \
  --tts-backend OPENAI \
  --tts-speed 1.0
```

## Configuration Options

In addition to using command-line arguments, you can configure the Wyoming OpenAI proxy server via environment variables. This is especially useful for containerized deployments.

### Table of Environment & Command Line Options

| **Command Line Argument**               | **Environment Variable**                   | **Default Value**                           | **Description**                                                      |
|-----------------------------------------|--------------------------------------------|-----------------------------------------------|----------------------------------------------------------------------|
| `--uri`                                 | `WYOMING_URI`                              | tcp://0.0.0.0:10300                           | The URI for the Wyoming server to bind to.                           |
| `--log-level`                           | `WYOMING_LOG_LEVEL`                        | INFO                                          | Sets the logging level (e.g., INFO, DEBUG).                          |
| `--log-file`                            | `WYOMING_LOG_FILE`                         | None                                          | Optional file path to additionally log to, in addition to stdout/stderr. Parent directories are created automatically if missing. The file is rotated once it reaches `--log-max-bytes`. |
| `--log-max-bytes`                       | `WYOMING_LOG_MAX_BYTES`                    | 10485760 (10 MiB)                             | Maximum size in bytes of the current log file before it is rotated. `0` disables rotation. Without a `--log-file` this setting has no effect. |
| `--log-backup-count`                    | `WYOMING_LOG_BACKUP_COUNT`                 | 5                                             | Number of rotated log files to keep (`.1` through the configured count). Older files are deleted, including leftovers from a previously larger count when the server starts. Without a `--log-file`, or with rotation disabled, this setting has no effect. |
| `--languages`                           | `WYOMING_LANGUAGES`                        | en                                            | Space-separated list of supported languages to advertise.            |
| `--stt-openai-key`                      | `STT_OPENAI_KEY`                           | None                                          | Optional API key for OpenAI-compatible speech-to-text services.      |
| `--stt-openai-url`                      | `STT_OPENAI_URL`                           | https://api.openai.com/v1                     | The base URL for the OpenAI-compatible speech-to-text API            |
| `--stt-models`                          | `STT_MODELS`                               | None (this, `STT_STREAMING_MODELS` or `STT_REALTIME_MODELS` is required for STT) | Space-separated list of models to use for the STT service. Example: `gpt-transcribe` |
| `--stt-streaming-models`                | `STT_STREAMING_MODELS`                     | None                                          | Space-separated list of STT models that support streaming (e.g. `gpt-transcribe`). Only these models will use streaming mode. |
| `--stt-realtime-models`                 | `STT_REALTIME_MODELS`                      | None                                          | Space-separated list of STT models that use OpenAI Realtime transcription sessions (e.g. `gpt-live-transcribe gpt-realtime-whisper`). These models stream audio over `/v1/realtime` and emit Wyoming `TranscriptChunk` snapshots before the final transcript. Not commonly supported by community projects. |
| `--stt-backend`                         | `STT_BACKEND`                              | None (autodetected)                           | Enable unofficial API feature sets.          |
| `--stt-temperature`                     | `STT_TEMPERATURE`                          | None (autodetected)                           | Sampling temperature for speech-to-text (ranges from 0.0 to 1.0)               |
| `--stt-prompt`                          | `STT_PROMPT`                               | None                                          | Optional prompt for STT requests (Text to guide the model's style).   |
| `--stt-extra-body`                      | `STT_EXTRA_BODY`                           | None                                          | JSON object merged into the `/v1/audio/transcriptions` request body via `extra_body` for backend-specific fields. `response_format` must remain `json`, and `stream` may override the model-derived default when set to `true` or `false`. Not applied to `STT_REALTIME_MODELS`. |
| `--stt-realtime-extra-body`             | `STT_REALTIME_EXTRA_BODY`                  | None                                          | JSON object merged into the transcription settings (`session.audio.input.transcription`) of `STT_REALTIME_MODELS` sessions, for example `{"keywords": ["Wyoming"]}`. Unlike `TTS_REALTIME_EXTRA_BODY` it is not merged into the session itself, so session settings such as `noise_reduction` cannot be set. Sent for every Realtime model, so only set fields all of them accept. A `null` value drops a field the proxy would otherwise send, for example `{"prompt": null}`. `model` cannot be set; use `STT_REALTIME_MODELS`. |
| `--tts-openai-key`                      | `TTS_OPENAI_KEY`                           | None                                          | Optional API key for OpenAI-compatible text-to-speech services.      |
| `--tts-openai-url`                      | `TTS_OPENAI_URL`                           | https://api.openai.com/v1                     | The base URL for the OpenAI-compatible text-to-speech API            |
| `--tts-models`                          | `TTS_MODELS`                               | None (this, `TTS_STREAMING_MODELS` or `TTS_REALTIME_MODELS` is required for TTS) | Space-separated list of models to use for the TTS service. Example: `gpt-4o-mini-tts` |
| `--tts-voices`                          | `TTS_VOICES`                               | Empty (autodetected)                          | Space-separated list of voices for TTS.        |
| `--tts-voice-labels`                    | `TTS_VOICE_LABELS`                         | None                                          | JSON object mapping a voice name to the display name shown by Wyoming clients such as Home Assistant, e.g. `{"af_heart": "Heart"}`. Only the displayed name changes (see below). |
| `--tts-backend`                         | `TTS_BACKEND`                              | None (autodetected)                           | Enable unofficial API feature sets.          |
| `--tts-speed`                           | `TTS_SPEED`                                | None (autodetected)                           | Speed of the TTS output (ranges from 0.25 to 4.0; capped at 1.5 for `TTS_REALTIME_MODELS`). |
| `--tts-instructions`                    | `TTS_INSTRUCTIONS`                         | None                                          | Optional instructions for TTS requests (Control the voice).    |
| `--tts-extra-body`                      | `TTS_EXTRA_BODY`                           | None                                          | JSON object merged into the TTS request body via `extra_body` for backend-specific fields. Audio settings such as `response_format`, `speed`, and `instructions` may override top-level values, but `stream` and `stream_format` are rejected. |
| `--tts-streaming-models`                | `TTS_STREAMING_MODELS`                     | None                                          | Space-separated list of TTS models to enable incremental streaming via [yasbd](https://github.com/speedyk-005/yasbd-lib) sentence chunking that powers the TTS streaming pipeline (e.g. `gpt-4o-mini-tts`) with up to three concurrent synthesis requests by default (see `TTS_CONCURRENT_REQUESTS`). |
| `--tts-realtime-models`                 | `TTS_REALTIME_MODELS`                      | None                                          | Space-separated list of TTS models that synthesize over OpenAI's `/v1/realtime` API instead of `/v1/audio/speech` (e.g. `gpt-realtime-2.1-mini`). These are added to the TTS models and can also be listed in `TTS_STREAMING_MODELS`. Experimental; see [OpenAI Model Deprecations](#openai-model-deprecations). |
| `--tts-realtime-extra-body`             | `TTS_REALTIME_EXTRA_BODY`                  | None                                          | JSON object merged into the Realtime session of `TTS_REALTIME_MODELS`, for example `{"reasoning": {"effort": "low"}}`. `TTS_EXTRA_BODY` is not applied to Realtime models. `audio.output.format` must remain 24 kHz `audio/pcm`, and `audio.output.voice`, a top-level `voice` or `speed`, `model` and `instructions` cannot be set (use `TTS_INSTRUCTIONS` for the delivery style). |
| `--tts-streaming-min-words`             | `TTS_STREAMING_MIN_WORDS`                  | None                                          | Minimum words per text chunk for incremental TTS streaming (optional). |
| `--tts-streaming-max-chars`             | `TTS_STREAMING_MAX_CHARS`                  | None                                          | Maximum characters per text chunk for incremental TTS streaming (optional). |
| `--tts-concurrent-requests`             | `TTS_CONCURRENT_REQUESTS`                  | 3                                             | Maximum number of simultaneous TTS requests per Wyoming connection while streaming TTS synthesizes several sentences at once. Lower it for a backend that cannot serve parallel requests, raise it for one that can. Must be a whole number of at least 1. For `TTS_REALTIME_MODELS` the limit counts requests in flight: the websocket of a finished request may still be closing when the next one opens. |

`STT_STREAMING_MODELS` and `STT_REALTIME_MODELS` select different STT transports. `STT_STREAMING_MODELS` still uses `/v1/audio/transcriptions` with response streaming after Wyoming `AudioStop`. `STT_REALTIME_MODELS` opens a `/v1/realtime` transcription session, sends 24 kHz mono PCM16 audio chunks as Wyoming audio arrives, commits on Wyoming `AudioStop`, and emits `TranscriptChunk` snapshots plus a final `Transcript`. Audio at another sample rate is resampled to 24 kHz continuously across chunks, so chunk boundaries do not change its timing; rates below 8 kHz are rejected. `audio-start` rate must be a whole number from 1 to 384000 Hz, channels a whole number from 1 to 65535, and width 1-4 bytes; anything else is rejected with an error event. Partial PCM frames at Wyoming chunk boundaries are buffered until the next chunk; an incomplete final frame is discarded when the session closes.

**Wyoming 1.11 streaming compatibility:** On both STT transports, each `TranscriptChunk.text` contains the entire transcript so far, replacing the previous chunk rather than appending to it. For example, backend deltas `"Hello"` and `" world"` produce Wyoming chunks `"Hello"` and `"Hello world"`. The final `Transcript` is authoritative and may correct earlier text. A backend that reports no text in its final event does not erase what it streamed: Speaches answers a streamed `/v1/audio/transcriptions` request with its deltas and an empty `transcript.text.done`, so the last chunk stands as the final transcript, as it did before this release. Custom clients using the pre-1.11 append semantics must be updated; there is no automatic negotiation of chunk semantics. Clients that only consume the final `Transcript`, including Home Assistant's current Wyoming STT integration, continue to work.

TTS requests can supply a voice name and text language together in `synthesize` and `synthesize-start`. Since wyoming 1.11 the language of a request that names a voice is no longer dropped, so it must be one the voice advertises, which every voice takes from `WYOMING_LANGUAGES`; otherwise the request fails. List every language your clients send in `WYOMING_LANGUAGES`. A regional tag matches its bare language either way (`en-US` and `en`), but two regions do not match each other (`en-GB` and `en-US`), and a tag with a script (`sr-Latn`) only matches itself. A `transcribe` language is matched against `WYOMING_LANGUAGES` the same way. The language also selects the sentence segmentation. A language-only request uses a voice of the selected program, or of the first program with voices when no program was selected, that supports the language, preferring the speech API over Realtime; if none does, the request fails. The proxy does not translate Wyoming `language` or `speaker` into backend-specific TTS request parameters.

Wyoming clients normally send `transcribe` before `audio-start`, as Home Assistant does, but it is optional. A stream without one uses the model of the last accepted `transcribe` on the connection, as long as the program chosen with `select-program` offers it, and otherwise the default model: the first model of that program, or of the first program. The transport (`/v1/audio/transcriptions` or `/v1/realtime`) is chosen at `audio-start` and kept until `audio-stop`. An `audio-start` before the previous stream's `audio-stop` abandons that stream, and so does a `transcribe` that is refused during it: the stream is not transcribed, a warning is logged, and a Realtime stream's transcript is closed with `transcript-stop` but no `transcript`. A refused `transcribe` also sends an `error` event and closes the connection, as it does outside a stream. Home Assistant 2026.8.2 to 2026.9, which upload all audio before reading, may hit the closed connection before reading that `error`; they report the failure either way. A `transcribe` that arrives during a `/v1/audio/transcriptions` recording can still change its model and language to another model of that transport, since the audio is only sent at `audio-stop`. Whether the response is streamed follows that model, so such a `transcribe` can change it, but a `select-program` cannot. A Realtime session keeps its model and language, and a recording never moves to the other transport. Whatever a `transcribe` cannot change is logged as a warning, and the selection applies from the next audio stream.

`STT_EXTRA_BODY`, `STT_REALTIME_EXTRA_BODY`, `TTS_EXTRA_BODY` and `TTS_REALTIME_EXTRA_BODY` must be valid JSON objects when used; empty or whitespace-only environment values are ignored. An explicit `--stt-extra-body`, `--stt-realtime-extra-body`, `--tts-extra-body` or `--tts-realtime-extra-body` overrides its environment value, even if that value is invalid. A `languages` override in either STT body must be a list of language codes. In `STT_REALTIME_EXTRA_BODY` it may also be `null`, which, like a `null` `language`, sends neither field, so every Realtime model detects the language itself. `--help` does not parse these environment values. The same holds for every other option read from the environment, such as `STT_BACKEND` or `TTS_SPEED`: an invalid value is reported as a usage error when it is used, and not at all when the option is given on the command line or with `--help`. `TTS_SPEED` and `STT_TEMPERATURE` must be finite numbers. The OpenAI client merges the values into the outgoing request body rather than sending a nested `extra_body` field. Supported overlaps still need to match Wyoming's transport expectations: STT continues to require `response_format="json"`, and a boolean `stream` override will update both the request body and the client's response parser for `/v1/audio/transcriptions`. TTS can override raw-audio fields such as `response_format`, `speed`, and `instructions`, but rejects `stream` and `stream_format` because the handler expects audio bytes instead of SSE-style framing.

TTS requests use `response_format="wav"` by default, and the proxy reads the sample rate, channels, and sample width from the WAV header. When `TTS_EXTRA_BODY` sets `{"response_format": "pcm"}`, there is no header, so the proxy treats the audio as **24 kHz, mono, 16-bit signed little-endian PCM**; it cannot detect a different sample rate from raw bytes. A backend that returns its voice's native rate instead will play at the wrong speed. If your backend's voices are not 24 kHz, leave `response_format` unset to keep WAV. For Speaches voices with a different native rate, you can instead request resampling with `TTS_EXTRA_BODY='{"response_format":"pcm","sample_rate":24000}'` (or the equivalent `--tts-extra-body` JSON). `sample_rate` is a Speaches request field, not part of the OpenAI API, and the proxy does not resample audio itself.

The proxy does not transcode TTS audio, and not every backend returns the format it is asked for. Some return WAV with 32-bit float samples instead of 16-bit PCM, and some ignore `response_format` and always send their own format, which may be compressed audio such as MP3. When a `wav` response has no PCM WAV header the proxy can read, it logs a warning that it is falling back to raw PCM and plays the bytes as 24 kHz PCM, which sounds like static; with `pcm` there is no header to check, so the same happens without a warning. If only the backend's WAV output is the problem, such as float samples, `{"response_format": "pcm"}` works around it as long as the backend honors it and returns 24 kHz audio. A backend that ignores `response_format` and returns anything other than PCM WAV is not supported for TTS.

`TTS_VOICE_LABELS` must be a valid JSON object whose values are non-empty strings. Wyoming clients such as Home Assistant show a voice's description in their voice picker, and this proxy uses the voice name as the description by default (for example `af_heart`). A label replaces the description (for example `Heart`) of the matching voice, whether the voice comes from `TTS_VOICES` or from the backend's voice list. The voice name that clients send back, and the name sent to the backend, do not change. A label is matched against the voice name first and then against the backend's voice name: when several TTS models offer the same voice, the voices are named like `af_heart (tts-1)`, so `{"af_heart": "Heart"}` labels all of them and keeps the model suffix so they stay distinguishable (`Heart (tts-1)`, `Heart (tts-1-hd)`), while `{"af_heart (tts-1)": "Heart (fast)"}` labels only that one, exactly as written. Surrounding whitespace in voice names and labels is ignored. A label that matches no voice is logged as a warning and ignored.

### OpenAI Model Deprecations

OpenAI has announced shutdown dates for most of its older audio models. This only affects the official OpenAI API; local and third-party backends are unaffected, including ones that reuse names such as `tts-1` for their own models. The old model names keep working until the dates below.

| **Models** | **Shutdown** | **OpenAI's replacement** |
|------------|--------------|--------------------------|
| `whisper-1`, `gpt-4o-transcribe`, `gpt-4o-mini-transcribe`, `gpt-4o-transcribe-diarize` | February 26, 2027 | `gpt-transcribe` (`STT_MODELS` / `STT_STREAMING_MODELS`) or `gpt-live-transcribe` (`STT_REALTIME_MODELS`) |
| `tts-1`, `tts-1-hd`, `gpt-4o-mini-tts` snapshots | January 6, 2027 | `gpt-realtime-2.1-mini` (`TTS_REALTIME_MODELS`) |

OpenAI's list names the dated `gpt-4o-mini-tts` snapshots and does not say whether the undated `gpt-4o-mini-tts` alias, the TTS model in the examples here, stays available after that date. Treat it as affected and plan the move to `TTS_REALTIME_MODELS` before then.

**Speech-to-text:** `gpt-transcribe` and `gpt-live-transcribe` take a `languages` list instead of the singular `language` field. On the `OPENAI` backend the proxy sends the Wyoming request language as `languages` for these models. A `languages` list in the extra body replaces the request language for every model that uses that body, on any backend, so the two fields are never sent together; only set it when all of those models accept `languages`. It is also how to use a differently named model that needs `languages`. Additional hints such as `{"keywords": ["Wyoming"]}` can be passed with `STT_EXTRA_BODY` for `/v1/audio/transcriptions` and with `STT_REALTIME_EXTRA_BODY` for `STT_REALTIME_MODELS` sessions; the two are kept separate because the Realtime API rejects fields it does not know. Each body is sent for every model on its transport, while the fields are model-specific: `keywords` and `languages` are for `gpt-transcribe` and `gpt-live-transcribe`, and `delay` is for the Realtime models `gpt-live-transcribe` and `gpt-realtime-whisper`. Only list the Realtime models that accept the fields you set. `STT_PROMPT` is left out of `gpt-realtime-whisper` Realtime sessions on the `OPENAI` backend, where it is not supported; for any other Realtime model that takes no prompt, set `{"prompt": null}` in `STT_REALTIME_EXTRA_BODY`. Since `gpt-realtime-whisper` accepts neither a prompt nor `keywords`, prefer `gpt-live-transcribe` when spoken commands include names a model could mishear, such as areas or devices. A Realtime session that delivers no final transcript within 30 seconds of the end of the audio fails like any other transcription (see [Failed Speech Recognition in Home Assistant](#failed-speech-recognition-in-home-assistant)).

**Text-to-speech:** OpenAI's replacement is not available on `/v1/audio/speech`, so `TTS_REALTIME_MODELS` opens a `/v1/realtime` session per synthesis request instead, or one per sentence chunk when the model is also listed in `TTS_STREAMING_MODELS`. Things to know before switching:

- Realtime models are conversational. The proxy instructs the model to read the text word for word, but this is not guaranteed; a warning is logged when the spoken transcript differs from the requested text (only at debug level when the text contains numbers or symbols, which a transcript spells out). Long text is the most likely to drift, so also list the model in `TTS_STREAMING_MODELS` to have it synthesized sentence by sentence. This applies to both plain `synthesize` requests and `synthesize-start/chunk/stop`, using the configured chunking limits and request concurrency.
- The voices are `alloy`, `ash`, `ballad`, `coral`, `echo`, `sage`, `shimmer`, `verse`, `marin` and `cedar`. `fable`, `onyx` and `nova` are not available on the official OpenAI Realtime API. Only on the official API are those three skipped for Realtime models when listed in `TTS_VOICES`, with a startup error if none remain; `ballad`, `verse`, `marin` and `cedar` are skipped for `tts-1` and `tts-1-hd` the same way, since those models do not offer them. Every other server keeps all configured voices, because it may offer voices of its own under those names, and a proxy on the `OPENAI` backend logs a warning when speech-only names are configured for Realtime models. Any other name in `TTS_VOICES` is passed through, so a voice OpenAI adds later can be used without an update.
- `TTS_INSTRUCTIONS` is appended to the read-aloud instructions as a delivery style, and `TTS_SPEED` is capped at 1.5.
- `TTS_EXTRA_BODY` only applies to `/v1/audio/speech`. Use `TTS_REALTIME_EXTRA_BODY` for the Realtime session, for example `{"reasoning": {"effort": "low"}}`. An `audio.output.format` other than 24 kHz `audio/pcm`, an `audio.output.voice`, a top-level `voice` or `speed` (use `audio.output.speed`), or an override of `type`, `output_modalities`, `tool_choice`, `tools`, `model` or `instructions` causes a startup error.
- A request that names no voice uses a voice of the program chosen with `select-program`, or else of the first program, which is the only one Home Assistant shows: the streaming program when `TTS_STREAMING_MODELS` is set. Within that program it takes the first voice of a `/v1/audio/speech` model, and a Realtime voice only when the program has no other. So with a Realtime model as the only streaming model, a request without a voice is spoken by it, sentence by sentence. A `synthesize-start` without a voice resolves it right away, so its sentences are spoken as they arrive when that voice's model is listed in `TTS_STREAMING_MODELS`, like those of a named voice.
- Voice names change with the number of TTS models. With one model a voice is advertised as `alloy`; with several, shared voices become `alloy (gpt-4o-mini-tts)` and `alloy (gpt-realtime-2.1-mini)`. A suffixed name that is no longer advertised still resolves to the matching voice with a warning, but re-select the voice in your Wyoming client after adding or removing models.
- A Realtime request that produces no audio, or stalls for 30 seconds, fails the synthesis. Text with nothing to read aloud, such as punctuation or emoji on their own, is skipped instead of being sent to the model; symbols with a spoken name, such as `+`, `&` or a currency sign, are still read. During sentence playback on `/v1/audio/speech`, a response without audio is skipped, with a warning if the sentence has speakable text. Later sentences continue playing, but the synthesis fails if no sentence produces audio and speakable text was skipped. An unsplit request without audio still fails for speakable text. A WAV that declares samples and delivers none always fails; partially delivered audio plays with a warning. A synthesis with nothing to read aloud at all, including one without any text, still ends as an empty audio stream.

```bash
python -m wyoming_openai \
  --tts-openai-key YOUR_TTS_API_KEY_HERE \
  --tts-realtime-models gpt-realtime-2.1-mini \
  --tts-streaming-models gpt-realtime-2.1-mini
```

## Docker (Recommended) [![Docker Image CI](https://github.com/roryeckel/wyoming_openai/actions/workflows/docker-image.yml/badge.svg)](https://github.com/roryeckel/wyoming_openai/actions/workflows/docker-image.yml)

### Prerequisites

- Ensure you have [Docker](https://www.docker.com/products/docker-desktop) and [Docker Compose](https://docs.docker.com/compose/install/) installed on your system.

### Deployment Options

You can deploy the Wyoming OpenAI proxy server in different environments depending on whether you are using official OpenAI services or a local alternative like Speaches. You can even run multiple wyoming_openai instances on different ports for different purposes. Below are example scenarios:

#### 1. Deploying with Official OpenAI Services

To set up the Wyoming OpenAI proxy to work with official OpenAI APIs, follow these steps:

- **Environment Variables**: Create a `.env` file in your project directory that includes necessary environment variables such as `STT_OPENAI_KEY`, `TTS_OPENAI_KEY`.

- **Docker Compose Configuration**: Use the provided `docker-compose.yml` template. This setup binds a Wyoming server to port 10300 and uses environment variables for OpenAI URLs, model configurations, and voices as specified in the compose file.

- **Command**:
  
  ```bash
  docker compose -f docker-compose.yml up -d
  ```

#### 2. Deploying with Speaches Local Service

If you prefer using a local service like Speaches instead of official OpenAI services, follow these instructions:

- **Docker Compose Configuration**: Use the `docker-compose.speaches.yml` template which includes configuration for both the Wyoming OpenAI proxy and the Speaches service.

- **Speaches Setup**:
  - The Speaches container is configured with specific model settings (`Systran/faster-distil-whisper-large-v3` for STT and `speaches-ai/Kokoro-82M-v1.0-ONNX` for TTS).
  - It uses a local port (8000) to expose the Speaches service.
  - NVIDIA GPU support is enabled, so ensure your system has an appropriate setup if you plan to utilize GPU resources.
  - Note: wyoming_openai disables Speaches VAD (Voice Activity Detection) by default, as it is not yet compatible with the Wyoming protocol.
  - [Learn more about Speaches](https://speaches.ai/)

- **Command**:
  
  ```bash
  docker compose -f docker-compose.speaches.yml up -d
  ```

#### 3. Deploying with LocalAI Local Service

LocalAI is a drop-in replacement for OpenAI API that runs completely locally, supporting both Whisper (STT) and Piper (TTS). This setup provides excellent privacy and performance without requiring external API keys.

- **LocalAI Setup**:
  - The provided example compose uses LocalAI's GPU-accelerated image with NVIDIA CUDA 12 support, but you can adjust things as needed.
  - Automatically downloads `whisper-base` model and multiple Piper TTS voices on first run
  - Provides OpenAI-compatible endpoints for seamless integration
  - No API keys required since everything runs locally
  - Includes automatic model initialization via dedicated init container
  - [Learn more about LocalAI](https://localai.io/)

- **Docker Compose Configuration**: Use the `docker-compose.localai.yml` template which includes configuration for both the Wyoming OpenAI proxy and LocalAI service.

- **Command**:

  ```bash
  docker compose -f docker-compose.localai.yml up -d
  ```

#### 4. Deploying with Kokoro-FastAPI and Speaches Local Services

For users preferring a setup that leverages Kokoro-FastAPI for TTS and Speaches for STT, follow these instructions:

- **Docker Compose Configuration**: Use the `docker-compose.kokoro-fastapi.yml` template which includes configuration for both the Wyoming OpenAI proxy and Kokoro-FastAPI TTS service (Kokoro).

- **Speaches Setup**:
  - Use it in combination with the Speaches container for access to STT.

- **Kokoro Setup**:
  - The Kokoro-FastAPI container provides TTS capabilities.
  - It uses a local port (8880) to expose the Kokoro service.
  - NVIDIA GPU support is enabled, so ensure your system has an appropriate setup if you plan to utilize GPU resources.
  - [Learn more about Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI)

- **Command**:

  ```bash
  docker compose -f docker-compose.speaches.yml -f docker-compose.kokoro-fastapi.yml up -d
  ```

#### 5. Deploying with Mistral AI Voxtral

For users who want high-quality multilingual speech transcription using Mistral AI's Voxtral model, this setup provides an excellent STT-only solution and strong multilingual accuracy with an option for self-hosting the model weights.

- **Mistral AI Voxtral Setup**:
  - Uses Mistral AI's Voxtral speech transcription API (requires Mistral API key, free tier available)
  - Supports multilingual transcription with high accuracy
  - Designed for long-form inputs (32k token context)
  - STT-only service (no TTS capabilities - combine with other services for TTS)
  - OpenAI-compatible API endpoints for seamless integration
  - [Learn more about Mistral AI](https://docs.mistral.ai/capabilities/audio_transcription)

- **Docker Compose Configuration**: Use the `docker-compose.voxtral.yml` template which includes configuration for the Wyoming OpenAI proxy with Mistral AI Voxtral backend.

- **Command**:
  
  ```bash
  docker compose -f docker-compose.voxtral.yml up -d
  ```

#### 6. Deploying with Microsoft OpenAI Edge TTS

For users who want high-quality text-to-speech without API costs, Microsoft Edge TTS provides excellent neural voices through a free cloud service. This setup requires no API keys and offers a wide variety of natural-sounding voices.

- **OpenAI Edge TTS Setup**:
  - Uses Microsoft's free cloud TTS service (no API key required)
  - Provides access to high-quality neural voices across multiple languages
  - The OpenAI Edge TTS container runs locally and proxies requests to Microsoft's service
  - Includes 17 English (US) voices by default, with support for many more languages
  - OpenAI-compatible API endpoints for seamless integration
  - [Learn more about the OpenAI-Compatible Edge-TTS API](https://github.com/travisvn/openai-edge-tts)

- **Docker Compose Configuration**: Use the `docker-compose.openai-edge-tts.yml` template which includes configuration for both the Wyoming OpenAI proxy and OpenAI Edge TTS service.

- **Command**:
  
  ```bash
  docker compose -f docker-compose.openai-edge-tts.yml up -d
  ```

#### 7. Deploying with Chatterbox TTS

For users who want high-quality local text-to-speech with voice cloning capabilities, Chatterbox TTS provides an OpenAI-compatible API with advanced voice cloning features. This setup runs completely locally and supports custom voice training.

- **Chatterbox TTS Setup**:
  - Local OpenAI-compatible text-to-speech API with voice cloning capabilities
  - Supports custom voice samples for personalized speech generation
  - Advanced voice library management with persistent storage
  - FastAPI-powered with real-time status monitoring and streaming support
  - GPU acceleration support for faster processing
  - No external API dependencies - runs completely offline
  - [Learn more about Chatterbox TTS API](https://github.com/travisvn/chatterbox-tts-api)

- **Docker Compose Configuration**: Use the `docker-compose.chatterbox.yml` template which includes configuration for both the Wyoming OpenAI proxy and Chatterbox TTS service.

- **Command**:
  
  ```bash
  docker compose -f docker-compose.chatterbox.yml up -d
  ```

#### 8. Deploying with Groq

For users who want to use Groq's fast inference cloud API, this setup provides access to high-performance Whisper models for speech recognition and Orpheus TTS for text-to-speech. Groq offers extremely fast inference speeds with their LPU (Language Processing Unit) architecture and a generous free tier that supports up to 2,000 STT requests and 100 TTS requests per day.

- **Groq Setup**:
  - Cloud-based OpenAI-compatible API with ultra-fast inference
  - Supports Whisper Large V3 for highly accurate speech recognition
  - Provides access to [Orpheus TTS](https://github.com/canopyai/Orpheus-TTS) by Canopy Labs - an open-source LLM-based TTS with human-like intonation
  - Orpheus supports emotion control via inline tags like `[cheerful]` or `[whisper]` embedded in the input text; for Home Assistant, this means prompting your LLM to include these tags in its responses
  - Generous free tier: 2,000 STT requests/day (up to 8 hours of audio) and 100 TTS requests/day
  - Requires Groq API key (sign up at [groq.com](https://groq.com))
  - No local infrastructure required
  - [Learn more about Groq](https://groq.com/)

- **Docker Compose Configuration**: Use the `docker-compose.groq.yml` template which includes configuration for the Wyoming OpenAI proxy connecting to Groq's API endpoints.

- **Environment Variables**: Create a `.env` file with your Groq API key as `STT_OPENAI_KEY` and/or `TTS_OPENAI_KEY`.

- **Command**:

  ```bash
  docker compose -f docker-compose.groq.yml up -d
  ```

#### 9. Development with Docker

If you are developing the Wyoming OpenAI proxy server and want to build it from source, use the `docker-compose.dev.yml` file along with the base configuration.

- **Command**:
  
  ```bash
  docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d --build
  ```

#### 10. Example: Development with Additional Local Service

For a development setup using the Speaches local service, combine `docker-compose.speaches.yml` and `docker-compose.dev.yml`. This also works for `docker-compose.kokoro-fastapi.yml`, `docker-compose.localai.yml`, `docker-compose.voxtral.yml`, `docker-compose.openai-edge-tts.yml`, `docker-compose.groq.yml`, and `docker-compose.chatterbox.yml`.

- **Command**:
  
  ```bash
  docker compose -f docker-compose.speaches.yml -f docker-compose.dev.yml up -d --build
  ```

#### 11. Docker Tags

We follow specific tagging conventions for our Docker images. These tags help in identifying the version and branch of the code that a particular Docker image is based on.

- **`latest`**: This tag always points to the latest stable release of the Wyoming OpenAI proxy server. It is recommended for users who want to run the most recent, well-tested version without worrying about specific versions.

- **`main`**: This tag points to the latest commit on the main code branch. It is suitable for users who want to experiment with the most up-to-date features and changes, but may include unstable or experimental code.

- **`major.minor.patch version`**: Specific version tags (e.g., `0.9.0`) correspond to specific stable releases of the Wyoming OpenAI proxy server. These tags are ideal for users who need a consistent, reproducible environment and want to avoid breaking changes introduced in newer versions.

- **`major.minor version`**: Tags that follow the `major.minor` format (e.g., `0.9`) represent a range of patch-level updates within the same minor version series. These tags are useful for users who want to stay updated with bug fixes and minor improvements without upgrading to a new major or minor version.

- **`pr-{number}`**: Pull request tags (e.g., `pr-123`) are automatically created for each pull request to allow testing of proposed changes before they are merged. These tags are automatically cleaned up when the pull request is closed or merged.

#### 12. Pull Request Docker Images

For contributors and maintainers who want to test changes from pull requests before they are merged, we automatically build and push Docker images for each pull request.

**How it works:**
- When a pull request is opened or updated, a Docker image is automatically built and pushed with the tag `pr-{pr_number}`
- Only pull requests from the main repository (not forks) will trigger image builds for security reasons
- When a pull request is closed or merged, the corresponding Docker image is automatically deleted to save storage space

**Using PR images:**
```bash
# Example: Test PR #123
docker run --rm -p 10300:10300 ghcr.io/roryeckel/wyoming_openai:pr-123

# Or with docker-compose, update your docker-compose.yml:
# image: ghcr.io/roryeckel/wyoming_openai:pr-123
```

**For contributors:**
- PR images are built automatically - no action needed
- Images are available within minutes of opening/updating a PR
- Check the Actions tab to see build status
- Images are automatically cleaned up when the PR is closed

### General Deployment Steps

1. **Start Services**: Run the appropriate Docker Compose command based on your deployment option.
2. **Verify Deployment**: Ensure that all services are running by checking the logs with `docker compose logs -f` or accessing the Wyoming OpenAI proxy through its exposed port (e.g., 10300) to ensure it responds as expected.
3. **Configuration Changes**: You can modify environment variables in the `.env` file or directly within your Docker Compose configuration files to adjust settings such as languages, models, and voices without rebuilding containers.

## Usage in Home Assistant

1. Install & set up your Wyoming OpenAI instance using one of the [deployment options](#deployment-options) above.
2. In HA, Go to Settings, Devices & Services, Add Integration, and search for Wyoming Protocol. Add the Wyoming Protocol integration with the URI of your Wyoming OpenAI instance.
3. The hard part is over! Configure your Voice Assistant pipeline to use the STT/TTS services provided by your new Wyoming OpenAI instance.

#### Reloading Configuration Changes in Home Assistant

When you make changes to your configuration such as updating models, voices, or URLs, it's important to reload the Wyoming OpenAI integration in Home Assistant to apply these changes. Here's how to do it:

1. Go to **Settings** > **Devices & Services**
2. Find and select your **Wyoming OpenAI** integration
3. Click on **Reload**

#### Failed Speech Synthesis in Home Assistant

A synthesis that fails, or is cut short because the server shuts down, sends a Wyoming `error` event before the connection closes. A failed `synthesize` request also ends without `audio-stop`, so the audio it got as far as sending is not taken for a complete result and cached. Home Assistant reports the `error` event from version 2026.8.2. Older versions ignore it: they still see a failed `synthesize` request as a lost connection, but cannot tell a failed `synthesize-start/chunk/stop` synthesis (used for models in `TTS_STREAMING_MODELS`) from one that ended early.

#### Failed Speech Recognition in Home Assistant

A transcription that fails sends a Wyoming `error` event and closes the connection. This covers a `transcribe` that names an unknown model or a language the model does not list, a failed `/v1/audio/transcriptions` request or response stream, a Realtime session that fails or delivers no final transcript within 30 seconds of the end of the audio, and a Realtime session that cannot be opened, or that the server rejects or closes while the audio is still arriving (for example over a field in `STT_REALTIME_EXTRA_BODY` the model does not accept). Those Realtime failures are reported as soon as they are known rather than when the audio ends; the rest of the audio is dropped, and the connection is closed when it ends. The cause is only written to this server's log. Every Home Assistant version takes the closed connection as a failed transcription, so the Assist pipeline fails with `stt-stream-failed` rather than reporting that nothing was said. From 2026.8.2 Home Assistant also reads the `error` event, which appears in its log, and from 2026.10 a Realtime session that fails while the audio is arriving ends the request without waiting for the speaker to finish.

### Sequence Diagrams

#### Home Assistant

Home Assistant uses the Wyoming Protocol integration to communicate with the Wyoming OpenAI proxy server. The proxy server then communicates with the OpenAI API to perform the requested ASR or TTS tasks. The results are then sent back to Home Assistant.

```mermaid
sequenceDiagram
  participant HA as Home Assistant
  participant WY as wyoming_openai Proxy
  participant OAPI as OpenAI-Compatible API
  
  Note over HA,OAPI: **Speech-to-Text (STT/ASR) Flow**
  HA->>WY: Transcribe event (initiate transcription)
  HA->>WY: AudioStart event (begin sending audio)
  loop While capturing microphone audio
    HA->>WY: AudioChunk events (WAV data)
    Note over WY: Accumulates/buffers WAV PCM chunks
  end
  HA->>WY: AudioStop event (end of input)
  
  alt Non-Streaming Transcription
    WY->>OAPI: Upload complete audio file
    OAPI-->>WY: Full text transcript
    WY->>HA: TranscriptStart event
    WY->>HA: Transcript event (full text result)
    WY->>HA: TranscriptStop event
  else Streaming Transcription
    WY->>OAPI: Send audio with `stream=true`
    WY->>HA: TranscriptStart event
    loop As partial results are returned
      OAPI-->>WY: Transcript delta
      WY-->>HA: TranscriptChunk event (transcript so far)
    end
    WY->>HA: Transcript event (final text)
    WY->>HA: TranscriptStop event
  end
  
  Note over HA,OAPI: **Text-to-Speech (TTS) Flow**
  
  alt Non-Streaming TTS (Synthesize)
    HA->>WY: Synthesize event (text + voice)
    WY->>OAPI: Speech synthesis request
    WY->>HA: AudioStart event
    loop While receiving audio data
      OAPI-->>WY: Audio stream chunks
      WY-->>HA: AudioChunk events
    end
    WY->>HA: AudioStop event
  else Streaming TTS (SynthesizeStart/Chunk/Stop)
    HA->>WY: SynthesizeStart event (voice config)
    Note over WY: Initialize incremental synthesis<br/>with yasbd-powered sentence boundary detection<br/>and up to three concurrent OpenAI TTS requests by default
    WY->>HA: AudioStart event
    loop Sending text chunks
      HA->>WY: SynthesizeChunk events
      Note over WY: Accumulate text and detect<br/>complete sentences using yasbd sentence chunking<br/>while prefetching audio in parallel (max 3 concurrent requests by default)
      alt Complete sentences detected
        loop For each complete sentence
          WY->>OAPI: Speech synthesis request
          loop While receiving audio data
            OAPI-->>WY: Audio stream chunks
            WY-->>HA: AudioChunk events (incremental)
          end
        end
      end
    end
    HA->>WY: SynthesizeStop event
    Note over WY: Process any remaining text<br/>and finalize synthesis
    WY->>HA: AudioStop event
    WY->>HA: SynthesizeStopped event
  end
```

#### Open WebUI

No proxy is needed for Open WebUI, because it has native support for OpenAI-compatible endpoints.

```mermaid
sequenceDiagram
    participant OW as Open WebUI
    participant OAPI as OpenAI API

    Note over OW,OAPI: Speech-to-Text (STT/ASR) Flow
    OW->>OAPI: Direct audio transcription request
    OAPI-->>OW: Text transcript response
    
    Note over OW,OAPI: Text-to-Speech (TTS) Flow
    OW->>OAPI: Direct speech synthesis request
    OAPI-->>OW: Audio stream response
    
```

## Contributing

Contributions are welcome! Please review [CONTRIBUTING.md](CONTRIBUTING.md) before opening an issue or submitting a pull request. For major changes, please first discuss the proposed changes in an issue.

For setup help and general support, use [GitHub Discussions](https://github.com/roryeckel/wyoming_openai/discussions). Use [GitHub Issues](https://github.com/roryeckel/wyoming_openai/issues) for reproducible bugs, compatibility reports, and scoped feature requests. Wyoming OpenAI targets OpenAI-compatible APIs and the Wyoming protocol; see [COMPATIBILITY.md](COMPATIBILITY.md) for the project scope and compatibility policy.

## Quality Assurance

### Linting (Ruff) [![Lint](https://github.com/roryeckel/wyoming_openai/actions/workflows/lint.yml/badge.svg)](https://github.com/roryeckel/wyoming_openai/actions/workflows/lint.yml)

This project uses [Ruff](https://github.com/astral-sh/ruff) for linting and code quality checks. Ruff is a fast Python linter written in Rust that can replace multiple tools like flake8, isort, and more.

To use Ruff during development:

1. Create and activate a virtual environment, then install development dependencies:
   ```bash
   pip install -e ".[dev]"
   ```

2. Run Ruff to check your code:
   ```bash
   ruff check .
   ```

A GitHub Action automatically runs Ruff on all pull requests and branch pushes to ensure code quality.

### Type Checking (Pyright)

This project uses [Pyright](https://github.com/microsoft/pyright) for static type checking. Pyright is a fast, full-featured Python type checker from Microsoft that helps catch type errors before runtime.

To use Pyright during development:

1. Create and activate a virtual environment, then install development dependencies:
   ```bash
   pip install -e ".[dev]"
   ```

2. Run Pyright to type-check your code:
   ```bash
   pyright
   ```

Pyright configuration is defined in `pyproject.toml` under `[tool.pyright]`.

### Testing (Pytest) [![Test](https://github.com/roryeckel/wyoming_openai/actions/workflows/test.yml/badge.svg)](https://github.com/roryeckel/wyoming_openai/actions/workflows/test.yml)

This project uses [pytest](https://pytest.org/) for unit testing. Tests are located in the [`tests/`](tests/) directory and cover core modules such as compatibility, constants, handlers, initialization, and utilities.

#### Running Tests

1. Install development dependencies:
   ```bash
   pip install -e ".[dev]"
   ```

2. In the [`tests/`](tests/) folder, run all tests with:
   ```bash
   pytest
   ```

3. Check the test coverage with:
   ```bash
   pytest --cov=wyoming_openai
   ```

All new code should include appropriate tests.
A GitHub Action automatically runs pytest on all pull requests and branch pushes to ensure tests pass.

#### End-to-End Integration Tests

The `tests/test_e2e.py` suite drives a real `wyoming_openai` server over TCP and a real Speaches backend in Docker, with no Home Assistant or microphone needed. TTS output is verified by feeding it back through STT and fuzzy-matching the transcript against the original phrase (with tolerance for minor mishearing).

These tests are gated behind the `integration` marker and skipped by default. To run locally, just:

```bash
pytest -m "integration and not ha_e2e" -v
```

The Home Assistant suite has its own `ha_e2e` marker and is described below.

The fixtures handle the backend automatically: a Speaches instance already healthy on `localhost:8000` is reused; otherwise the Docker Compose stack (`docker-compose.speaches-cpu.yml` layered with `docker-compose.e2e.yml`, which swaps in a smaller Whisper model for testing) is started and torn down; if Docker is unavailable, the suite skips. Useful environment variables:

- `WYOMING_OPENAI_KEEP_SPEACHES=1` - leave the Speaches container running between invocations for faster iteration (model load is the slow part).
- `WYOMING_E2E_URI=tcp://host:port` - target an already-running proxy instead of spawning a subprocess (the autodetection test skips in this mode, since it must control the server's environment).
- `WYOMING_E2E_ARTIFACT_DIR=<dir>` - save synthesized WAVs and server logs for debugging (CI uses this to upload artifacts on failure).

To test the actual Docker image end-to-end (what the containerized CI smoke job does):

```bash
docker build -t wyoming_openai:e2e .
docker compose -f docker-compose.speaches-cpu.yml -f docker-compose.e2e.yml up -d
WYOMING_E2E_URI=tcp://127.0.0.1:10300 pytest -m "integration and smoke" -v
```

Or in PowerShell:

```powershell
docker build -t wyoming_openai:e2e .
docker compose -f docker-compose.speaches-cpu.yml -f docker-compose.e2e.yml up -d
$env:WYOMING_E2E_URI = "tcp://127.0.0.1:10300"; pytest -m "integration and smoke" -v
```

Windows note: this works under Docker Desktop; run pytest from a native Windows Python (not WSL) so subprocess spawning and localhost port publishing line up.

Realtime models (`STT_REALTIME_MODELS` and `TTS_REALTIME_MODELS`) are intentionally not covered - they use the OpenAI websocket API, which Speaches does not implement.

The [Integration](.github/workflows/integration.yml) GitHub Actions workflow runs this suite on pushes and PRs against `main`: one job with the subprocess proxy, plus a smoke job against the freshly built Docker image.

#### Home Assistant End-to-End Tests

`tests/ha_e2e/` drives a real, headless Home Assistant against the proxy. Instead of a hand-written Wyoming client, it onboards Home Assistant through the REST API, adds the Wyoming config entry through the config-flow API, creates an Assist pipeline over the websocket API, and runs each turn through `assist_pipeline/run` from the STT stage to the TTS stage. A test-only custom conversation agent streams scripted replies through Home Assistant's ChatLog delta API, so streaming TTS is exercised deterministically without an LLM.

The suite covers a multi-sentence reply with numbers, a ~2000-character reply with pauses between deltas, a one-word reply, an invalid STT model, and a proxy `docker stop` during a long streaming reply. TTS output is fetched from Home Assistant's `tts_output` URL and transcribed directly with Speaches, independently of the proxy.

Run it locally:

```bash
docker build -t wyoming_openai:e2e .
docker compose -f docker-compose.speaches-cpu.yml -f docker-compose.e2e.yml -f docker-compose.ha-e2e.yml up -d
pytest -m ha_e2e -v --timeout=600
```

Or in PowerShell:

```powershell
docker build -t wyoming_openai:e2e .
docker compose -f docker-compose.speaches-cpu.yml -f docker-compose.e2e.yml -f docker-compose.ha-e2e.yml up -d
pytest -m ha_e2e -v --timeout=600
```

The harness generates a test-only Home Assistant user and password on a fresh instance and stores them, along with the config-entry IDs it creates, in `tests/ha_e2e/.state/` (gitignored). Re-runs reuse the stored refresh token and reload those entries instead of adding duplicates. If Home Assistant is already onboarded but that state is missing, reset the stack:

```bash
docker compose -f docker-compose.speaches-cpu.yml -f docker-compose.e2e.yml -f docker-compose.ha-e2e.yml down -v
```

Use `HA_VERSION` to pin the Home Assistant image (default `2026.10.1`); `down -v` is required when switching versions or resetting onboarding:

```powershell
$env:HA_VERSION = "2026.9.0"
docker compose -f docker-compose.speaches-cpu.yml -f docker-compose.e2e.yml -f docker-compose.ha-e2e.yml up -d
```

The pipeline's TTS URL returns mp3 by default, so the harness needs `ffmpeg` on the host to decode it before transcription. It is present on GitHub's Ubuntu runners; install it locally if `ffmpeg -version` fails.

Each run writes input WAVs, TTS output WAVs, event JSON, and `report.md` under `tests/ha_e2e/artifacts/run-<timestamp>/`. Override the location with `HA_E2E_ARTIFACT_DIR`. Other useful variables are `HA_E2E_BASE_URL`, `HA_E2E_SPEACHES_URL`, `HA_E2E_STT_MODEL`, `HA_E2E_PROXY_CONTAINER`, and `HA_E2E_HA_CONTAINER`.

The fixture WAVs are committed under `tests/ha_e2e/fixtures/audio/`. They are generated once with Speaches Kokoro, not by the proxy under test, and are 16 kHz mono with trailing silence for Home Assistant's VAD. Regenerate them with Speaches running:

```bash
docker compose -f docker-compose.speaches-cpu.yml -f docker-compose.e2e.yml up -d speaches
python tests/ha_e2e/generate_fixtures.py
```

The failure lanes require the Docker CLI: the invalid-STT proxy runs next to the good one, and the streaming-reply lane stops the proxy container to exercise the SIGTERM path, then restarts it for later runs.

The [Home Assistant E2E](.github/workflows/ha-e2e.yml) workflow runs on `workflow_dispatch` and on pull requests labeled `ha-e2e` (not on every push). It is also called by the release workflows: the version bump cannot open a release PR until it passes, and `draft-release.yml` will not create a draft release until it passes against the exact release commit. Publishing the draft therefore cannot ship a release that skipped the Home Assistant check. Set the repository variable `HA_E2E_VERSION` to override the Home Assistant image tag used by release runs.

Realtime models (`STT_REALTIME_MODELS` and `TTS_REALTIME_MODELS`) are not covered by the Speaches lane. The compose overlay and workflow contain a clearly marked, unwired hook for a later OpenAI HTTP/Realtime lane that runs only by `workflow_dispatch` with an API key secret.
