Metadata-Version: 2.5
Name: livekit-plugins-zoom-ai
Version: 1.8.11
Summary: Agent Framework plugin for speech-to-text with Zoom Scribe.
Project-URL: Documentation, https://docs.livekit.io
Project-URL: Website, https://livekit.io/
Project-URL: Source, https://github.com/livekit/agents
Author: Zoom
License-Expression: Apache-2.0
Keywords: audio,livekit,realtime,scribe,speech-to-text,stt,webrtc,zoom
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Topic :: Multimedia :: Sound/Audio
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10.0
Requires-Dist: livekit-agents>=1.8.11
Description-Content-Type: text/markdown

# Zoom AI plugin for LiveKit Agents

Support for [Zoom Scribe](https://developers.zoom.us/docs/ai-services/scribe/) speech-to-text, in two modes:

- **Live** ([docs](https://developers.zoom.us/docs/ai-services/scribe/live-mode/)): streaming over WebSocket. Scribe's own VAD ends each speech turn and returns its final transcript.
- **Fast** ([docs](https://developers.zoom.us/docs/ai-services/scribe/fast-mode/)): one HTTP request per utterance. Not streaming, so `AgentSession` pairs it with a VAD.

See [zoom.ai](https://www.zoom.ai/) for all Zoom AI Services APIs and features.

## Installation

```bash
pip install livekit-plugins-zoom-ai
```

Or install with the LiveKit Agents extra:

```bash
uv add "livekit-agents[zoom-ai]"
```

## Pre-requisites

The plugin needs a Zoom AI Services API key. Create one in [Zoom Platform Studio](https://platform.zoom.us/) under Credentials → API Keys, with the Scribe permissions.

Set it in your `.env` file:

```
ZOOM_AI_API_KEY=<your_zoom_api_key>
```

## Usage

### Scribe Live (streaming)

Scribe detects the end of each turn, so let the STT drive turn detection:

```python
from livekit.agents import AgentSession
from livekit.plugins import zoom_ai

session = AgentSession(
    stt=zoom_ai.STT(language="en-US"),
    turn_handling={"turn_detection": "stt"},
    # ... llm, tts, etc.
)
```

### Scribe Fast (per utterance)

```python
from livekit.agents import AgentSession
from livekit.plugins import silero, zoom_ai

session = AgentSession(
    stt=zoom_ai.STT(mode="fast"),
    vad=silero.VAD.load(),  # cuts the audio into utterances
    # ... llm, tts, etc.
)
```

### Without an agent

Both modes also work on their own, for example for transcription. Outside an agent job, pass an `aiohttp` session:

```python
stt = zoom_ai.STT(http_session=session, silence_duration_ms=600)  # Live
stream = stt.stream()  # push_frame() audio, then iterate the stream for SpeechEvents

event = await zoom_ai.STT(mode="fast", http_session=session).recognize(frames)  # Fast
```

For transcription, a longer `silence_duration_ms` (such as 600) keeps sentences with pauses together and can improve accuracy; the 350 ms default is tuned for voice agents.

## Options

Options map to the Scribe API fields of the same name; see the [Scribe docs](https://developers.zoom.us/docs/ai-services/scribe/) for what each one does and its availability. For a Scribe feature without an option here, pass its fields in `extra_kwargs`.

| Option | Default | Description |
|---|---|---|
| `mode` | `"live"` | `"live"` (streaming) or `"fast"` (one request per utterance) |
| `api_key` | `$ZOOM_AI_API_KEY` | Zoom AI Services API key |
| `language` | `"en-US"` | Recognition language ([supported languages](https://developers.zoom.us/docs/ai-services/scribe/speech-understanding/)); a regional variant Scribe doesn't list uses its base language's locale (`en-GB` → `en-US`) |
| `model` | `None` | Recognition model; `None` uses Scribe's default |
| `language_detection` | `None` | Language detection mode (Live supports `"continuous"`); the language Scribe detects is in `SpeechData.language`. With `zoom-scribe-multilingual` in Live, set it to get each final's language |
| `silence_duration_ms` | `350` | Live: silence (ms) that ends a turn; `None` uses Scribe's default |
| `diarization` | `False` | Speaker labels: `SpeechData.speaker_id`, or `metadata["speaker_segments"]` when several people speak in one final |
| `word_time_offsets` | `False` | Fast: word timings in `SpeechData.words` |
| `channel_separation` | `False` | Fast: recognize each stereo channel separately; channels in `metadata["channel_segments"]` |
| `vocabulary` | `None` | Words and phrases to recognize, such as names and product terms |
| `text_polishing` | `False` | Polished text as the transcript, with the original in `metadata["transcript"]`; if polishing fails, the original text. `en-US` only |
| `extra_kwargs` | `None` | Other Scribe request fields, e.g. `{"config": {"language_detection": {"candidate_languages": ["en-US", "fr-FR"]}}}` to restrict language detection |
| `base_url` | `https://api.zoom.us/v2/aiservices/scribe` | Scribe API base URL |
