Metadata-Version: 2.5
Name: elevenlabs-msteams-bridge
Version: 0.2.5
Summary: Bridge Microsoft Teams voice/video calls to an ElevenLabs Agent. Terminates the StandIn media bridge wire protocol on one side and the ElevenLabs Agent WebSocket on the other. No transcoding: PCM 16k verbatim both ways, barge-in, on-demand vision, call governors.
Project-URL: Homepage, https://komaa-com.github.io/elevenlabs-msteams-bridge-py/
Project-URL: Documentation, https://komaa-com.github.io/elevenlabs-msteams-bridge-py/
Project-URL: StandIn docs, https://docs.komaa.com/elevenlabs/installation
Project-URL: Repository, https://github.com/komaa-com/elevenlabs-msteams-bridge-py
Project-URL: Issues, https://github.com/komaa-com/elevenlabs-msteams-bridge-py/issues
Author: Komaa DigiTech
License-Expression: MIT
License-File: LICENSE
Keywords: agent,bridge,conversational-ai,elevenlabs,microsoft-teams,telephony,voice,websocket
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: AsyncIO
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Communications :: Telephony
Classifier: Topic :: Multimedia :: Sound/Audio
Requires-Python: >=3.10
Requires-Dist: aiohttp<4,>=3.9
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff==0.15.21; extra == 'dev'
Description-Content-Type: text/markdown

# Microsoft Teams Bridge for ElevenLabs Agents (Python)

[![CI](https://github.com/komaa-com/elevenlabs-msteams-bridge-py/actions/workflows/ci.yml/badge.svg)](https://github.com/komaa-com/elevenlabs-msteams-bridge-py/actions/workflows/ci.yml)
[![PyPI version](https://img.shields.io/pypi/v/elevenlabs-msteams-bridge.svg)](https://pypi.org/project/elevenlabs-msteams-bridge/)
[![Python versions](https://img.shields.io/pypi/pyversions/elevenlabs-msteams-bridge.svg)](https://pypi.org/project/elevenlabs-msteams-bridge/)
[![docs](https://img.shields.io/badge/docs-komaa--com.github.io-2563eb.svg)](https://komaa-com.github.io/elevenlabs-msteams-bridge-py/)
[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](https://github.com/komaa-com/elevenlabs-msteams-bridge-py/blob/main/CONTRIBUTING.md)

Put an [ElevenLabs Agent](https://elevenlabs.io/docs/eleven-agents/api-reference/eleven-agents/websocket) on a real **Microsoft Teams call**. The hosted **StandIn media bridge** ([standin.komaa.com](https://standin.komaa.com)) joins the Teams call and dials into this bridge over an HMAC-authenticated WebSocket; the bridge opens one ElevenLabs Agent conversation per call and relays between them. You run no Teams media stack yourself.

```text
Microsoft Teams call
       |
       v
StandIn media bridge      (hosted; joins the call)
       |   HMAC WebSocket, PCM 16 kHz
       v
this bridge               (you run it)
       |   WebSocket
       v
ElevenLabs Agent          (STT + LLM + TTS + turn-taking)
```

The hot path is **copy-only**: both sides speak base64 PCM 16 kHz mono (`pcm_16000`), so caller audio and agent audio are relayed **verbatim** in both directions. No resampling, no re-encoding, no transcoding.

## Features

- **Realtime voice, end to end** - the caller talks to your ElevenLabs agent and hears it reply. Turn-taking, VAD and interruption are the agent's own (server-side); the bridge adds nothing to the latency budget beyond a relay hop.
- **Barge-in done right** - when the caller interrupts, the bridge cancels playback on the Teams side and drops stale in-flight agent audio by `event_id`, so no "audio ghosts" play after the cut.
- **Per-call personalization** - caller name, tenant and call direction are injected as `dynamic_variables` at conversation start; an optional localized greeting or spoken disclosure rides `first_message`; per-caller memory uses the caller's AAD id as `user_id` (guests get none, never a shared identity).
- **Vision on demand** - a `look` client tool lets the agent see the caller's camera or screen-share: describe-then-answer via any OpenAI-compatible vision endpoint, or native multimodal upload (recording-gated). See [Vision and recording](#vision-and-recording).
- **Agent client tools** - `end_call`, `express` (avatar emotion), `show_image` (image on the bot's video tile, SSRF-guarded), `look`.
- **Two call governors** - a StandIn-side cutoff the bridge speaks a goodbye for, and a bridge-side `MAX_CALL_MINUTES` hard cap with a deterministic TTS goodbye.
- **Observability** - `GET /healthz` for liveness and `GET /metrics` (Prometheus text format): calls, durations, rejects, relay/drop counters.
- **Hardened transport** - replay-proof HMAC upgrade, single-use handshake guard, connection caps, payload caps, pre-start timeout, dead-peer detection, graceful SIGTERM drain, and an `EL_HOST` allowlist so your API key can only be sent to ElevenLabs.

> **Not yet at parity with the Node.js sibling.** Five of its features are not implemented here: the
> group-call gate (`GROUP_CALL_REQUIRE_ADDRESS` / `GROUP_CALL_WAKE_PHRASES`, "speak only when
> addressed" in a meeting), ambient vision (`AMBIENT_VISION_ENABLED`, continuous visual awareness),
> the per-call vision spend cap (`MAX_VISION_PER_MINUTE`), the no-answer reaper
> (`STALE_CALL_REAPER_SECONDS`, ending a call that never went live), and viseme lip-sync. All are on
> the roadmap below. One environment variable is also named differently: the pairing secret is
> `WORKER_SHARED_SECRET` here and `BRIDGE_SECRET` there. Everything else, including the wire protocol
> and the remaining variable names, is the same.

## Install

```bash
pip install elevenlabs-msteams-bridge
```

Requires Python 3.10+.

## Run

This is the whole configuration - three values, all required, no optional keys. Everything else has a
default that is already correct. Of the four backends this is the least to stand up: the agent itself
lives in the ElevenLabs dashboard, so there is no gateway to run beside the bridge. Put them in a
`.env` file in the working directory, which is loaded automatically (an existing environment variable
always wins):

```bash
# Your ElevenLabs account key.
ELEVENLABS_API_KEY=sk_...

# The Conversational AI agent this bridge speaks for. Create it at elevenlabs.io first -
# its prompt, voice and tools are configured there, not here.
ELEVENLABS_AGENT_ID=agent_...

# The connection secret from the StandIn portal. Must byte-match, or the HMAC handshake is
# rejected with 401 - which looks, from the caller's side, like the bot simply never answering.
WORKER_SHARED_SECRET=paste-the-value-from-the-StandIn-portal
```

Then run it:

```bash
elevenlabs-msteams-bridge
```

The bridge listens on **`:8080`** (override with `PORT`) and binds `0.0.0.0`. It takes the **last
path segment** as the call id, so it accepts whatever path you register in the portal - StandIn
appends `/{callId}` per call. Expose the port with a tunnel and register the public `wss://` URL as
your identity's **Agent voice URL** in the StandIn dashboard, never the local `ws://` bind.

Your ElevenLabs agent's audio input **and** output format must be **PCM 16000 Hz** - the bridge ends
the call with a clear error if the agent negotiates anything else.

## Embed

```python
import asyncio
from elevenlabs_msteams_bridge import load_config, start_server

async def main():
    server = await start_server(load_config())
    await asyncio.Event().wait()  # run until cancelled

asyncio.run(main())
```

Pass your own async `vision` callable to answer the agent's `look` tool with any model you like -
the raw frame never leaves your process:

```python
async def describe(frame: dict, question: str) -> str:
    ...  # call your vision model with frame["dataBase64"] / frame["mime"]
    return "a person holding a badge"

server = await start_server(load_config(), vision=describe)
```

A complete runnable project lives in [`examples/basic-bridge/`](./examples/basic-bridge/).

## Configuration

Every setting is an environment variable, and [`.env.example`](./.env.example) ships fully commented
with the package.

**[Configuration reference](https://komaa-com.github.io/elevenlabs-msteams-bridge-py/configuration-reference/)**
documents all of them: what each does, its default, and when to change it.

Three that catch people out:

- `WORKER_SHARED_SECRET` must byte-match the pairing secret from the StandIn portal. A mismatch is
  rejected with 401, which from the caller's side looks like the bot simply never answering.
- The agent's audio in/out format **must** be `pcm_16000` (agent settings). The bridge validates the
  conversation metadata at call start and ends the call on mismatch - anything else means garbled
  audio.
- `conversation_config_override` fields (first message, prompt, voice) are **rejected by ElevenLabs
  unless allowlisted** in the agent's security settings.

## Endpoints

- `GET /healthz` - liveness.
- `GET /metrics` - Prometheus counters (calls, rejections, relayed/dropped frames).
- `GET /{...}/{callId}` + WebSocket upgrade - the worker wire, HMAC-signed with
  `X-StandIn-Timestamp` / `X-StandIn-Signature` over
  `"{timestampMs}.{callId}"`.

Notes for operators:

- `/healthz` and `/metrics` are **unauthenticated** (only the WebSocket upgrade is HMAC-gated).
  They expose no call content, just liveness and counters, but if you would rather not leak call
  volumes, keep the port behind your ingress/tunnel rules.
- One bridge process serves **one agent id** (`ELEVENLABS_AGENT_ID`). Run one process per agent if
  you route multiple agents.

## Vision and recording

The `look` tool prefers your `VISION_API_URL` endpoint: the frame is described transiently and only
the **text** enters the conversation. Without one, the bridge falls back to uploading the frame to
ElevenLabs (multimodal) - that persists the frame with a third party, so it is only allowed while
Teams recording is active. Note that even path-2 descriptions become ElevenLabs conversation
content, which ElevenLabs retains per your agent's settings; enable the agent's zero-retention mode
if callers' surroundings must not be stored.

## Roadmap

Where the bridge stands today, and what each of these needs to move:

- **Group-call gate**: the Node.js sibling can withhold the agent's audio in a meeting until a caller
  addresses it by name (`GROUP_CALL_REQUIRE_ADDRESS`, `GROUP_CALL_WAKE_PHRASES`). Not implemented
  here yet, so in a group call this bridge relays the agent's audio the whole time.
- **Ambient vision**: `look` is on demand, so the agent has to decide to look. Continuous awareness
  (each scene change described once and injected as non-interrupting context) is not implemented
  here yet.
- **Vision spend cap**: there is no per-call sliding-window ceiling on paid vision calls
  (`MAX_VISION_PER_MINUTE`). Budget at your vision endpoint instead until it lands.
- **No-answer reaper**: a worker can authenticate, send `session.start`, and then never get a live
  relay. The Node.js sibling ends such a call after `STALE_CALL_REAPER_SECONDS`; here the dead-peer
  timer only fires on silence, so a worker that keeps heartbeating holds the call id and a
  connection slot.
- **Viseme lip-sync**: the bridge relays audio without estimating a viseme timeline, so the Teams
  tile shows no mouth movement synchronized to the agent's speech.

## License

MIT (c) Komaa DigiTech
