Metadata-Version: 2.4
Name: vsip-sdk
Version: 0.2.1
Summary: Official Python SDK for the VSIP Voice Events API — real-time turn detection, identity-gated barge-in, and speaker verification for voice agents.
Author: VSIP
License-Expression: MIT
Project-URL: Documentation, https://api.vsip.online/docs
Project-URL: Changelog, https://api.vsip.online/docs#sdk
Keywords: voice,voice-agent,barge-in,speaker-verification,turn-detection,vad,websocket,pipecat,realtime
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Classifier: Framework :: AsyncIO
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: websockets>=12.0
Provides-Extra: audio
Requires-Dist: numpy>=1.24; extra == "audio"
Provides-Extra: pipecat
Requires-Dist: pipecat-ai<2,>=1.5; extra == "pipecat"
Provides-Extra: livekit
Requires-Dist: livekit-agents<2,>=1.0; extra == "livekit"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: numpy>=1.24; extra == "dev"
Dynamic: license-file

# VSIP Python SDK

Real-time voice events for AI voice agents: semantic turn detection,
identity-gated barge-in, and speaker verification over one WebSocket.

```bash
pip install vsip-sdk            # once published; for now: pip install -e sdk/python
```

## 60-second integration

```python
import asyncio
from vsip import VSIPClient, VoiceSession, Turn, int16_to_wav

client = VSIPClient(api_key="vsip_...", url="wss://api.vsip.online/v1/stream")

async def on_turn(turn: Turn):
    if not turn.should_respond:
        return                                  # verified: not your enrolled user
    text = await my_stt(int16_to_wav(turn.audio))   # STT-ready WAV slice
    reply = await my_llm(text, interrupted=turn.interrupted_agent)
    await session.speak_gate()                  # never talk over an open turn
    await client.agent_speaking()               # arm barge-in detection
    await my_tts_play(reply)                    # stop this in on_barge_in
    await client.agent_idle()

def on_barge_in(event):
    my_tts_stop()                               # session already sent agent_idle

session = VoiceSession(client, on_turn=on_turn, on_barge_in=on_barge_in)

async def main():
    async with client:
        await session.start()
        async for frame in my_mic_frames():     # 16kHz mono int16 PCM
            await session.send_audio(frame)

asyncio.run(main())
```

## What `VoiceSession` does for you

These behaviors are the recommended protocol usage (each one exists because
skipping it caused a real failure in field testing):

| Behavior | Why |
|---|---|
| Turn audio includes **onset lookback** (1.0s / 1.5s for barge-ins) | `turn_start` arrives 1–2 ticks after real speech onset — naive slicing loses first syllables |
| Post-lock turns are **held for `speaker_verification`** (12s cap) | the live `speaker_id` on `turn_end` can be a whole turn stale |
| Pre-lock turns dispatch immediately | first-conversation UX — the first sentence enrolls AND gets answered |
| `on_barge_in` + automatic `set_agent_state idle` | TTS must stop within ~250ms of the event |
| `speak_gate()` | starting TTS over an open user turn makes the server treat the tail of that turn as a barge-in |
| Turn handlers are queued, never dropped | a barge-in mid-response must not discard the interrupting utterance |
| Rolling buffer capped outside turns | an hour of silence costs no memory |

## Lower level

`VSIPClient` alone gives you: typed events (`async for event in client.events()`),
control commands (`agent_speaking/idle`, `set_barge_in_mode`, `enroll_speaker`,
`toggle_lock`, `reset`), tagged frames for `aec="server"`
(`send_mic_frame` / `send_tts_reference`), and transparent reconnect+resume
(sessions survive network blips ≤60s — enrollment and lock state included).

Unknown event types parse as `GenericEvent` and unknown fields are ignored:
the wire protocol is additive-only, so this SDK won't break as the API grows.

## Pipecat adapter

```bash
pip install "vsip-sdk[pipecat]"
```

```python
from vsip import VSIPClient
from vsip.adapters.pipecat import VSIPProcessor

vsip = VSIPProcessor(VSIPClient(api_key="vsip_...", url="wss://.../v1/stream"))

pipeline = Pipeline([
    transport.input(),
    stt,
    vsip,          # identity-gated interruption + verification gating
    context_aggr,
    llm,
    tts,
    transport.output(),
])
```

- **Identity-gated interruption**: VSIP's `barge_in` becomes a pipeline
  interruption — only your enrolled speaker can cut the bot off. Set
  `allow_interruptions=False` on the transport so VSIP is the sole authority.
- **Verification gating**: once locked, TranscriptionFrames are held until
  the `speaker_verification` verdict — impostor turns never reach your LLM.
  Each verdict is also emitted as a `VSIPVerificationFrame`.
- **Automatic TTS sync**: Bot speaking-state frames drive `set_agent_state`.

## LiveKit Agents plugin

```bash
pip install "vsip-sdk[livekit]"
```

```python
from livekit.agents import AgentSession
from vsip import VSIPClient
from vsip.adapters.livekit import VSIPVAD

vsip_vad = VSIPVAD(
    VSIPClient(api_key="vsip_...", url="wss://.../v1/stream"),
    on_verification=lambda v: None,   # optional: gate the agent's response
)

session = AgentSession(vad=vsip_vad, stt=..., llm=..., tts=...)
```

VSIP plugs in as the session's **VAD / turn detector**: `turn_start` and
verified `barge_in` become `START_OF_SPEECH`, `turn_end` becomes
`END_OF_SPEECH`. So VSIP's semantic endpointing and identity-gated barge-in
drive the agent's turn-taking — a cough or bystander can't interrupt. Use the
`on_verification` callback to additionally gate the agent's *response* on
`should_respond`.

## Twilio Media Streams bridge

Connect a phone call's audio to VSIP — VSIP ingests Twilio's μ-law natively.
The bridge is framework-agnostic (no extra dependency); wire it to your
FastAPI / Starlette / `websockets` endpoint:

```python
from vsip import VSIPClient
from vsip.adapters.twilio import TwilioBridge

@app.websocket("/twilio-stream")
async def twilio_stream(ws):
    await ws.accept()
    client = VSIPClient(api_key="vsip_...", url="wss://.../v1/stream",
                        audio_format="mulaw8k")
    bridge = TwilioBridge(client,
                          on_turn_end=lambda e: ...,
                          on_verification=lambda v: ...)   # gate on should_respond
    async with client:
        await bridge.run(ws.iter_text(), send=ws.send_text)
```

On `barge_in` the bridge sends Twilio a `clear` message — flushing buffered
playback so the caller's interruption stops your TTS instantly.

## Availability

| Target | Status |
|---|---|
| Python SDK (`VSIPClient`, `VoiceSession`) | ✅ available |
| Pipecat (`vsip-sdk[pipecat]`) | ✅ available |
| LiveKit Agents (`vsip-sdk[livekit]`) | ✅ available |
| Twilio Media Streams (`vsip.adapters.twilio`) | ✅ available |
| JavaScript / TypeScript (`@vsip/sdk`) | ✅ available |

## Examples

- [`examples/echo_agent.py`](examples/echo_agent.py) — minimal end-to-end agent
  loop against a local API with your microphone.
