Metadata-Version: 2.4
Name: livekit-plugins-clevrlabs
Version: 0.1.4
Summary: LiveKit Agents TTS plugin for the Clevr Labs conversational speech model
Project-URL: Homepage, https://theclevr.com
License: Copyright (c) 2026 Clevr Labs. All rights reserved.
        
        This software and associated documentation files (the "Software") are the
        proprietary and confidential property of Clevr Labs. Unauthorized copying,
        distribution, modification, or use of the Software, in whole or in part,
        is strictly prohibited without the express prior written permission of
        Clevr Labs.
        
        The Software is provided to authorized users solely for use with the
        Clevr Labs API service. No license, express or implied, is granted to any
        intellectual property rights in the Software.
License-File: LICENSE
Requires-Python: >=3.11
Requires-Dist: httpx>=0.28
Requires-Dist: livekit-agents>=1.4
Requires-Dist: num2words>=0.5
Requires-Dist: numpy>=1.24
Requires-Dist: scipy>=1.10
Description-Content-Type: text/markdown

# livekit-plugins-clevrlabs

A [LiveKit Agents](https://docs.livekit.io/agents/) TTS plugin for the Clevr Labs conversational speech model. It sends text to the Clevr Labs voice servers and streams audio back — no ML code runs on your machine, everything happens on Clevr's servers.

## What it does

You drop it into a LiveKit agent in place of any other TTS provider. Your agent sends text, Clevr sends back audio. The model keeps the same voice across the whole conversation as long as you pass user audio context after each user turn (see below).

## Install

```bash
uv add livekit-plugins-clevrlabs
```

You'll also need the LiveKit Agents SDK and a few plugins for STT/LLM/VAD:

```bash
uv add "livekit-agents>=1.4" livekit-plugins-groq livekit-plugins-openai livekit-plugins-silero python-dotenv numpy
```

## Quick start

```python
from livekit.plugins import clevrlabs

tts = clevrlabs.TTS(api_key="clevr_...")

# Wire into a LiveKit AgentSession:
session = AgentSession(tts=tts, ...)
```

By default the plugin talks to the hosted Clevr Labs API at `https://api.theclevr.com`. Pass `server_url=...` only if you're pointing at a different endpoint.

## Complete example

Save as `agent.py`. This is a full, runnable LiveKit voice agent that uses Clevr for TTS:

```python
import asyncio
import os

import numpy as np
from dotenv import load_dotenv
from livekit import rtc
from livekit.agents import Agent, AgentSession, JobContext, WorkerOptions, cli
from livekit.plugins import clevrlabs, groq, openai as lk_openai, silero

load_dotenv()


async def entrypoint(ctx: JobContext):
    await ctx.connect()

    # 1. Create the TTS plugin
    tts_plugin = clevrlabs.TTS(api_key=os.environ["CLEVR_API_KEY"])
    ctx.add_shutdown_callback(tts_plugin.aclose)

    # 2. Buffer user audio while they're speaking (used for voice context)
    audio_buffer: list[np.ndarray] = []
    user_is_speaking = False

    async def _capture_user_audio(track: rtc.Track):
        stream = rtc.AudioStream.from_track(track=track, sample_rate=48000, num_channels=1)
        async for event in stream:
            if not user_is_speaking:
                continue
            frames = np.array(event.frame.data, dtype=np.int16).astype(np.float32) / 32768.0
            audio_buffer.append(frames)

    @ctx.room.on("track_subscribed")
    def on_track_subscribed(track, publication, participant):
        if track.kind == rtc.TrackKind.KIND_AUDIO:
            asyncio.ensure_future(_capture_user_audio(track))

    # 3. Wire up the agent session (swap STT/LLM for whatever providers you use)
    session = AgentSession(
        stt=groq.STT(model="whisper-large-v3-turbo", language="en"),
        llm=lk_openai.LLM(model="gpt-4o-mini"),
        tts=tts_plugin,
        vad=silero.VAD.load(),
    )

    # 4. Track when the user is speaking so we know what audio to buffer
    @session.on("user_state_changed")
    def _on_user_state(ev):
        nonlocal user_is_speaking
        user_is_speaking = (ev.new_state == "speaking")

    # 5. After each user turn, send the audio + transcript to the TTS server as context
    @session.on("user_input_transcribed")
    def _on_transcript(ev):
        if ev.is_final and ev.transcript and audio_buffer:
            audio_np = np.concatenate(audio_buffer)
            audio_buffer.clear()
            tts_plugin.add_user_turn(text=ev.transcript, audio=audio_np, sample_rate=48000)

    await session.start(
        agent=Agent(instructions="You are a helpful voice assistant."),
        room=ctx.room,
    )


if __name__ == "__main__":
    cli.run_app(WorkerOptions(entrypoint_fnc=entrypoint))
```

## Set up your environment

Create a `.env` file next to `agent.py`:

```env
# LiveKit Cloud — get from https://cloud.livekit.io
LIVEKIT_URL=wss://your-project.livekit.cloud
LIVEKIT_API_KEY=APIxxxxxxxxx
LIVEKIT_API_SECRET=your-secret

# STT — Groq has a free tier for Whisper
GROQ_API_KEY=gsk_...

# LLM — OpenAI (swap for any provider supported by livekit-plugins-openai)
OPENAI_API_KEY=sk-...

# Clevr TTS
CLEVR_API_KEY=clevr_...
```

## Run it

```bash
python agent.py dev
```

Then connect a LiveKit room (e.g. [agents-playground.livekit.io](https://agents-playground.livekit.io)) and talk to your agent.

## Notes on the example

- **`add_user_turn` requires both `text` and `audio`.** Passing only one will cause the voice to drift over the conversation. The agent's own turns are handled automatically — `clevrlabs.TTS` appends each synthesized response to its context for you.
- **Feed clean transcripts into `add_user_turn`.** Since user text is paired with real audio in the model's context, low-quality STT output can drift the voice over a long conversation. If you use a Whisper-family STT, filter its known phantom transcripts (e.g. `"Thanks for watching!"` on silent audio) before calling `add_user_turn`.

## Get an API key

[theclevr.com](https://theclevr.com)
</content>
</invoke>
