Metadata-Version: 2.1
Name: estuary-sdk
Version: 0.8.0
Summary: Python SDK for the Estuary real-time AI conversation platform
Author-Email: Estuary <team@estuary-ai.com>
License: MIT
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Framework :: AsyncIO
Project-URL: Homepage, https://estuary-ai.com
Project-URL: Documentation, https://docs.estuary-ai.com
Project-URL: Repository, https://github.com/estuary-ai/estuary-python-sdk
Requires-Python: >=3.11
Requires-Dist: python-socketio[asyncio_client]<6,>=5.11
Requires-Dist: aiohttp<4,>=3.9
Provides-Extra: audio
Requires-Dist: sounddevice<1,>=0.4; extra == "audio"
Requires-Dist: numpy<3,>=1.24; extra == "audio"
Provides-Extra: livekit
Requires-Dist: livekit<2,>=1.0; extra == "livekit"
Provides-Extra: all
Requires-Dist: estuary-sdk[audio,livekit]; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23; extra == "dev"
Requires-Dist: black; extra == "dev"
Requires-Dist: isort; extra == "dev"
Requires-Dist: flake8; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Description-Content-Type: text/markdown

# Estuary Python SDK

Python SDK for the [Estuary](https://estuary-ai.com) real-time AI conversation platform. Supports text chat, streaming voice (WebSocket and LiveKit WebRTC), vision, and memory.

## Installation

```bash
pip install estuary-sdk
```

Install with optional extras:

```bash
pip install estuary-sdk[audio]    # Microphone recording + speaker playback
pip install estuary-sdk[livekit]  # LiveKit WebRTC voice
pip install estuary-sdk[all]      # Everything
```

Or with PDM:

```bash
pdm install              # Core only
pdm install -G audio     # + audio
pdm install -G livekit   # + LiveKit
pdm install -G all       # Everything
```

Requires Python 3.11+.

## Getting Your Credentials

To use the SDK you need an **API key** and a **character ID** from the [Estuary Dashboard](https://app.estuary-ai.com):

1. **Sign up or log in** at [app.estuary-ai.com](https://app.estuary-ai.com)
2. **Create a character** — go to **Characters** and click **Create Character**. Configure your character's name, personality, and voice, then save.
3. **Copy the character ID** — on the character's page, copy the UUID shown under the character name (or from the URL).
4. **Generate an API key** — go to **Settings → API Keys** and click **Create Key**. Copy the key (it starts with `est_`).

Use these values for `api_key` and `character_id` in the examples below.

## Quick Start

```python
import asyncio
from estuary_sdk import EstuaryClient, EstuaryConfig, BotResponse

async def main():
    config = EstuaryConfig(
        server_url="https://api.estuary-ai.com",
        api_key="est_...",
        character_id="your-character-uuid",
        player_id="player-1",
    )

    async with EstuaryClient(config) as client:
        client.on("bot_response", lambda r: print(r.text, end="" if not r.is_final else "\n"))

        await client.connect()
        client.send_text("Hello!")

        await asyncio.sleep(5)  # Wait for response

asyncio.run(main())
```

## Voice

### Continuous Mode

Audio streams continuously. The server uses VAD (voice activity detection) to detect turn boundaries.

```python
from estuary_sdk import VoiceMode

await client.start_voice(VoiceMode.CONTINUOUS)

# Send raw PCM16 audio (16-bit signed, mono, 16kHz)
await client.send_audio(pcm_bytes)

await client.stop_voice()
```

### Push-to-Talk

You control when audio is captured and when the turn ends.

```python
await client.start_voice(VoiceMode.PUSH_TO_TALK)

await client.start_recording()
await client.send_audio(pcm_bytes)
await client.stop_recording()  # Triggers end-of-turn

await client.stop_voice()
```

### With Microphone (requires `audio` extra)

```python
from estuary_sdk.audio import AudioRecorder, AudioPlayer

recorder = AudioRecorder(on_audio=client.send_audio)
player = AudioPlayer()

client.on("bot_voice", player.enqueue)

await client.start_voice()
await recorder.start()
```

### VoiceSession

`VoiceSession` is a high-level wrapper that owns the `AudioPlayer`, `AudioRecorder`, and event handlers as an async context manager:

```python
from estuary_sdk import VoiceSession, VoiceMode

async with EstuaryClient(config) as client:
    await client.connect()

    async with VoiceSession(client, mode=VoiceMode.CONTINUOUS) as session:
        # Microphone and speaker are wired up, voice is active.
        await asyncio.Event().wait()
```

For push-to-talk, control recording within the session:

```python
async with VoiceSession(client, mode=VoiceMode.PUSH_TO_TALK) as session:
    await session.start_recording()   # User presses button
    await asyncio.sleep(3)
    await session.stop_recording()    # User releases button
```

Requires the `audio` extra (`pip install estuary-sdk[audio]`).

### Utilities

```python
client.toggle_mute()                        # Toggle microphone mute
print(client.is_muted)                      # Check mute state
print(client.is_voice_active)               # Check if voice is running
client.interrupt()                          # Interrupt current bot response
client.notify_audio_playback_complete()     # Tell server playback finished

# Script a specific line (skips LLM, goes straight to TTS)
client.say_line("Welcome back, friend.")
client.say_line("Thinking...", text_only=True)  # No TTS audio, just chat history
```

## Vision

Send a camera image for multimodal VLM processing:

```python
client.send_camera_image(image_base64, "image/jpeg")

# Respond to server-initiated capture requests
client.on("camera_capture_request", lambda req: client.send_camera_image(b64, "image/jpeg", request_id=req.request_id))
```

## Streaming Responses

Bot responses stream token-by-token. Use `is_final` to detect the complete message.

```python
def on_bot_response(response: BotResponse):
    if response.is_final:
        print(f"[{response.message_id}] {response.text}")
    else:
        print(response.text, end="", flush=True)

client.on("bot_response", on_bot_response)
```

## Send and Wait

For simple request-response flows, `send_text_and_wait()` sends a message and returns the final `BotResponse` directly — no manual event wiring needed:

```python
response = await client.send_text_and_wait("What is the capital of France?")
print(response.text)  # "The capital of France is Paris."

# With options
response = await client.send_text_and_wait(
    "Summarize our conversation.",
    text_only=True,   # Suppress voice response
    timeout=30.0,     # Max seconds to wait (default: 20)
)
```

Raises `asyncio.TimeoutError` if no final response arrives within the timeout. The listener is cleaned up automatically whether the call succeeds or times out.

## Memory

The REST memory API is available after `connect()` via `client.memory`.

```python
# Semantic search (limit is 1 to 50, server default 10)
found = await client.memory.search("favorite color", limit=5)
for hit in found.results:
    print(hit.memory_id, f"{hit.similarity:.2f}", hit.content)

# List with filters
memories = await client.memory.get_memories(memory_type="preference", limit=20)

# Other endpoints
stats = await client.memory.get_stats()
facts = await client.memory.get_core_facts()
timeline = await client.memory.get_timeline(group_by="week")
graph = await client.memory.get_graph()

# Delete all memories
result = await client.memory.delete_all(confirm=True)
print(result.deleted_count, result.message)
```

Memory, player, character generation and 3D model calls all use the canonical `/api/v1/characters/...` routes. `search()` sends `POST` with a JSON body and each hit carries `memory_id`, `similarity` and `content`; earlier releases sent `GET ?q=`, which the gateway always answered with 405.

Real-time memory extraction events:

```python
from estuary_sdk import MemoryUpdatedEvent

def on_memory(event: MemoryUpdatedEvent):
    print(f"Extracted {event.memories_extracted} memories")

client.on("memory_updated", on_memory)
```

## Characters

Manage characters via `client.characters` (available immediately — no `connect()` required):

```python
char = await client.characters.create("My Character", personality="Friendly helper")
chars = await client.characters.list()
char = await client.characters.get("character-uuid")
char = await client.characters.update("character-uuid", name="New Name")
await client.characters.delete("character-uuid")
```

New characters default to Inworld `inworld-tts-2-flash`. For the Inworld TTS-2
family (`inworld-tts-2` and `inworld-tts-2-flash`), pass the optional
`tts_delivery_mode` to `create()` or `update()`:

```python
char = await client.characters.create("Narrator", tts_delivery_mode="BALANCED")
char = await client.characters.update(char.id, tts_delivery_mode="CREATIVE")
```

Values are `STABLE`, `BALANCED`, or `CREATIVE` (case-insensitive on input). The
response returns the uppercase value, or `None` when unset. Other TTS models
ignore this setting.

## Players

Query player data via `client.players` (available after `connect()`):

```python
stats = await client.players.get_stats()
players = await client.players.list(limit=20)
player = await client.players.get("player-123")
messages = await client.players.get_messages("player-123")
deleted = await client.players.delete("player-123")
print(deleted["message"])  # "Player conversation deleted"; server answers 204
```

## Character Generation

Generate a character from a photo via `client.generate` (available immediately):

```python
with open("photo.jpg", "rb") as f:
    character = await client.generate.image_to_character(f.read())

# Start the animated humanoid pipeline (omit rigged=True for a static model)
started = await client.generate.generate_model(character.id, rigged=True)
print(started["modelStatus"])  # "posing"

# Poll 3D model progress
status = await client.generate.get_model_status(character.id)
print(status.model_status, f"{status.progress}%", status.rigged)
```

Use `wait_for_model()` to poll until the 3D model is ready (or fails), with an optional progress callback:

```python
from estuary_sdk import ModelStatus

status = await client.generate.wait_for_model(
    character.id,
    poll_interval=2.0,   # Seconds between polls (default: 2)
    timeout=600.0,       # Rigged generation can take several minutes (default: 300)
    on_progress=lambda s: print(f"{s.model_status} {s.progress}%"),
)
print("Final:", status.model_status, status.model_url)
if status.rigged:
    print("Animation clips:", status.animations)
```

`rigged=True` works for humanoid characters and produces one skinned GLB with
animation clips. `ModelStatus.animations` contains full glTF clip names such as
`preset:biped:idle`. The character action name is the suffix (`idle`). If
`wait_for_model()` returns `rig_failed` or `animation_failed`, check `model_url`:
the generated static model may still be usable. The default `generate_model(id)`
continues to request a static model.

## Agent-to-Agent Conversations

Start a conversation between two AI characters and observe the exchange in real-time:

```python
from estuary_sdk import AgentTurnText, AgentTurnComplete, AgentConversationComplete

client.on("agent_turn_text", lambda t: print(f"[Agent {t.agent_id}] {t.text}", end="" if not t.is_final else "\n"))
client.on("agent_turn_complete", lambda t: print(f"--- Turn {t.turn_number} complete ---"))
client.on("agent_conversation_complete", lambda c: print(f"Done after {c.total_turns} turns: {c.reason}"))

client.start_agent_conversation(
    agent_a_id="character-uuid-1",
    agent_b_id="character-uuid-2",
    conversation_context="Discuss the meaning of life.",
    max_turns=8,
    timeout_seconds=90,
)

# To stop early:
client.stop_agent_conversation()
```

## CLI Tester

An interactive chat program is included for quick testing:

```bash
python examples/chat.py --api-key [API_KEY] --character-id [CHARACTER_ID]
```

Or via PDM:

```bash
pdm run chat --api-key [API_KEY] --character-id [CHARACTER_ID]
```

| Flag | Short | Default | Description |
|------|-------|---------|-------------|
| `--api-key` | `-k` | *required* | API key |
| `--character-id` | `-c` | *required* | Character UUID |
| `--server-url` | `-s` | `https://api.estuary-ai.com` | Server URL |
| `--player-id` | `-p` | `python-sdk-tester` | Player ID |
| `--debug` | `-d` | off | Verbose logging |
| `--text-only` | `-t` | off | Suppress voice responses |

In-chat commands: `/quit`, `/voice`, `/stop`, `/memories`, `/help`

## Configuration

`EstuaryConfig` fields:

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `server_url` | `str` | *required* | Server URL |
| `api_key` | `str` | *required* | API key |
| `character_id` | `str` | *required* | Character UUID |
| `player_id` | `str` | *required* | Player identifier |
| `audio_sample_rate` | `int` | `16000` | Audio sample rate (Hz) |
| `auto_reconnect` | `bool` | `True` | Auto-reconnect on disconnect |
| `max_reconnect_attempts` | `int` | `5` | Max reconnection attempts |
| `reconnect_delay` | `float` | `1.0` | Initial reconnect delay (seconds) |
| `debug` | `bool` | `False` | Enable debug logging |
| `voice_transport` | `str` | `"websocket"` | `"websocket"`, `"livekit"`, or `"auto"` |
| `realtime_memory` | `bool` | `False` | Enable realtime memory updates |
| `capabilities` | `SessionCapabilities \| None` | `None` | Device capability declaration (see below). `None` sends all-true. |

### Capability Declaration

Tell the server what the host device can physically do. The server hides tools that require a
missing capability — e.g. `request_camera_image` is suppressed when `camera=False`, so the
character never asks a headless integration to look through a camera it doesn't have.

Declare device capability, not current usage mode: a voice-capable client sets
`microphone=True` even while it is being used for text.

```python
from estuary_sdk import EstuaryConfig, SessionCapabilities

config = EstuaryConfig(
    server_url=..., api_key=..., character_id=..., player_id=...,
    capabilities=SessionCapabilities(
        camera=False,      # Headless service — no camera to capture from
        microphone=True,
        speaker=True,
    ),
)
```

All fields default to `True`. Omitting `capabilities` entirely sends
`{"version": "1", "camera": True, "microphone": True, "speaker": True}` — identical to the
server's own default for clients that declare nothing, so upgrading changes no behavior.

## Events

| Event | Payload | Description |
|-------|---------|-------------|
| `connected` | `SessionInfo` | Connected and authenticated |
| `disconnected` | `str` | Disconnected (reason) |
| `reconnecting` | `int` | Reconnection attempt number |
| `connection_state_changed` | `ConnectionState` | State transition |
| `bot_response` | `BotResponse` | Streaming text response |
| `bot_voice` | `BotVoice` | Audio chunk (base64) |
| `stt_response` | `SttResponse` | Speech-to-text result |
| `interrupt` | `InterruptData` | Bot response interrupted |
| `client_action` | `ClientActionData` | Character performed a typed in-world action (fires once per call, on arrival) |
| `quota_exceeded` | `QuotaExceededData` | Usage quota hit |
| `camera_capture_request` | `CameraCaptureRequest` | Server requests a camera image |
| `voice_started` | — | Voice session started |
| `voice_stopped` | — | Voice session stopped |
| `voice_error` | `str` | Voice session error message |
| `audio_received` | `bytes` | Raw PCM audio from LiveKit transport |
| `memory_updated` | `MemoryUpdatedEvent` | New memories extracted |
| `agent_turn_text` | `AgentTurnText` | Streaming text from an agent-to-agent turn |
| `agent_turn_voice` | `AgentTurnVoice` | Streaming voice audio from an agent-to-agent turn |
| `agent_turn_complete` | `AgentTurnComplete` | An agent finished its turn |
| `agent_conversation_complete` | `AgentConversationComplete` | Agent-to-agent conversation finished |
| `livekit_connected` | `str` | LiveKit room name |
| `livekit_disconnected` | — | LiveKit disconnected |
| `error` | `Exception` | Error occurred |
| `auth_error` | `str` | Authentication failed |

Register listeners with `client.on()`, `client.once()`, or `client.off()`:

```python
client.on("bot_response", handle_response)     # Persistent listener
client.once("connected", handle_first_connect)  # One-time listener
client.off("bot_response", handle_response)     # Remove listener
```

Both sync and async callbacks are supported.

## Error Handling

```python
from estuary_sdk import EstuaryError, ErrorCode

try:
    await client.connect()
except EstuaryError as e:
    print(e.code)     # ErrorCode enum
    print(e.details)  # Optional extra info
```

Error codes:

| Code | Description |
|------|-------------|
| `CONNECTION_FAILED` | Could not connect to server |
| `AUTH_FAILED` | Authentication rejected |
| `CONNECTION_TIMEOUT` | Connection timed out |
| `QUOTA_EXCEEDED` | Usage quota exceeded |
| `VOICE_NOT_SUPPORTED` | Voice not supported |
| `VOICE_ALREADY_ACTIVE` | Voice session already running |
| `VOICE_NOT_ACTIVE` | No active voice session |
| `VOICE_START_FAILED` | Voice session failed to start |
| `LIVEKIT_UNAVAILABLE` | LiveKit dependency not installed |
| `NOT_CONNECTED` | Not connected to server |
| `REST_ERROR` | REST API request failed |
| `UNKNOWN` | Unknown error |

REST failures raise `EstuaryError` with `REST_ERROR`, and `e.details` holds `{"status", "body"}`. An API key that lacks the scope a route needs (for example `memories:read`) gets status 403 with a body of `{"detail": {"error": "insufficient_scope", "requiredScope": "..."}}`. Keys created before scopes existed hold every scope.

Every REST request carries `X-Estuary-Client: estuary-python-sdk/<version>` (also available as `estuary_sdk.__version__`) so the gateway can attribute traffic by SDK version.

## Optional Dependencies

| Extra | Packages | Purpose |
|-------|----------|---------|
| `audio` | sounddevice, numpy | Microphone capture + speaker playback |
| `livekit` | livekit | LiveKit WebRTC voice transport |
| `all` | All of the above | Everything |
| `dev` | pytest, black, isort, flake8, mypy | Development tools |

## Development

```bash
cd estuary-python-sdk
pdm install -G dev

pdm run test          # pytest
pdm run format        # black
pdm run lint          # flake8
pdm run sort-imports  # isort
pdm run typecheck     # mypy (strict)
```

## Restricted end-user credentials (0.8.0)

Keep your Estuary developer key on your backend. Verify the user's account,
profile and allowed characters there, then request `/api/v1/client-tokens`.
Supply `EstuaryConfig.token_provider`, an async callback returning
`ClientCredential(token, player_id, expires_at)`. It uses your own backend's
headers/cookies; those credentials are not sent to Estuary. Set `api_key=""`;
the returned canonical player ID overrides the configured player.

The callback is shared by Socket.IO, REST and multipart uploads. The SDK caches
credentials, renews the active socket before expiry, rejects profile changes,
and releases voice after expiration/revocation or failed renewal. Changing
profiles requires a new client. The backend must retain and renew the same
Estuary session ID for an active call; changing operations requires reconnecting.

Restricted grants cover character/player data and permitted text/voice/vision.
Administrative routes, share creation and encounters remain unavailable to these
tokens. Existing trusted API-key flows are unchanged. See
[the authenticated backend example](examples/client_tokens.py).

Install this flow with `python -m pip install --upgrade "estuary-sdk>=0.8.0"`.
It requires a gateway with the client-token endpoints.
