# OpenRTC-Python: Strategic Audit for Multi-Session Density

**Date:** 2026-05-02
**Audit type:** Read-only discovery + architectural recommendation
**Goal:** Decide build-on vs replace approach for 30x density target (50+ sessions per worker process vs ~1 in livekit-agents at ~3 GB each)
**Source commit:** `3d44875` (main, clean tree)

---

## STEP 1: LiveKit Architecture (Verified Against Their Docs)

This section is verified against LiveKit's own documentation as of 2026-05-03 — not from memory.

### Process model (verified)

From `/agents/worker/job` ([live docs](https://docs.livekit.io/agents/server/job.md)):

> "When an agent server accepts a job request from LiveKit Cloud, it starts a new process and runs your agent code inside. **Each job runs in a separate process to isolate agents from each other.** If a session instance crashes, it doesn't affect other agents running on the same agent server."

From `/agents/worker/options`:

> "For isolation and performance reasons, the framework runs each agent job in its own process. Agents often need access to model files that take time to load. To address this, you can use a `prewarm` function to warm up the process before assigning any jobs to it. You can control the number of processes to keep warm using the `num_idle_processes` parameter."

So the user's premise is verified: **livekit-agents uses process-per-job by default**, and the only mechanism documented is keeping idle processes warm (the cost is paid per process).

### Critical undocumented finding: `JobExecutorType.THREAD`

LiveKit's source code reveals an option **not mentioned in the public docs**:

`livekit/agents/job.py`:
```python
@unique
class JobExecutorType(Enum):
    PROCESS = "process"
    THREAD = "thread"
```

`livekit/agents/worker.py:130`:
```python
if sys.platform.startswith("win"):
    _default_job_executor_type = JobExecutorType.THREAD
else:
    _default_job_executor_type = JobExecutorType.PROCESS
```

**THREAD mode exists** as a `ThreadJobExecutor` (`livekit/agents/ipc/job_thread_executor.py`). It is the **default on Windows** (because of fork/multiprocessing limitations) but not on Linux/macOS.

However, looking at `proc_pool.py` and `job_proc_lazy_main.py` more closely, **THREAD mode does not give us what we want**:

1. Each `ThreadJobExecutor` still creates its own `JobProcess` instance with separate `userdata`.
2. `setup_fnc` (the prewarm) is invoked once per thread executor, not once per real OS process.
3. Each thread has its own `socket.socketpair()` IPC channel and its own asyncio event loop.
4. When `setup_fnc` calls `silero.VAD.load()`, it produces a fresh model per thread.
5. Each thread creates its own `rtc.Room()` per job in `_JobProc._start_job` — meaning N threads → N WebRTC peer connections, just like N processes.

So even with `JobExecutorType.THREAD`, livekit-agents gives us **1 thread per session, with separate models per thread**. The "shared prewarm" benefit of THREAD mode is illusory unless we route around `JobProcess`/`setup_fnc` ourselves.

### Found example that proves the multi-session-per-process pattern works

`examples/other/transcription/multi-user-transcriber.py` in the agents repo demonstrates the pattern we want:

```python
class MultiUserTranscriber:
    def __init__(self, ctx: JobContext):
        self._sessions: dict[str, AgentSession] = {}

    async def _start_session(self, participant: rtc.RemoteParticipant) -> AgentSession:
        session = AgentSession(vad=self.ctx.proc.userdata["vad"])  # SHARED VAD
        await session.start(
            agent=Transcriber(participant_identity=participant.identity),
            room=self.ctx.room,  # SAME ROOM
            room_options=room_io.RoomOptions(participant_identity=participant.identity),
        )
        return session
```

This shows that **multiple `AgentSession`s can coexist in one job process, sharing prewarmed models and a single `rtc.Room`**. But the example is restricted to *multiple participants in a single LiveKit Room* — not *N independent calls in N independent Rooms*. To get N rooms in one process we have to bypass the `proc_pool` and run job entrypoints as coroutines in the same event loop.

### Worker WS protocol (replicable)

From `worker.py:_connection_task`, the worker-to-LiveKit protocol is:

1. **Connect**: aiohttp WebSocket to `<LIVEKIT_URL>/agent`
2. **Auth**: `Authorization: Bearer <jwt>` where the JWT has `agent=True` claim
3. **Register**: send protobuf `agent.WorkerMessage{register: {type, allowed_permissions, agent_name, version}}`
4. **Job dispatch**: server sends `agent.ServerMessage{availability: ...}`; worker responds with accept/reject
5. **Heartbeat**: 30s
6. **Status updates**: every 2.5s with current load (0..1)
7. **All wire messages**: protobuf from `livekit-protocol`

**This is fully replicable**, and `livekit-protocol` already ships the .proto definitions.

### `livekit-rtc` vs `livekit-agents` distinction

| Package (PyPI) | Import | Purpose | What's exposed |
|---|---|---|---|
| `livekit` (1.0.25) | `livekit.rtc` | Low-level WebRTC client | `Room`, `RemoteParticipant`, `LocalParticipant`, `Track`, `AudioStream`, `AudioFrame`, `RoomOptions`, `RtcConfiguration` |
| `livekit-api` (1.1.0) | `livekit.api` | Server-side REST/JWT helpers | `LiveKitAPI`, `AccessToken`, `VideoGrants`, `RoomServiceClient`, etc. |
| `livekit-protocol` (1.1.3) | `livekit.protocol` | Protobuf message definitions | `agent.WorkerMessage`, `agent.ServerMessage`, `agent.Job`, `models.Room`, etc. |
| `livekit-agents` (1.5.0) | `livekit.agents` | High-level framework | `AgentServer`, `AgentSession`, `JobContext`, `JobProcess`, `Agent`, prewarm, proc pool, job dispatch, IPC |

For the build-on vs replace decision:
- **`livekit` (rtc)** gives us WebRTC connectivity per session — we need this either way.
- **`livekit-api`** gives us JWTs and room mgmt — we need this either way.
- **`livekit-protocol`** gives us the wire format — we need this either way.
- **`livekit-agents`** gives us `AgentServer` (worker WS protocol), `proc_pool`, `JobContext`, `AgentSession` (the high-level voice loop with STT/LLM/TTS orchestration), prewarm, drain.

`AgentSession` is the high-value piece — replicating it would be ~3000 LOC of voice loop, turn detection, interruption handling, function tool calling, etc. We do NOT want to replace it. `AgentServer` and `proc_pool` are the **process-model** pieces we want to bypass.

---

## STEP 2: Code Audit

### A. Repository structure

```
openrtc-python/
├── AGENTS.md                    # AI coding guidelines (395 lines)
├── CONTRIBUTING.md
├── LICENSE                      # MIT
├── Makefile
├── README.md                    # Primary docs (12.3 KiB)
├── pyproject.toml               # Hatchling build, hatch-vcs versioning
├── uv.lock                      # 454 KiB
├── .github/workflows/           # 5 workflows (test, lint, publish, docs, deploy-docs)
├── .agents/skills/openrtc-python/  # Agent skill defs
├── .pre-commit-config.yaml
├── .env.example                 # Documents LIVEKIT_*, OPENAI_*, etc.
├── docs/                        # VitePress site
│   ├── api/pool.md              # API reference
│   ├── changelog.md             # Auto-updated by publish.yml
│   ├── cli.md                   # CLI reference
│   ├── concepts/architecture.md # Text-only, no diagram
│   ├── deployment/github-pages.md
│   ├── examples.md
│   ├── getting-started.md
│   └── index.md
├── examples/
│   ├── main.py                  # Programmatic AgentPool entry
│   ├── agents/
│   │   ├── dental.py            # 3-tool scheduling agent
│   │   └── restaurant.py        # 3-tool reservation agent
│   └── frontend/                # React Router demo (LiveKit Session APIs)
├── src/openrtc/                 # 14 files, 3,374 LOC
│   ├── __init__.py              # 6 public symbols (20 LOC)
│   ├── pool.py                  # Core: AgentPool, routing, discovery, serialisation (962 LOC)
│   ├── resources.py             # Runtime metrics, RSS, savings estimate (450 LOC)
│   ├── metrics_stream.py        # JSONL schema + JsonlMetricsSink (144 LOC)
│   ├── provider_types.py        # ProviderValue alias (13 LOC)
│   ├── cli.py                   # Lazy entrypoint (56 LOC)
│   ├── cli_app.py               # Typer commands + main() (368 LOC)
│   ├── cli_livekit.py           # argv stripping, env, lifecycle (310 LOC)
│   ├── cli_dashboard.py         # Rich rendering (382 LOC)
│   ├── cli_reporter.py          # Background metrics reporter (141 LOC)
│   ├── cli_params.py            # SharedLiveKitWorkerOptions (119 LOC)
│   ├── cli_types.py             # Annotated Typer aliases (226 LOC)
│   ├── tui_app.py               # Textual sidecar (180 LOC)
│   ├── _version.py              # Dead fallback (3 LOC)
│   └── py.typed                 # PEP 561 marker
└── tests/                       # 10 files, 124 tests, 2,999 LOC, ≥80% coverage
```

### Project type

**Hatchling** with **hatch-vcs** for git-tag-driven versioning. `pyproject.toml` declares:
- `requires-python = ">=3.11,<3.14"`
- `dependencies = ["livekit-agents[openai,silero,turn-detector]~=1.4"]`
- Optional extras: `cli` (rich, typer), `tui` (rich, typer, textual)
- Dev group: pytest, ruff, mypy, pre-commit, textual

### PyPI status

- Package name: **`openrtc`** (`pyproject.toml:2`)
- Latest release per `docs/changelog.md`: **v0.0.17** (2026-04-03)
- Install: `pip install openrtc` (library), `pip install 'openrtc[cli,tui]'` (full CLI + TUI)
- No git tags present in the local checkout (`git tag --list` returns empty); tags presumably live on the remote — `hatch-vcs` falls back to dev versions locally

### Full runtime dependency list (from `uv.lock`)

| Dep | Version | Purpose | Flag |
|---|---|---|---|
| `livekit-agents` | 1.5.0 | Framework: `AgentServer`, `AgentSession`, `JobContext`, prewarm, proc pool | **livekit-** |
| `livekit` | 1.0.25 | WebRTC client (`livekit.rtc`) | **livekit-** |
| `livekit-api` | 1.1.0 | Server REST + JWT (`livekit.api`) | **livekit-** |
| `livekit-protocol` | 1.1.3 | Protobuf wire definitions | **livekit-** |
| `livekit-blingfire` | 1.1.0 | Tokenization (used by turn detector) | **livekit-** |
| `livekit-plugins-openai` | 1.5.0 | OpenAI STT/LLM/TTS adapters | **livekit-** |
| `livekit-plugins-silero` | 1.5.0 | Silero VAD model | **livekit-** |
| `livekit-plugins-turn-detector` | 1.5.0 | Multilingual turn detection model | **livekit-** |
| (transitive) | — | aiohttp, protobuf, onnxruntime, opentelemetry, pydantic, etc. | |

Optional extras: `rich>=13`, `typer>=0.12`, `textual>=0.47,<2`.

### B. The connection layer

**This is the most important section: we are using livekit-agents directly. We have no thin WebSocket layer.**

Every WebSocket / connection touchpoint in our code goes through `livekit.agents`:

| File:Line | What we do | Layer |
|---|---|---|
| `pool.py:20` | `from livekit.agents import Agent, AgentServer, AgentSession, JobContext, JobProcess, cli` | **livekit-agents (high)** |
| `pool.py:284` | `self._server = AgentServer()` (no kwargs — accepts all defaults) | **livekit-agents (high)** |
| `pool.py:291` | `self._server.setup_fnc = partial(_prewarm_worker, ...)` | **livekit-agents (high)** |
| `pool.py:292` | `self._server.rtc_session()(partial(_run_universal_session, ...))` | **livekit-agents (high)** |
| `pool.py:127` | `room=ctx.room` (we pass through the rtc.Room) | passthrough |
| `pool.py:481` | `cli.run_app(self._server)` | **livekit-agents (high)** |
| `cli_types.py:168` | help text: `"WebSocket URL of the LiveKit server"` | metadata only |

**There are zero references to `livekit.rtc`, `livekit.api`, `livekit-protocol`, raw WebSocket, aiohttp, or any low-level connection plumbing in our `src/openrtc/`.** We are 100% on the livekit-agents side of the boundary.

### Current session model

**Process-per-session, inherited unchanged from livekit-agents.**

`pool.py:284`: `self._server = AgentServer()` is called with no arguments. So:
- `job_executor_type` defaults to `_default_job_executor_type`, which on Linux/macOS is `JobExecutorType.PROCESS`.
- `num_idle_processes` defaults to `min(math.ceil(cpu_count()), 4)` on prod, 0 on dev.
- For each accepted job, `proc_pool.launch_job(info)` grabs a warmed `ProcJobExecutor` (= a separate Python process) and runs the entrypoint there.

So **today, our `pool.run()` produces exactly the livekit-agents process-per-job model**, with our `_run_universal_session` running inside that subprocess. The only thing OpenRTC adds is *agent multiplexing within a single subprocess via routing* — i.e., one process can handle 3 agent types but only 1 concurrent session.

Memory math today:
- N agent types × M concurrent sessions
- Without OpenRTC: N worker fleets × M processes each = N×M processes
- With OpenRTC: 1 worker fleet × M processes = M processes
- **Per-session cost is unchanged** (one full Python process + Silero + turn detector + WebRTC peer + audio buffers per call)

The 30x density goal cannot be reached without changing the session model.

### C. The session lifecycle

`pool.py:107-138` `_run_universal_session(runtime_state, ctx)`:

1. Resolve target agent from job/room metadata (`pool.py:114`, `_resolve_agent_config`)
2. Build session kwargs with default turn handling (VAD interruption + multilingual turn detector) (`pool.py:115`, `_build_session_kwargs`)
3. Construct `AgentSession(stt, llm, tts, vad=ctx.proc.userdata["vad"], **session_kwargs)` (`pool.py:116`)
4. Record `session_started` metric (`pool.py:124`)
5. `await session.start(agent=config.agent_cls(), room=ctx.room)` (`pool.py:125`)
6. `await ctx.connect()` (`pool.py:129`)
7. If greeting configured: `await session.generate_reply(instructions=config.greeting)` (`pool.py:131`)
8. On any exception: `record_session_failure` (`pool.py:135`)
9. Always: `record_session_finished` (`pool.py:138`)

**Per-session state is held inside the `AgentSession`**, which is a livekit-agents object. We do not own the conversation history, audio buffers, or participant state — `AgentSession` does.

**Session ends** when: (a) the room disconnects (livekit-agents triggers shutdown via `_shutdown_fut`), (b) the user code calls `ctx.shutdown()`, or (c) an exception in our entrypoint propagates. We have `add_shutdown_callback` available but don't use it.

**Isolation between sessions today: full process isolation** (we inherit livekit-agents' default). A crash in one session terminates that process only, leaving sibling processes alive. This is exactly the "expensive isolation" we want to relax.

### D. The inference layer

#### Where STT/LLM/TTS plug in

`pool.py:116`:
```python
session: AgentSession = AgentSession(
    stt=config.stt,
    llm=config.llm,
    tts=config.tts,
    vad=ctx.proc.userdata["vad"],
    **session_kwargs,
)
```

We pass user-supplied provider objects *directly* to `livekit.agents.AgentSession`. We do **not** wrap them, and we do **not** define a plugin interface ourselves. The user passes whatever LiveKit accepts: a string like `"openai/gpt-4.1-mini"` (string passthrough) or a plugin instance like `openai.STT(model="gpt-4o-mini-transcribe")`.

`provider_types.py:13`:
```python
ProviderValue: TypeAlias = str | object  # effectively `Any` to type checkers
```

So today's plugin pattern is: *whatever livekit-agents accepts, we accept*. This is clean (no leaky abstraction) but also offers zero type safety and zero ability to add our own optimizations (e.g., a shared HTTP/2 client across sessions).

#### Per-session vs shared

| Component | Loaded | Shared across sessions? |
|---|---|---|
| Silero VAD (`silero.VAD.load()`) | Once at prewarm (`pool.py:103`) | Yes within a worker process — `proc.userdata["vad"]` is read by every session in that process. **But** since each session gets its own process, there's still one VAD per session in practice. |
| Multilingual turn detector | Class loaded at prewarm (`pool.py:104`); instances created per session via `proc.userdata["turn_detection_factory"]()` (`pool.py:690`) | Class loaded once, **instances created per session**. Plus each turn detector instance uses an inference subprocess from livekit-agents' `InferenceProcExecutor` — that's actually shared across the whole worker. |
| OpenAI STT/LLM/TTS | Plugin instance per agent registration | The provider object is shared if reused. But each `AgentSession` opens its own HTTP/WS connection to the provider. |
| `rtc.Room` (WebRTC peer) | Created per session in `JobContext._start_job` | Never shared. One WebRTC peer connection per call. |

#### Smallest example: swap Deepgram → AssemblyAI for STT

```python
# Current Deepgram:
from livekit.plugins import deepgram
pool.add("support", SupportAgent, stt=deepgram.STT(model="nova-3"))

# Swap to AssemblyAI:
from livekit.plugins import assemblyai
pool.add("support", SupportAgent, stt=assemblyai.STT())
```

That's it. Because we're a passthrough, the swap is a one-liner — but **all the leverage belongs to livekit-agents' plugin ecosystem**, not us. We add no provider abstraction.

### E. The developer-facing API

#### What `from openrtc import ...` exposes

`__init__.py:13-20`:
```python
__all__ = [
    "AgentConfig",
    "AgentDiscoveryConfig",
    "AgentPool",
    "ProviderValue",
    "__version__",
    "agent_config",
]
```

Six public symbols. Five types/functions plus the version.

#### Smallest user agent definition

```python
from livekit.agents import Agent
from livekit.plugins import openai
from openrtc import AgentPool

class SupportAgent(Agent):
    def __init__(self) -> None:
        super().__init__(instructions="Help callers with support questions.")

pool = AgentPool()
pool.add(
    "support",
    SupportAgent,
    stt=openai.STT(model="gpt-4o-mini-transcribe"),
    llm=openai.responses.LLM(model="gpt-4.1-mini"),
    tts=openai.TTS(model="gpt-4o-mini-tts"),
)
pool.run()
```

#### Side-by-side with livekit-agents

| | livekit-agents | OpenRTC |
|---|---|---|
| Agent class | `class Foo(Agent): ...` | `class Foo(Agent): ...` (identical) |
| Server | `server = AgentServer()` | `pool = AgentPool()` |
| Entrypoint | `@server.rtc_session()` | `pool.add("name", Foo, stt=..., llm=..., tts=...)` |
| Prewarm | `server.setup_fnc = my_prewarm` | (handled internally) |
| Run | `cli.run_app(server)` | `pool.run()` |

**Drop-in factor: ~85%.** The Agent class itself is unchanged. The differences are: we add a name+route registration step (you have N agents in one pool, livekit-agents has 1), and we hide prewarm. A user moving from livekit-agents to OpenRTC keeps every `Agent`/`@function_tool`/`RunContext` line as-is.

### F. Examples & tests

| Path | Summary |
|---|---|
| `examples/main.py` | Discovers agents from `./agents/`, runs `pool.run()` (~12 LOC) |
| `examples/agents/restaurant.py` | Restaurant Agent, 3 function tools, `@agent_config` decorator |
| `examples/agents/dental.py` | Dental Agent, 3 function tools, `@agent_config` decorator |
| `examples/frontend/` | React Router app with `/dentist` and `/restaurant` routes; uses LiveKit JS SDK; sets room metadata to dispatch the right agent |

**10 test files, 124 test functions, 2,999 LOC, ≥80% coverage enforced by CI** across Python 3.11/3.12/3.13.

`conftest.py` provides a complete shim of `livekit.agents` (Agent, AgentServer, AgentSession, JobContext, JobProcess, RunContext, function_tool) so tests can run without the real SDK installed. This is a **strong indicator** that nothing in our test suite actually exercises end-to-end voice flow — the shim has no audio, no WebRTC, no STT/LLM/TTS. All tests are unit tests of registration, routing, serialization, CLI argv handling, JSONL framing, RSS measurement, TUI rendering.

#### Hello world

```bash
pip install 'openrtc[cli]'
export LIVEKIT_URL=wss://your-project.livekit.cloud
export LIVEKIT_API_KEY=...
export LIVEKIT_API_SECRET=...
export OPENAI_API_KEY=sk-...

openrtc dev ./examples/agents \
  --default-stt openai/gpt-4o-mini-transcribe \
  --default-llm openai/gpt-4.1-mini \
  --default-tts openai/gpt-4o-mini-tts
```

---

## STEP 3: Build-On vs Replace — The Strategic Question

### 1. What does our current code already do?

**We are 100% built ON livekit-agents.** No replacement, not even partial. Specifically:

- We instantiate `AgentServer()` directly with no overrides (`pool.py:284`)
- We call `cli.run_app(self._server)` (`pool.py:481`)
- We use livekit-agents' `setup_fnc` and `@server.rtc_session()` decorator
- We pass `ctx.room` and `ctx.proc.userdata["vad"]` from livekit-agents straight to `AgentSession`
- We do not touch `livekit.rtc`, `livekit.api`, or `livekit-protocol` directly

What we add on top is **agent multiplexing**: one livekit-agents worker process can host N agent classes, dispatched per-session via metadata. That's it. The session model — process-per-job, ~3 GB per process — is unchanged.

### 2. Can we get multi-session-per-process WITHOUT replacing livekit-agents?

**Three options to evaluate**:

**Option A: `JobExecutorType.THREAD`** — change one line: `AgentServer(job_executor_type=JobExecutorType.THREAD)`.

What it gives us: All "job processes" become threads in the worker process. One Python interpreter instead of N.

What it doesn't give us:
- `setup_fnc` still runs once per ThreadJobExecutor → each thread loads its own VAD copy (not shared)
- Each thread still creates its own `rtc.Room()` per job (one WebRTC peer per session, unavoidable)
- Each thread has its own asyncio event loop running inside `_ProcClient.run()`
- GIL contention for any CPU work across threads
- Models must be made shareable manually (refactor `setup_fnc` to use a thread-local cache or move to a shared module-level singleton)

**Density estimate**: maybe 5-10x improvement. We lose the Python interpreter overhead (~80-150 MB per process) but pay for separate event loops, separate model copies (unless we hack the prewarm), and the GIL.

Effort: **2-3 days** to flip the flag, refactor prewarm to share a module-global VAD instance, and verify nothing in livekit-agents assumes per-thread isolation that breaks.

**Option B: Bypass `proc_pool` with a custom `JobExecutor`** — implement the `JobExecutor` Protocol from `ipc/job_executor.py` and make it run the job entrypoint as an asyncio task in the **main worker process's event loop** (not a thread, not a subprocess). This pattern is the same one `multi-user-transcriber.py` uses internally — we externalize it.

We'd still depend on `AgentServer` for: WS protocol, job dispatch, registration, drain, load reporting. We'd replace just the per-job execution with our own coroutine-based runner that calls our entrypoint with a shared `JobProcess` and a shared model dictionary.

Density estimate: 30-50x. One process, one event loop, N coroutines, shared everything.

Effort: **1-2 weeks**. The hard parts are:
- Implementing the `JobExecutor` Protocol correctly (start, initialize, launch_job, aclose, monitoring, status)
- Subclassing `AgentServer` (or monkey-patching `proc_pool.ProcPool` selection) to use our executor
- Sharing `JobProcess`/`userdata` across all coroutines
- Routing the `InferenceExecutor` IPC channel correctly (currently per-process)
- Handling crash isolation: an unhandled exception in one session must not take down siblings

**Option C: Replace `AgentServer` (worker WS protocol) entirely** — talk to LiveKit's worker WS endpoint ourselves using `livekit.rtc` + `livekit.api` + `livekit-protocol`.

Effort: **3-5 weeks**. The worker WS state machine is ~600 LOC in `worker.py` (registration, heartbeat, load reporting, status updates, drain, reconnect with retry, job acceptance dance). Replicable but non-trivial. We'd then build our own coroutine pool for jobs.

Benefit over Option B: full control, no dependency on internal livekit-agents protocols (`JobExecutor`, IPC duplex, proto messages between `proc_pool` and `_ProcClient`). We can also drop `InferenceProcExecutor` (which is its own subprocess) and run inference inline in the main loop.

### 3. What `livekit-rtc` primitives we need (if we replace)

**For the connection layer (worker side):**
- `livekit.api.AccessToken` + `VideoGrants(agent=True)` to mint the worker JWT
- `aiohttp.ClientSession.ws_connect("<url>/agent")` for the worker WS
- `livekit.protocol.agent.WorkerMessage` / `agent.ServerMessage` (protobuf) for register/availability/job assignment/ping/pong
- That's the full worker protocol.

**For the session layer (per call):**
- `livekit.rtc.Room()` + `room.connect(url, token, options)` per call (cannot be shared across calls — it's a WebRTC peer)
- `livekit.rtc.RoomOptions(auto_subscribe=True, rtc_config=...)`
- `livekit.api.AccessToken(...).with_identity(agent_identity).with_grants(VideoGrants(room_join=True, room=room_name, agent=True)).to_jwt()`

**What's hidden / would still need livekit-agents:**
- `AgentSession` (the voice loop): turn detection, interruption handling, STT/LLM/TTS streaming, function tool dispatch, ChatContext management, recording, telemetry — ~3000 LOC of orchestration we definitely should not replicate.
- All `livekit.plugins.*` (silero, openai, deepgram, etc.) — these are first-party LiveKit and we should keep using them as-is.

So even in "replace" mode, **we keep `livekit.agents.AgentSession` and the plugin ecosystem**. We replace only `AgentServer` + `proc_pool`.

### 4. Effort delta

| Approach | Effort to v0.1 | Density gain | Risk |
|---|---|---|---|
| **A: THREAD mode** (single line + prewarm refactor) | **2-3 days** | 5-10x | Low (using documented option) |
| **B: Custom `JobExecutor`** (bypass `proc_pool`, keep `AgentServer`) | **1-2 weeks** | 30-50x ✓ | Medium (depends on `JobExecutor` Protocol stability across livekit-agents versions) |
| **C: Replace `AgentServer`** (build our own worker) | **3-5 weeks** | 30-50x ✓ | High (we own the worker WS protocol, need to track LiveKit changes) |

### 5. Recommendation

**Start with Option B (custom `JobExecutor`), with a specific roadmap:**

**Phase 1 (week 1)** — Prove the density win:
- Subclass `AgentServer`, override `_proc_pool` instantiation to use a custom `CoroutinePool` we own
- Implement a `CoroutineJobExecutor` that satisfies the `JobExecutor` Protocol and runs job entrypoints as `asyncio.Task`s in the main event loop
- Share a single `JobProcess` instance across all jobs in the pool
- Run `setup_fnc` once globally
- Validate: 50 simulated jobs (`AgentServer.simulate_job`) running concurrently in one process

**Phase 2 (week 2)** — Productionize:
- Per-job error isolation (an unhandled exception in one session does not crash the worker)
- Backpressure / load shedding (when CPU/memory hits a threshold, stop accepting jobs)
- Memory monitoring per job (we lose `job_memory_limit_mb` from `ProcJobExecutor` since there's no separate process to measure)
- Clean drain (waiting for in-flight sessions on SIGTERM)
- Update CLI: `openrtc dev` should default to coroutine mode; keep a `--isolation=process|thread|coroutine` flag for users who want classic behavior

**Why B over C:**
- Reuses LiveKit's worker WS state machine (which they maintain, with tested reconnect/retry/heartbeat)
- We don't have to track LiveKit protocol changes
- We can ship to v0.1 in 2 weeks instead of 5
- If `JobExecutor` Protocol changes between livekit-agents minor versions, the blast radius is one file

**Why not A:**
- 5-10x is below the 30x target
- We'd still pay for per-thread VAD copies unless we monkey-patch `setup_fnc` to use globals — at which point we're closer to B anyway
- THREAD mode was designed for Windows compatibility, not density — we'd be repurposing it

**Why not C (yet):**
- The benefit (full control of worker WS) doesn't materialize until we want to do something LiveKit's protocol can't express
- 3-5 weeks of replicating a state machine that already works is hard to justify before validating that B has any blockers
- If/when we hit a real wall with B (e.g., we want sub-process isolation per group of N sessions, or we want to multiplex multiple LiveKit clusters in one worker), C becomes the natural next step

---

## STEP 4: Gap Analysis vs v0.1 Target

Target: multi-session asyncio worker, livekit-agents-compatible API, single Docker container, scales horizontally.

### Already pointed in this direction

- ✅ **API shape is aligned**: agents are still standard `livekit.agents.Agent` subclasses, plugin objects pass through unchanged. A user moving from livekit-agents to OpenRTC keeps every `Agent`/`@function_tool` line.
- ✅ **Routing exists**: 6-tier resolution chain (`pool.py:781-853`) means one process can host N agent types and dispatch per-session via metadata. The multi-agent-per-worker piece is done.
- ✅ **Shared prewarm hook**: `_prewarm_worker` (`pool.py:95`) loads VAD + turn detector into `proc.userdata` once. This refactors cleanly into a "load once globally" pattern under coroutine mode.
- ✅ **Spawn-safe serialization**: provider objects already round-trip via `_ProviderRef` (`pool.py:602-619`). This work becomes irrelevant under coroutine mode (no more spawn) but does not block the change.
- ✅ **Metrics infrastructure**: `RuntimeMetricsStore` (`resources.py:119-281`) already tracks per-session counters thread-safely; per-coroutine counters need no change.
- ✅ **JSONL stream + TUI sidecar**: works regardless of session model.
- ✅ **CI hygiene**: 80% coverage gate, multi-version Python matrix, ruff + mypy + Codecov, automated PyPI publish.

### Currently entangling connection logic with inference logic

- **`pool.py:107-138` (`_run_universal_session`)** is the only place inference + routing + connection meet, and it's already small (~30 lines). The split is clean: routing → session construction → `session.start(room=ctx.room)`. Moving from process-per-job to coroutine-per-job means changing how `ctx` is constructed and how `proc.userdata` is populated, not changing this entrypoint.
- **`pool.py:284` `AgentServer()`** is the line that locks us into the process-per-job model. Changing it to `AgentServer(job_executor_type=JobExecutorType.THREAD)` is the cheapest experiment; subclassing or replacing `proc_pool` is the v0.1 change.

### Smallest architectural change to get to v0.1

A new module `src/openrtc/_coroutine_pool.py` that:

1. Defines `CoroutineJobExecutor` implementing `livekit.agents.ipc.job_executor.JobExecutor`
2. Defines `CoroutinePool` mirroring `ProcPool`'s public surface but running jobs as coroutines
3. Wires it into `AgentServer` either by subclassing or by patching `_proc_pool` after `__init__`

Plus a one-line change in `pool.py:284`:
```python
self._server = _CoroutineAgentServer()  # subclass that swaps proc_pool
```

Plus changing `_prewarm_worker` to set module-level globals instead of per-process `proc.userdata` (so all coroutines read the same VAD instance).

That's the architectural delta. Everything else (routing, serialization, metrics, CLI, dashboard, TUI) survives unchanged.

### Code actively fighting against this goal

1. **`AgentConfig.__getstate__` / `__setstate__`** (`pool.py:173-198`): exists only because livekit-agents serializes config across the process boundary. **In coroutine mode, this is dead weight** — there is no boundary to cross. Not actively harmful, but ~50 lines of code that exist solely to pay for the process model.

2. **`_PROVIDER_REF_KEYS` and `_try_build_provider_ref`** (`pool.py:86-619`): same story. Spawn-safe serialization for provider objects becomes unnecessary in coroutine mode.

3. **`_DEPRECATED_TURN_HANDLING_KEYS` migration logic** (`pool.py:42-778`): ~130 lines of deprecated kwarg translation. Orthogonal to the density goal, but worth flagging as a candidate for a separate module if `pool.py` keeps growing.

4. **`pool.py` is 962 lines** — it would be much easier to evolve if split into `pool.py` (AgentPool + AgentConfig), `routing.py`, `serialization.py`, and `turn_handling.py`. Not a v0.1 blocker; a refactoring-debt note.

5. **`AgentPool._resolve_agent` (`pool.py:483-497`) and `_handle_session` (`pool.py:498-500`)** — both wrap module-level functions and are never called externally. Dead methods.

6. **`_version.py`** (`src/openrtc/_version.py:1-3`): never imported. Dead file from before `hatch-vcs`.

7. **`tests/conftest.py` LiveKit shim** (lines 34-99): 65 lines stubbing `livekit.agents` for environments without the real SDK. **None of these stubs do any voice processing**, so the test suite gives us no integration coverage for the changes we're about to make. We'll need real-server tests for v0.1.

---

## STEP 5: Open Questions

These are genuine unknowns the audit cannot resolve. I need clarification before building:

1. **What is "one session"?** Is it (a) one room with one human caller, (b) one job dispatched by LiveKit, or (c) one phone call routed through SIP that may bridge into a room? If sessions can cross-room or share rooms (as in `multi-user-transcriber`), our architecture choices change. Assumption I made: one session = one job = one room = one caller, which matches the README's "multiple agents in one worker" framing.

2. **Memory budget per session at the v0.1 target.** With 50 sessions per worker and a 3 GB total budget, that's 60 MB/session. Realistic floor is: ~30-50 MB for the WebRTC peer connection + ~5-10 MB audio buffers + ~5 MB conversation state = ~50 MB. **The math is tight but feasible.** If we want headroom (or to include any local model state per session), we may need to revise the target downward.

3. **Acceptable density vs isolation tradeoff.** Today, one crashed session leaves siblings alive (process boundary). Under coroutine mode, an unhandled exception in one session that escapes its task can affect the worker. **Acceptance criterion needed**: do we require zero cross-session blast radius (mandates careful exception handling everywhere), or can we tolerate "failed sessions disconnect, others continue, restart-on-N-crashes" semantics?

4. **What happens to `livekit-agents`' `InferenceExecutor` (the subprocess that runs turn detection inference)?** Today it's one extra process per worker. Under coroutine mode we could (a) keep it as-is (one inference subprocess shared by all coroutines), (b) move inference inline (saves a process but adds GIL pressure), or (c) skip inference and use the simpler VAD-based turn detection. Default in our code is (a) when `LIVEKIT_REMOTE_EOT_URL` is set or an `inference_executor` is available, else fall back to "vad" (`pool.py:688-696`). Recommendation needed.

5. **GPU support.** Some users will want to load a local Whisper model or TTS model on GPU. In coroutine mode, GPU memory is shared across all sessions automatically — this is a *huge* win. Should v0.1 advertise this? Validating it requires actual GPU testing, which I can't do from this audit.

6. **Backwards compatibility semantics.** Does v0.1 keep PROCESS mode as an option (`AgentPool(isolation="process")`) or remove it? Removing simplifies code; keeping it lets users with crash-isolation requirements opt back. My read of AGENTS.md ("Preserve compatibility unless the change explicitly calls for a breaking release") suggests keeping both modes with `coroutine` as default.

7. **`AgentServer` API stability.** We'd be subclassing or patching internal-ish parts of livekit-agents (`_proc_pool` field, `JobExecutor` Protocol). Has LiveKit committed to those surfaces, or are they free to break across minor versions? Worth a quick conversation with the LiveKit team or a dive into their changelog before committing to Option B.

8. **Are there git tags on the remote?** Locally `git tag --list` is empty, but `docs/changelog.md` runs through v0.0.17. If tags do exist remotely, `hatch-vcs` works fine; if not, the version reported by the wheel may surprise users.

9. **Frontend example coverage.** `examples/frontend/` ships a React Router demo with `/dentist` and `/restaurant` routes. None of our v0.1 work changes the frontend contract (room metadata still routes to agents). Confirm this is in scope or can be deferred.

10. **The `integration` pytest marker exists** (`pyproject.toml:91-93`) but no test uses it. Is there a dormant integration suite somewhere, or did we plan and never write one? For v0.1, we will need real-LiveKit-server integration tests to validate density claims.

---

## Appendix A: File-Level Reference Map

| Concern | Primary File:Line | Test File |
|---|---|---|
| Pool, registration, session lifecycle | `src/openrtc/pool.py:256-557` | `tests/test_pool.py` (34 tests, 711 LOC) |
| Discovery, `@agent_config` | `src/openrtc/pool.py:378-431, 220-253` | `tests/test_discovery.py` (8 tests) |
| Routing | `src/openrtc/pool.py:781-853` | `tests/test_routing.py` (14 tests) |
| Provider serialization | `src/openrtc/pool.py:573-646` | `tests/test_pool.py:248-287` |
| Turn handling translation | `src/openrtc/pool.py:649-778` | `tests/test_pool.py:638-700` |
| Runtime metrics, RSS | `src/openrtc/resources.py:119-450` | `tests/test_resources.py` (7 tests) |
| JSONL schema, sink | `src/openrtc/metrics_stream.py` | `tests/test_metrics_stream.py` (15 tests) |
| CLI commands | `src/openrtc/cli_app.py` | `tests/test_cli.py` (25 tests) |
| LiveKit argv handoff | `src/openrtc/cli_livekit.py:37-118` | `tests/test_cli.py` |
| TUI tail/render | `src/openrtc/tui_app.py` | `tests/test_tui_app.py` (18 tests) |

## Appendix B: LiveKit Internals We Touched

| Source | What it tells us |
|---|---|
| `livekit/agents/job.py:86-89` | `JobExecutorType.PROCESS` and `THREAD` both exist |
| `livekit/agents/worker.py:127-131` | THREAD is the default on Windows only |
| `livekit/agents/worker.py:283-292` | `AgentServer.__init__` accepts `job_executor_type` parameter |
| `livekit/agents/ipc/proc_pool.py:107-150` | `_proc_spawn_task` chooses `ThreadJobExecutor` vs `ProcJobExecutor` based on `_job_executor_type` |
| `livekit/agents/ipc/job_thread_executor.py:70-95` | Each thread executor has its own `socketpair`, its own thread, its own `JobProcess` |
| `livekit/agents/ipc/job_proc_lazy_main.py:_JobProc._start_job` | Each job creates its own `rtc.Room()` instance |
| `livekit/agents/worker.py:_connection_task` | Worker WS protocol: aiohttp WS to `/agent` with JWT, protobuf register message, heartbeat 30s |
| `examples/other/transcription/multi-user-transcriber.py` | **Multiple `AgentSession`s in one entrypoint sharing `proc.userdata["vad"]` is supported** |
