Metadata-Version: 2.5
Name: smart-turn-livekit
Version: 0.1.0
Summary: Smart Turn semantic end-of-turn detection for LiveKit Agents — any Smart Turn model
Project-URL: Homepage, https://github.com/santhosh-005/smart-turn-livekit
Project-URL: Research, https://github.com/santhosh-005/tamil-eot
Project-URL: Tamil model, https://huggingface.co/santhosh-005/smart-turn-tamil
Project-URL: Tamil dataset, https://huggingface.co/datasets/santhosh-005/tamil-eot
Project-URL: Upstream, https://github.com/pipecat-ai/smart-turn
Author: Santhosh
License-Expression: BSD-2-Clause
License-File: LICENSE
Keywords: end-of-turn,endpointing,indic,livekit,smart-turn,tamil,turn-detection,voice-agent
Classifier: Framework :: AsyncIO
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Requires-Python: >=3.10
Requires-Dist: huggingface-hub>=0.23
Requires-Dist: numpy>=1.24
Requires-Dist: onnxruntime>=1.16
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=7; extra == 'dev'
Requires-Dist: soundfile>=0.12; extra == 'dev'
Provides-Extra: livekit
Requires-Dist: livekit-agents>=1.0; extra == 'livekit'
Requires-Dist: soxr>=0.3; extra == 'livekit'
Provides-Extra: resample
Requires-Dist: soxr>=0.3; extra == 'resample'
Description-Content-Type: text/markdown

# smart-turn-livekit

Run [Smart Turn](https://github.com/pipecat-ai/smart-turn) end-of-turn models on
**LiveKit Agents**. Given the last 8 seconds of a speaker's audio, the model
predicts whether they have finished talking — from prosody, without waiting for
STT.

```bash
pip install 'smart-turn-livekit[livekit]'
```

```python
from livekit.agents import AgentSession
from livekit.plugins import silero
from smart_turn_livekit import SmartTurnDetector

session = AgentSession(
    vad=silero.VAD.load(min_silence_duration=0.25),   # must be >= 0.25
    turn_handling={
        "turn_detection": SmartTurnDetector(model="smart-turn-tamil-tiny"),
        "endpointing": {"min_delay": 0.3, "max_delay": 2.5},
    },
)
```

Weights download from HuggingFace on first use and are cached. Nothing to
configure, nothing bundled in the wheel.

## Models

```python
SmartTurnDetector()                                  # smart-turn-tamil-tiny
SmartTurnDetector(model="smart-turn-tamil-base")
SmartTurnDetector(model="smart-turn-v3")
```

| key | languages | size | accuracy | on |
|---|---|---|---|---|
| **`smart-turn-tamil-tiny`** | ta | **8.7 MB** | 83.71% | 4,168 real Tamil telephone clips |
| **`smart-turn-tamil-base`** | ta | 21 MB | **86.13%** | same |
| `smart-turn-v3` | 23 | 8.7 MB | 92.63% | upstream's benchmark, mostly TTS audio |

**Note:** Accuracies are not directly comparable as they use different data sources and test sets.

Every Smart Turn checkpoint shares one ONNX signature — `input_features
(batch, 80, 800)` in, a `logits` output that is **already a sigmoid** out —
which is why one code path runs all of them.

### Tamil

End-of-turn detection for Tamil, expanding the languages supported by Smart Turn. These two models are fine-tuned specifically for Tamil from real Tamil telephone conversations.

| | accuracy | ROC-AUC | FP/N | p50 |
|---|---|---|---|---|
| `smart-turn-v3` zero-shot | 70.30% | 0.751 | – | – |
| **`smart-turn-tamil-tiny`** | 83.71% | 0.905 | **7.94%** | **83 ms** |
| **`smart-turn-tamil-base`** | **86.13%** | **0.921** | 9.17% | 143 ms |

`tiny` is the same architecture as v3, so it is a true drop-in. `FP/N` is Smart
Turn's convention (FP/N + FN/N = error rate), not FP/(FP+TN) — they differ by
~3×. Latency is 1 thread, batch 1, idle i5-12450H, inference only.

Replayed through the real Silero VAD and this adapter, the model gives the
**identical verdict on 90.9%** of boundaries and costs **2.6 accuracy points**
against scoring the pre-cut clips.

Method and ablations: <https://github.com/santhosh-005/tamil-eot>

## Two things that break it

- **`min_silence_duration >= 0.25`.** LiveKit will not request a prediction
  below `MIN_SILENCE_DURATION_MS + 50` ms and raises at session start if the VAD
  is faster. A floor, not a ceiling — it fails loudly.
- **The language must be one the model claims, or unknown.**
  `supports_language()` returns `False` otherwise, using the languages the model
  was *benchmarked* on. This one fails **silently** — the detector is skipped and
  the agent falls back to fixed timing.

Under LiveKit the threshold picks *how long to wait* — below it the session
takes `max_delay` instead of `min_delay` — so a false `complete` costs a shorter
pause, not an interruption.

## Threshold

0.5 is what every number above is quoted at. The Tamil models also ship two
tuned operating points, both picked on a held-out `dev` split:

| operating point | tiny | base | effect |
|---|---|---|---|
| `inherited` *(default)* | 0.50 | 0.50 | tables above |
| `polite` | 0.75 | 0.92 | FP/N roughly halves, costs ~3 points |
| `balanced` | 0.34 | 0.24 | maximises dev accuracy |

```python
SmartTurnDetector(operating_point="polite")
SmartTurnDetector(threshold=0.72)
```

Tuning for accuracy did not transfer: `base`'s dev-argmax scores 85.10% on test
against 86.23% at plain 0.5. That is why 0.5 is the default. `smart-turn-v3`
offers only `inherited` — there is no dev split of its data to pick one on.

## Other uses

```python
from smart_turn_livekit import SmartTurn

st = SmartTurn("smart-turn-tamil-tiny")
st.probability(wave_16k)      # P(speaker has finished), 0.0 .. 1.0
st.is_complete(wave_16k)      # bool, against st.threshold
```

16 kHz mono float, last 8 seconds only. Other rates need `soxr`.

**Pipecat** needs no adapter — its built-in analyzer takes any Smart Turn ONNX:

```python
from smart_turn_livekit import resolve_model
from pipecat.audio.turn.smart_turn.local_smart_turn_v3 import LocalSmartTurnAnalyzerV3

analyzer = LocalSmartTurnAnalyzerV3(
    smart_turn_model_path=str(resolve_model("smart-turn-tamil-tiny")),   # the file
)
```

Pipecat hardcodes the 0.5 threshold, and there the decision **ends the turn**.

**Pinned or air-gapped:**

```python
SmartTurnDetector(model_path="/opt/models/smart-turn-tamil.onnx")
SmartTurnDetector(local_only=True)                  # cached weights only
```

or `SMART_TURN_MODEL=/path/to.onnx`, or `HF_HUB_OFFLINE=1`. An explicit
`model_path` claims **no** language until you pass `languages=(...)`.

## Tests

```bash
pip install -e '.[dev,livekit]'
pytest                                  # 22 tests
python -m smart_turn_livekit.bench
```

Checks features against Pipecat's own extractor (<1e-3), the LiveKit Protocols
by `isinstance`, and that **every registered model loads and returns a
probability** — including upstream's, which we do not control.

## Limitations

- **The Tamil models are narrowband telephony.** Wideband or close-mic speech is
  out of distribution.
- **Not a VAD.** It answers "did they finish?", not "is anyone talking?" You
  still need Silero upstream.
- **Audio-only, no language token.** Nothing stops any of these models running
  on any language — `supports_language()` reflects what was *measured*.
- Run-to-run spread at identical config is ~0.9 points; treat smaller
  differences as noise.

## Author

Built and maintained by **Santhosh** ([@santhosh-005](https://github.com/santhosh-005)).

## Licence

**This package** is BSD-2-Clause. It bundles Whisper's mel filterbank (MIT,
OpenAI); nothing else in it is third-party.

**The weights** are downloaded, not distributed here, and licensed separately —
`smart-turn-v3` is BSD-2-Clause by Daily; the Tamil models are BSD-2-Clause,
trained on data derived from SPRING_INX Tamil R1 (CC BY 4.0), SPRING Lab, IIT
Madras.
