Metadata-Version: 2.4
Name: silicon-asr
Version: 0.1.0
Summary: Fast local speech-to-text runners for Apple silicon (CoreML/ANE, MLX) with honest benchmarks
Author: WT-MM
License-Expression: MIT
Project-URL: Repository, https://github.com/WT-MM/silicon-asr
Keywords: asr,speech-to-text,coreml,ane,apple-silicon,parakeet,whisper,mlx,transcription
Classifier: Development Status :: 4 - Beta
Classifier: Operating System :: MacOS
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.26
Requires-Dist: coremltools>=8.0
Requires-Dist: huggingface_hub>=0.23
Provides-Extra: dev
Requires-Dist: ruff; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Provides-Extra: mlx
Requires-Dist: parakeet-mlx>=0.5; extra == "mlx"
Requires-Dist: mlx-whisper>=0.4; extra == "mlx"
Requires-Dist: numba>=0.59; extra == "mlx"
Dynamic: license-file

# silicon-asr

[![Tests](https://github.com/WT-MM/silicon-asr/actions/workflows/test.yml/badge.svg)](https://github.com/WT-MM/silicon-asr/actions/workflows/test.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
![Python 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)

Fast local speech-to-text on Apple silicon, from Python. Runs the
[FluidInference CoreML conversions](https://huggingface.co/FluidInference) of
NVIDIA's Parakeet models on the **Apple Neural Engine** — roughly **2x
faster than the MLX equivalents** at matched accuracy, at a fraction of the
power — plus MLX baselines behind the same interface, and a benchmark CLI to
measure all of them on your own machine and audio.

**Why this exists:** the CoreML models are excellent but only had Swift
(and Node/Rust) bindings — the maintainer has
[no plans for a Python SDK](https://github.com/FluidInference/FluidAudio/issues/82).
This package is the missing Python path: model download, the TDT/CTC decode
loops, long-audio windowing with silence-aware stitching, timestamps, and
SRT output.

```bash
uv tool install silicon-asr        # or: pip install silicon-asr

silicon-asr transcribe movie.mp4 --srt movie.srt
silicon-asr bench movie.mp4 --runners parakeet-coreml,parakeet-mlx
```

```python
from silicon_asr.runners import create_runner

runner = create_runner("parakeet-coreml")      # downloads once from the Hub
result = runner.transcribe_file("movie.mp4")   # any format ffmpeg reads
for seg in result.segments:
    print(f"[{seg.start:7.2f} → {seg.end:7.2f}] {seg.text}")
```

## Runners

| runner | model | params | languages | notes |
|---|---|---|---|---|
| `parakeet-coreml` | Parakeet TDT 0.6B **v3** | 0.6B | EN + 24 European | default; best accuracy/speed |
| `parakeet-v2-coreml` | Parakeet TDT 0.6B v2 | 0.6B | English | slightly better English WER |
| `parakeet-ctc-coreml` | Parakeet CTC 110M | 110M | English | tiny/fast tier, no autoregressive loop |
| `parakeet-mlx` | Parakeet TDT v3 via [parakeet-mlx](https://github.com/senstella/parakeet-mlx) | 0.6B | EN + 24 | MLX baseline (`[mlx]` extra) |
| `whisper-mlx-turbo` | whisper-large-v3-turbo via mlx-whisper | 0.8B | ~100 | multilingual baseline (`[mlx]` extra) |

Models download from Hugging Face on first use (0.6–1.2 GB). The very first
CoreML load also compiles for the ANE (~40 s, once per model — the OS caches
it; later loads take ~0.2 s).

## Benchmarks

Measured on an **M3 Max**, 12 minutes of speech-dense audio, warm model
caches, best-of runs (first-ever load adds one-time ANE compilation, see
above):

| runner | load (s) | transcribe (s) | x realtime | WER* |
|---|---|---|---|---|
| parakeet-coreml (ANE) | 2.1 | 5.1 | 142x | 0.116 |
| parakeet-v2-coreml (ANE) | 0.4 | 4.9 | 146x | 0.104 |
| parakeet-ctc-coreml (ANE) | 0.3 | **0.7** | **1007x** | 0.170 |
| parakeet-mlx (GPU) | 0.4 | 8.6 | 84x | 0.097 |
| whisper-mlx-turbo (GPU) | 0.0 | 220.8 | 3x | 0.098 |

*The test script's quirks ("VLCaption" spoken as one word) put a ~0.09 WER
floor under every runner here — read the deltas between rows, not the
absolute values. The 0.6B CoreML rows trade ~1-2 WER points (15-second
window stitching vs MLX's 120-second chunks) for ~1.7x speed; CTC-110M
trades accuracy for another 7x on top.

Reproduce with `silicon-asr bench <audio> --runners ... --reference script.txt`
(WER is case- and punctuation-insensitive word error rate).

Compute-unit notes, from measurement and the literature:

- The FastConformer encoder runs on the ANE (`--compute-units auto`/`ane`)
  or GPU (`gpu`); the LSTM prediction network has **no ANE kernel** and is
  pinned to CPU on purpose.
- On this M3 Max the ANE slightly beats the GPU; reports on M5-class Max
  chips show the GPU pulling ahead. Benchmark your own machine — that's what
  the CLI is for.
- The ANE draws single-digit watts vs tens for the GPU — for battery-bound
  work (long transcriptions on a laptop), ANE is the right default even at
  speed parity.

## How the long-audio decoding works

The CoreML models take fixed 15-second windows. silicon-asr decodes
overlapping windows and cuts each overlap at its **largest inter-token gap**
— i.e. at a silence — so words are never split at window boundaries. TDT
decoding threads the LSTM state through the fused JointDecision model
(argmax + duration folded into the graph), keeping the Python loop cheap;
CTC decoding is a single argmax-collapse pass.

## Roadmap

- **Nemotron 3.5 streaming 0.6B** (FluidInference CoreML): true streaming;
  currently Discord-gated on the Hub + OpenMDW license — pending access.
- **WhisperKit CoreML Whisper** (Argmax open models): needs a Python
  KV-cache decoder loop; would make the table cross-family.
- **Energy benchmarks**: `powermetrics`-based joules-per-audio-hour column.
- v2 `.mlpackage` re-export (the v2 repo only ships compiled `.mlmodelc`).

## Credits

- Model weights: [NVIDIA Parakeet](https://huggingface.co/nvidia) (CC-BY-4.0)
- CoreML conversions: [FluidInference](https://huggingface.co/FluidInference)
  (CC-BY-4.0) — this package would not exist without their work
- MLX baselines: [senstella/parakeet-mlx](https://github.com/senstella/parakeet-mlx),
  [mlx-whisper](https://pypi.org/project/mlx-whisper/)

Apple-silicon Macs only (the ANE/CoreML path requires bare-metal arm64
macOS). Needs `ffmpeg` on PATH for non-WAV inputs.

## Development

```bash
uv sync --extra dev --extra mlx
make format && make static-checks && make test
```

## License

MIT (this package). Model weights and conversions carry their own licenses
(CC-BY-4.0 for the Parakeet family).
