Metadata-Version: 2.4
Name: codecscope
Version: 0.1.0
Summary: Measure how efficiently neural audio codecs tokenize audio: token rate, bitrate, codebook utilization, and reconstruction quality across DAC, EnCodec, SNAC, and Mimi
Author-email: Ravindu Pabasara Karunarathna <karurpabe@gmail.com>
Maintainer-email: Ravindu Pabasara Karunarathna <karurpabe@gmail.com>
License: MIT License
        
        Copyright (c) 2026 Ravindu Pabasara Karunarathna
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE. 
Project-URL: Homepage, https://github.com/RavinduPabasara/codecscope
Project-URL: Documentation, https://github.com/RavinduPabasara/codecscope#readme
Project-URL: Repository, https://github.com/RavinduPabasara/codecscope
Project-URL: Bug Tracker, https://github.com/RavinduPabasara/codecscope/issues
Keywords: audio codec,neural audio codec,tokenizer,dac,encodec,snac,mimi,rvq,benchmark,audio llm
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.21
Provides-Extra: audio
Requires-Dist: soundfile; extra == "audio"
Requires-Dist: librosa; extra == "audio"
Provides-Extra: quality
Requires-Dist: pystoi; extra == "quality"
Requires-Dist: pesq; extra == "quality"
Provides-Extra: dac
Requires-Dist: descript-audio-codec; extra == "dac"
Provides-Extra: encodec
Requires-Dist: transformers; extra == "encodec"
Requires-Dist: torch; extra == "encodec"
Provides-Extra: snac
Requires-Dist: snac; extra == "snac"
Requires-Dist: torch; extra == "snac"
Provides-Extra: hf
Requires-Dist: transformers; extra == "hf"
Requires-Dist: torch; extra == "hf"
Provides-Extra: all
Requires-Dist: soundfile; extra == "all"
Requires-Dist: librosa; extra == "all"
Requires-Dist: pystoi; extra == "all"
Requires-Dist: pesq; extra == "all"
Requires-Dist: descript-audio-codec; extra == "all"
Requires-Dist: transformers; extra == "all"
Requires-Dist: snac; extra == "all"
Requires-Dist: torch; extra == "all"
Provides-Extra: dev
Requires-Dist: pytest>=6.0; extra == "dev"
Requires-Dist: pytest-cov>=2.0; extra == "dev"
Requires-Dist: black>=21.0; extra == "dev"
Requires-Dist: flake8>=3.8; extra == "dev"
Requires-Dist: mypy>=0.800; extra == "dev"
Dynamic: license-file

# codecscope

**Measure how efficiently neural audio codecs tokenize your audio.**

[![PyPI version](https://img.shields.io/pypi/v/codecscope.svg)](https://pypi.org/project/codecscope/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

When a neural codec feeds a language model, the number that decides your cost
is not bitrate — it's **tokens per second**. A 12.5 Hz codec with 8 codebooks
burns 100 tokens for every second of audio, and a 75 Hz codec with 4 codebooks
burns 300, even at a similar bitrate. Papers report these inconsistently, and
every comparison ships as a one-off eval script.

`codecscope` makes it one command: **token rate**, **bitrate**, **codebook
utilization**, and **reconstruction quality** across DAC, EnCodec, SNAC, Mimi,
and a raw-PCM baseline.

## Installation

```bash
pip install codecscope                 # core: numpy only
pip install 'codecscope[all]'          # + every codec and quality metric
pip install 'codecscope[dac,quality]'  # or pick what you need
```

## CLI

```bash
codecscope sample.wav \
    -c dac:44khz \
    -c encodec:24khz@6 \
    -c snac:24khz \
    -c hf:kyutai/mimi@8 \
    -c pcm:16 \
    --csv results.csv
```

Real output on real audio — 8 seconds of LibriVox narration, every codec run
against real weights:

```
Codec             Books  Frame Hz  Tokens/s  Bitrate   Util    Compress  Multi  Delay  SI-SNR  SI-SNR+  STOI    PESQ
----------------------------------------------------------------------------------------------------------------------
snac:24khz        3      46.88     82.03     984.4     0.0382  260.06    yes     -1    -0.47   -0.44    0.8412   -
kyutai/mimi@8     8      12.5      100.0     1100.0    0.0435  232.73    no      -1     4.26    4.27    0.8886   -
encodec:24khz@6   8      75.0      600.0     6000.0    0.3008  42.67     no       -     8.83    8.83    0.9374   -
pcm:16            1      16000.0   16000.0   256000.0  0.3099  1.0       no       -    79.65   79.65    1.0      4.6433
```

and DAC on the same clip, from its own environment:

```
dac:16khz         12     50.0      600.04    6000.4    0.3089  42.66     no      -8    -0.23   12.92
dac:44khz         9      86.13     775.2     7752.0    0.4347  33.02     no       -    14.39   14.39
dac:24khz         32     75.0      2400.1    24001.0   0.4270  10.67     no      -8     2.49   21.12
```

The bitrate column is derived independently — `token_rate × log2(codebook_size)` —
and lands on every codec's published figure:

| Codec | codecscope | Published |
|---|---|---|
| SNAC 24 kHz | 984.4 bps | 0.98 kbps |
| Mimi @8 | 1100.0 bps | 1.1 kbps |
| EnCodec 24 kHz @6 | 6000.0 bps | 6 kbps |
| DAC 16 kHz | 6000.4 bps | 6 kbps |
| DAC 24 kHz | 24001.0 bps | 24 kbps |
| DAC 44 kHz | 7752.0 bps | 8 kbps nominal (7752 is the exact math) |

Six independent agreements are the tightest available check that the metric
math is right.

### On a full song

The same codecs over a complete 2:13 track (Karissa Hobbs, *Let's Go Fishin'*):

```
codec                   tok/s   bitrate    util  tokens(track)  SI-SNR+   secs
------------------------------------------------------------------------------
snac:24khz               82.0       984  0.2869         10,913    -7.56   33.0
kyutai/mimi@8           100.0      1100  0.3315         13,304     1.26    9.7
encodec:24khz@6         600.0      6000  0.8608         79,800     7.35    7.3
pcm:16                22050.0    352800  0.6235      2,932,408    83.04    0.1
```

Token rate is identical to the 8-second measurement to the decimal — 82.0,
100.0, 600.0 — confirming it is a genuine per-second constant and that
estimating a track's cost by multiplying is sound. Encoding a 2-minute song
takes 7–33 seconds per codec.

Three things these tables make visible:

**Token rate is not bitrate.** SNAC has 3 codebooks at a 46.9 Hz finest rate,
yet costs 82 tokens/s rather than the 141 you would get by multiplying — the
multi-scale case, and the reason this tool exists. And SNAC costs *fewer*
tokens per second than Mimi despite Mimi running at a quarter of its frame
rate. If a language model consumes these codes, that column is your bill.

**Delay masquerades as distortion.** DAC's 16 kHz and 24 kHz models return
8 samples early. Uncorrected, that costs them 13 and 19 dB, ranking the
24 kbps model *below* the 8 kbps one. The `SI-SNR+` column compensates and
restores the expected order: 21.12 > 14.39 > 12.92, monotonic in bitrate.

**Codebook utilization needs real audio.** These figures are 3–4× higher than
the same codecs measured on a synthetic sweep, because a chirp exercises a
narrow slice of any codebook.

Run it with no file to sweep a synthetic chirp — useful for a quick smoke test
of a new backend:

```bash
codecscope -c pcm:16 -c pcm:8 -c pcm:4 --duration 3
```

## Python API

```python
import codecscope
from codecscope import audio

signal, rate = audio.load("sample.wav")

report = codecscope.analyze(signal, rate, "snac:24khz")
report.token_rate            # 207.0   codes/sec — what an LM actually pays
report.frame_rate            # 83.3    finest codebook's rate
report.bitrate               # 2070.0  bits/sec
report.codebook_utilization  # 0.88    fraction of entries actually emitted
report.compression_ratio     # 123.7   vs 16-bit PCM
report.si_snr                # 9.7     dB
report.is_multiscale         # True

reports = codecscope.compare(signal, rate, ["dac:44khz", "snac:24khz", "pcm:16"])
for r in reports:            # sorted by token rate, cheapest first
    print(r.as_dict())
```

## What the metrics mean

| Metric | Why it's here |
|---|---|
| `token_rate` | Codes per second. The real cost driver when a codec feeds an LM — and the number most papers bury. |
| `frame_rate` | Frames/sec of the **finest** codebook. |
| `bitrate` | `token_rate × log2(codebook_size)`. Two codecs at equal bitrate can differ 3× in token rate. |
| `codebook_utilization` | Fraction of codebook entries actually emitted. Low values mean vocabulary you pay for but never use — a known RVQ collapse signature, and the audio analogue of a tokenizer's UNK rate. |
| `compression_ratio` | Against the 16-bit PCM source. |
| `si_snr` | Scale-invariant SNR in dB. Always available; scale-invariant because a codec is free to change gain and shouldn't be punished for it. |
| `si_snr_aligned` | The same, after compensating for measured codec delay. Compare codecs on this; compare against papers on `si_snr`. |
| `delay_samples` | Reconstruction offset in samples, positive for late. Reported, never silently corrected. |
| `stoi` / `pesq` | Speech intelligibility. Opt-in via `--speech`. |

### Codec delay is reported, not silently corrected

Codecs can shift their output in time, and an offset of a few samples is
enough to destroy a score: DAC's 16 kHz and 24 kHz models return 8 samples
early, which costs them 13 and 19 dB of apparent quality on real speech.

Rather than pick one answer, `codecscope` reports both. `si_snr` is raw and
matches how codec papers measure, so published figures stay reproducible.
`si_snr_aligned` compensates for the measured delay. A large gap between them
means the difference is timing, not fidelity.

**Caveat on the delay figure itself.** It comes from cross-correlation, which
on strongly periodic material can lock onto a pitch period instead of the
true offset. Measuring SNAC 24 kHz gives −40 samples on a sweep, −1 on
speech, 6 on orchestral music, and 196 on a trumpet loop — a codec's latency
does not actually vary that way. Trust it on broadband or transient-rich
audio; distrust it on sustained tones.

Detection is FFT-based over a bounded, highest-energy window, so its cost
does not grow with track length: 0.024 s on a 2-minute song, 0.037 s on five
minutes.

### PESQ rarely applies to a codec comparison

PESQ is defined only at 8 and 16 kHz. Most neural codecs operate at 24 kHz or
above, so in a typical run it is computed for the baseline and skipped
everywhere else — as in the speech table above. STOI has no such restriction
and is the more useful of the two here.

### Chunked codecs emit more codes than their nominal bitrate

EnCodec's 48 kHz model is stereo and chunks at 1 second with 1% overlap; its
24 kHz model does neither. Because the overlapping frames are genuinely
transmitted, `codecscope` counts them: `encodec:48khz@6` reports 6045 bps
against a nominal 6000, and 151.1 Hz against a true frame rate of 150. The
0.75% excess *is* the overlap redundancy, and it is a real cost that
per-frame arithmetic hides.

Mono input to a stereo codec is duplicated across channels rather than
rejected, and the decoded channels are averaged back before scoring.

### Rates count padded samples

Codecs pad their input up to a whole frame: SNAC turns 48,000 samples into
49,152. The codes describe the *padded* signal, so rates are computed against
that length. Dividing by the original input length instead inflates every
rate by the padding ratio — 2.4% on a 2-second SNAC clip, enough to miss the
published 0.98 kbps figure.

### Codebook utilization needs a real corpus, and needs length

Utilization counts the entries a codec actually emitted **on the audio you
gave it**, so short or narrowband input under-uses any codebook. Measured on
the same codecs across three input lengths:

| Codec | 2s synthetic sweep | 8s real speech | 2:13 full song |
|---|---|---|---|
| SNAC 24 kHz | 0.0099 | 0.0382 | **0.2869** |
| Mimi @8 | 0.0094 | 0.0435 | **0.3315** |
| EnCodec 24 kHz @6 | 0.0833 | 0.3008 | **0.8608** |

A short clip understates utilization by up to 7×, and it changes the verdict:
on a full track EnCodec is using 86% of its codebook space while SNAC uses
29%. Measure over a representative corpus, and compare codecs only on
identical input.

### Variable-rate codecs must declare their configuration

Most neural codecs are variable-rate, and their libraries pick a default that
is rarely the configuration people quote:

- **EnCodec** keeps 2 to 32 codebooks depending on bandwidth, and transformers
  defaults to the **lowest** (1.5 kbps). An unqualified `encodec:24khz` would
  enter a comparison at 1.5 kbps against codecs running flat out.
- **Mimi** ships `num_quantizers=32` in its config, but its headline 1.1 kbps
  is the **8**-codebook setting. Unqualified, it reports 4.4 kbps — four times
  the number everyone cites.

`codecscope` resolves both explicitly and always prints the resolved value in
the codec name, so a table cannot misattribute a rate:

```bash
codecscope sample.wav -c encodec:24khz@6 -c encodec:24khz@24 -c hf:kyutai/mimi@8
```

### Multi-scale codecs

SNAC runs its codebooks at *different* frame rates. The identity most eval
scripts assume —

```
token_rate == frame_rate × n_codebooks
```

— is false for those codecs, and reporting a single "frame rate" for them is
the usual way published comparisons go wrong. `codecscope` stores per-codebook
counts, flags the row with `is_multiscale`, and prints a note when one is in
the table.

### PESQ is not a general audio metric

PESQ is defined only for 8 kHz and 16 kHz **speech**. It returns a confident
number for music that means nothing, so `codecscope` never computes it unless
you pass `--speech`, and refuses outright at other sample rates rather than
silently resampling into a score you can't compare to published figures.

## The PCM baseline

`pcm:N` is a plain uniform quantizer with no ML dependencies. At 16 bits
against a 16-bit source it compresses by exactly 1.0 and emits one code per
sample — the fixed reference every neural codec is measured against, and the
reason the core package installs with only numpy.

It's also a live correctness check: SI-SNR falls about 6.02 dB per bit
removed, the textbook quantization law.

```bash
$ codecscope -c pcm:16 -c pcm:8 -c pcm:4 -c pcm:2 --duration 3
pcm:16   ...  compress 1.0   SI-SNR 92.08
pcm:8    ...  compress 2.0   SI-SNR 44.05
pcm:4    ...  compress 4.0   SI-SNR 19.91
pcm:2    ...  compress 8.0   SI-SNR  6.31
```

## Codec specs

| Spec | Backend | Extra |
|---|---|---|
| `pcm:16` | uniform PCM quantizer | — (built in) |
| `dac:44khz`, `dac:24khz`, `dac:16khz` | Descript Audio Codec | `[dac]` |
| `encodec:24khz@6`, `encodec:48khz@12` | Meta EnCodec via transformers (bandwidth in kbps after `@`; default 6) | `[encodec]` |
| `snac:24khz`, `snac:32khz`, `snac:44khz` | multi-scale SNAC | `[snac]` |
| `hf:kyutai/mimi@8`, any repo id | transformers codecs (quantizer count after `@`) | `[hf]` |

A spec with no prefix is treated as a Hugging Face repo id.

### Installing the DAC extra

`descript-audio-codec` pulls `descript-audiotools`, which pins
`protobuf<3.20` and leaves its torch requirement unbounded. Installing it into
a shared environment can downgrade both protobuf and torch out from under
everything else — in testing it took torch from 2.10 to 2.7, which in turn
broke torchvision and every transformers codec. Give it its own virtualenv:

```bash
python -m venv .venv-dac && .venv-dac/bin/pip install 'codecscope[dac]'
```

## Notes on measurement

- Audio is resampled to each codec's operating rate before encoding, and
  quality is scored against that **resampled** signal — a codec shouldn't be
  charged for the resampler's error.
- Install `librosa` (`[audio]`) for band-limited resampling. Without it,
  rate matching falls back to linear interpolation and `codecscope` says so,
  because that aliasing lands in the quality scores.
- Reconstructions are truncated to the input length before scoring; codecs
  pad to whole frames, and scoring that padding as error is a silent bias.

## Tests

Two environments, because the DAC extra cannot share one (see above):

```bash
python -m venv .venv     && .venv/bin/pip     install -e '.[dev,hf,snac,quality,audio]'
python -m venv .venv-dac && .venv-dac/bin/pip install -e '.[dev,dac]'
```

```bash
pytest                  # 69 offline tests, no downloads, ~1s
pytest -m integration   # real-weights checks; 31 in .venv, 6 in .venv-dac
pytest -m ""            # everything available in the current environment
```

The integration suite asserts against each codec's *published* configuration
— EnCodec's 75 Hz frame rate and 2/4/8/16/32 codebook ladder, SNAC's three
4096-entry codebooks at 0.98 kbps with `vq_strides` [4, 2, 1], Mimi's 1.1 kbps
at 8 quantizers, DAC's 44100/512 hop — so it fails if an upstream release
changes shapes, rate handling, or frame math. Each file skips cleanly when its
backend is not installed.

## Related projects

- [tokscope](https://github.com/RavinduPabasara/tokscope) — the same question
  for text: how efficiently does a tokenizer handle your language?

## License

MIT © Ravindu Pabasara Karunarathna
