Metadata-Version: 2.5
Name: spectrorama
Version: 1.1.0
Summary: Log-frequency Gabor transforms, time-frequency projections and spectrogram animations with sound
Project-URL: Homepage, https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama
Project-URL: Documentation, https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/tree/main/notebooks
Project-URL: Issues, https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/issues
Project-URL: Changelog, https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/CHANGELOG.md
Project-URL: Mirror, https://github.com/joergrychen/spectrorama
Author-email: Jörg Rychen <jrychen@ethz.ch>
License: GPL-3.0-or-later
License-File: LICENSE
Keywords: Gabor frame,bioacoustics,constant-Q,hyperbolic,spectrogram,video
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
Classifier: Topic :: Multimedia :: Video
Classifier: Topic :: Scientific/Engineering
Requires-Python: >=3.10
Requires-Dist: matplotlib>=3.7
Requires-Dist: numpy>=1.24
Requires-Dist: scipy>=1.10
Requires-Dist: tomli>=1.1; python_version < '3.11'
Provides-Extra: docs
Requires-Dist: ipykernel; extra == 'docs'
Requires-Dist: jupyterlab>=4; extra == 'docs'
Requires-Dist: nbconvert>=7; extra == 'docs'
Requires-Dist: pillow; extra == 'docs'
Provides-Extra: test
Requires-Dist: pytest>=7; extra == 'test'
Requires-Dist: ruff>=0.4; extra == 'test'
Description-Content-Type: text/markdown

# spectrorama

[![PyPI](https://img.shields.io/pypi/v/spectrorama)](https://pypi.org/project/spectrorama/)
[![Python](https://img.shields.io/pypi/pyversions/spectrorama)](https://pypi.org/project/spectrorama/)
[![License: GPL-3.0-or-later](https://img.shields.io/badge/license-GPL--3.0--or--later-blue)](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/LICENSE)
[![pipeline](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/badges/main/pipeline.svg)](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/pipelines)

Log-frequency Gabor transforms, time-frequency projections, and spectrogram
panoramas: videos of sound (*dramas*), with the sound.

![One sound, the projections](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/raw/main/math/projections.png)

spectrorama is built in three layers. Each one is usable on its own, and each
has its mathematics written down:

| Layer | What it does | Depends on | Math |
|---|---|---|---|
| **1 · Transform** | Constant-Q Gabor frame; complex coefficients from one FFT, **exact inversion**; regions and masks for atom-wise filtering; level overviews | numpy, scipy | [frame](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/math/01-frame.md) · [inversion](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/math/02-inversion.md) |
| **2 · Projection** | Maps the time-frequency plane to a drawing plane — a catalogue classified by what it preserves, like map projections; a `View` (projection + rectangle + pixels) and a `Lookup` (pixels → atoms, with footprints) give a still at any moment in ~0.1 s | numpy | [projections](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/math/03-projections.md) |
| **3 · Stills and video** | Stills: any view, any moment, from scratch (`plot`, `picture`, `save_picture`). Video: a `Drama` swept through time -- tracks computed once, exposure motion blur, burned-in annotation and clock, a mono or stereo `Soundtrack` played as a tape-speed change (slowed down, ultrasound becomes audible), MP4 metadata and a sidecar JSON; frames painted in parallel, x264 or a hardware encoder | + matplotlib, ffmpeg | [stills and video](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/math/04-animation.md) |

Decisions, the measurements behind them, and the vocabulary are in the
[design notes](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/dev/design-notes.md); where the package stands, how it got
there and what could come next in the [status](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/dev/status.md).

spectrorama is general: it knows samples, Hz, seconds, an optional absolute
start time, a level reference and its unit. Reading recordings, choosing
channels, calibration conventions, file names and branding are the caller's
(for the WhaleSonic recordings: example notebooks in the whalesonic repo).

**Start here:** the notebooks in [notebooks](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/tree/main/notebooks/):
[QuickStart](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/notebooks/00-QuickStart.ipynb) ·
[Transform](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/notebooks/01-Transform.ipynb) ·
[Projections](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/notebooks/02-Projections.ipynb) ·
[Video](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/notebooks/03-Video.ipynb).
They need no data; install with `pip install -e ".[docs]"`.

## Install

```bash
pip install spectrorama           # Python >= 3.10
pip install -e ".[test]"         # from a clone, with the tests
```

Videos need [ffmpeg](https://ffmpeg.org) on the PATH.

## Quick start

```python
import spectrorama as sr

fs = 48000
x = ...                                  # 1-D signal, NaN marks missing samples

# Layer 1: transform and exact inverse
frame = sr.LogFrequencyFrame.for_range(20, 20000, bands_per_octave=48)
coefs = sr.analyze(x, fs, frame)         # complex, ragged per band
y = sr.synthesize(coefs)                 # == x (single-precision storage: ~1e-7)

# Layer 2: views and stills
exponent = sr.PowerTime.exponent_for((20, 60), (2000, 3))   # 60 s at 20 Hz, 3 s at 2 kHz
view = sr.View.fit(sr.PowerTime(Q=frame.Q, exponent=exponent),
                   f_range=(frame.f_min, frame.f_max), seconds=(20, 60), size=(1600, 800))
sr.plot(coefs, view, now=30.0)           # a still with axes, looking from t = 30 s
levels = sr.Lookup(view, coefs).sample(now=31.0)   # (H, W) dB, ~0.1 s

# Layer 3: a video with sound, 4x slower (ultrasound transposed down 2 octaves)
spec = sr.Spec(projection='power-time', exponent=exponent, seconds=(20, 60), speed=0.25)
print(spec.describe(fs))                 # what that choice implies
drama = spec.drama(coefs, sound=sr.Soundtrack(x, fs), path='out.mp4',
                   metadata={'title': 'test', 'copyright': '...'})
drama.preview(30.0)                      # exactly the video frame at t = 30 s
drama.write()                            # out.mp4 + out.json; or sr.render(x, fs, 'out.mp4', spec)
drama.write(encoding=sr.Encoding(codec='h264_qsv'))   # GPU encoder (Intel; also h264_nvenc)
```

For calibrated levels pass the reference and its unit: a signal in Pa with
`analyze(..., ref=1e-6, unit='µPa')` gives dB re 1 µPa²/Hz. Signals whose
coefficients would exceed the memory budget (`max_bytes`, default 8 GB) are
rejected before any work.

## Pipelines: a look file, the command line, masters

A `Spec` is plain data (`Spec.from_dict` / `to_dict`, unknown keys rejected),
so a look can live in a TOML file. It has two parts: the `[analysis]` is what
the transform computes (changing it needs new coefficients), everything else
is drawn from the same coefficients:

```toml
canvas = [1920, 1080]
projection = "power-time"
exponent = 1.21
seconds = [20, 120]          # 120 s visible at 20 Hz
[analysis]                   # f_min, f_max, bands_per_octave, support, psd
f_min = 20                   # bands from 19.7 Hz (third octaves at or below)
bands_per_octave = 48
[colour]
cmap = "inferno"
vlim = [40, 100]             # fixed: the same colours in every video of a series
[overlay]                    # the burned-in layers; `overlay = false`: bare picture
iso_ages = []
clock = true
```

```bash
spectrorama describe --spec look.toml --fs 48000          # what the look implies
spectrorama vlim clips/*.wav --spec look.toml             # one vlim for a series
spectrorama still clip.wav clip.png --spec look.toml --now 60 --now 90
spectrorama render clip.wav --spec look.toml --encoding preview --part 60 80
spectrorama render clip.wav master.mp4 --spec look.toml --encoding master --sound none     --ref 1e-7 --unit µPa --t0 2023-12-21T09:53:18 --title "..."
```

The level reference describes the data, not the look, so it comes with the
data: `--ref 1e-7 --unit µPa` when 1.0 in the WAV is 10 Pa (levels in dB re
1 µPa), in Python `spec.analysis.analyze(x, fs, ref=1e-7, unit='µPa')`.

The picture always comes from one channel (`--channel`, default 0); the sound
has one or two (`--sound 0,1`, default the file's channels if at most two),
never mixed into the picture. Every video gets a sidecar `<video>.json` with
its parameters and its **geometry**: canvas, view rectangle, now-line, the
y of round frequencies, clock and colour-bar boxes, frame rate, frame count,
time range, colour limits -- everything an editor needs to composite on top.
`Encoding.master()` (x264 crf 12, 4:4:4 at 10 bits: SSIM 0.988 at 28 Mbit/s,
where 4:2:0 reaches 0.955) is meant for a second encoding by an editor or
upload site, not for browsers; a drama without sound gives a file without an audio
stream. In Python the same is `Spec.from_dict(tomllib.load(f))`,
`spec.analysis.analyze(...)`, `spec.drama(coefs, ...)` (which checks that the
coefficients are of the spec's analysis), `drama.geometry()`, `sr.series_vlim(dramas)`,
`drama.write(encoding=sr.Encoding.master(), part=(60, 80))`.

**The ends of the signal.** A frame at `now` reads the signal around it: across
the view, plus about one atom (`Q / f`) at each band. Near the start the low
bands fill up while their windows fill; before the start and after the end there
is no data, shown in `colour.nodata` (grey; `nodata = "low"` shows it as silence,
in the colour map's lowest colour). Cut from a longer recording? Include extra
audio around the clip and play only the part you want (`render --part`).
Otherwise trim the video.

**The legend** is a separate layer for an editor to fade in and out: a
frequency ruler on the now-line and a colour bar on one rounded, semi-opaque
box, as a transparent RGBA PNG at canvas size (`render --legend` writes
`<video>.legend.png` next to the video, `spectrorama legend` only the PNG; in
Python `drama.legend(path)`). A `[legend]` table sets its side, ticks, spacing,
opacity and font size; the box is in the sidecar's `geometry.legend`. Pixels
outside the projection's domain (around a disc) take the `background` colour,
not the nodata colour.

## The projections

| | preserves | time shift on screen |
|---|---|---|
| `Rectangular` | — (classic constant-Q spectrogram) | rigid scroll |
| `Horocyclic` | **area**; exact shape on the now-line; clicks are vertical | each row scrolls at a speed proportional to f |
| `PowerTime(exponent)` | — ; window ∝ f^−exponent, from Rectangular (0) through Horocyclic (1) and beyond; `PowerTime.exponent_for((f1, s1), (f2, s2))` from two windows | rows scroll at speed ∝ f^exponent |
| `HalfPlane` | **angles** (atoms stay round) | rigid scroll |
| `LinearFrequency` | **area** (looks like an STFT spectrogram) | rigid scroll |
| `PoincareDisk` | **angles**, and the whole plane fits in a disc | Möbius motion |
| `KleinDisk` | **geodesics** are straight lines | projective motion |

New projections subclass `Projection` (implement `forward` and `inverse`) or
`RowProjection` (implement `x_scale`, `y_of_f`, `f_of_y`) to get the fast
row lookup. `Projection.tissot` measures distortion anywhere.

## The repository

| | |
|---|---|
| `src/spectrorama/` | the package |
| `tests/` | the tests |
| `math/` | the mathematics, one page per topic |
| `notebooks/` | the tutorials; their videos go to `notebooks/output/` (not in git), `02-Projections` draws the picture above |
| `dev/` | for developers: [design notes](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/dev/design-notes.md) (decisions, measurements, vocabulary, the contract with callers) and [status](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/dev/status.md) (history, speed, open items) |

## Development

```bash
pip install -e ".[test,docs]"
pytest                # reconstruction, frame bounds, calibration, projection properties,
                      # overview exactness, footprints, exposure, a real ffmpeg render
ruff check src tests
```

Conventions, for people and coding assistants alike:

- Three layers, imported bottom-up only. Keep layers 1 and 2 free of
  matplotlib; import it lazily where needed.
- spectrorama renders the data it is given. Reading recordings, rates,
  calibration, channel choice, file names and branding are the caller's job.
- Callers rely on `Spec.from_dict`, the sidecar geometry, `Encoding.master()`
  and the command line: keep them stable (the contract is in the design notes).
- Frozen dataclasses for configuration; plain functions or small classes for
  computation. Docstrings state definitions and *why*; formulas match `math/`.
- Every mathematical claim has a numerical test.
- Vocabulary: *frame* is the mathematical frame only; say *video frame*,
  *band*, *level* (dB), and *t* / *t0* / *now* / *tau* (see the design notes).
- The notebooks run without data; execute them after changes
  (`jupyter nbconvert --to notebook --execute --inplace notebooks/*.ipynb`).
- The version is single-sourced in `src/spectrorama/__init__.py`; keep
  `dev/status.md` current.
- On Windows with conda, the env's `Library/bin` must be on `PATH`, or numpy's
  BLAS fails to load without a message.
- New files carry the header
  `Copyright (C) <year> J. Rychen, ETH Zürich` /
  `SPDX-License-Identifier: GPL-3.0-or-later`.

## License

GPL-3.0-or-later. Copyright (C) 2026 J. Rychen, ETH Zürich.

To cite spectrorama, see [CITATION.cff](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/CITATION.cff); changes per version are in [CHANGELOG.md](https://gitlab.switch.ch/cetacean-communication/expeditions/whalesonic/spectrorama/-/blob/main/CHANGELOG.md).

The icon is a still of humpback whale song (Poincaré disk) from the WhaleSonic
dataset, [doi:10.3929/ethz-b-000722140](https://doi.org/10.3929/ethz-b-000722140), CC BY 4.0.
