Metadata-Version: 2.4
Name: ensemble-pitch-extractor
Version: 0.1.0
Summary: FCPE TTA and pYIN ensemble pitch extraction for singing voice.
Author: ensemble-pitch-extractor contributors
License: MIT
Project-URL: Homepage, https://github.com/qiuqiao/ensemble-pitch-extractor
Project-URL: Repository, https://github.com/qiuqiao/ensemble-pitch-extractor
Project-URL: Issues, https://github.com/qiuqiao/ensemble-pitch-extractor/issues
Keywords: pitch-extraction,f0,fundamental-frequency,singing-voice,fcpe,pyin,test-time-augmentation,dynamic-programming,viterbi,audio,music-information-retrieval
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Multimedia :: Sound/Audio :: Analysis
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: librosa>=0.11.0
Requires-Dist: matplotlib>=3.10.9
Requires-Dist: numpy>=1.26
Requires-Dist: torch>=2.1
Requires-Dist: torchfcpe>=0.0.4
Dynamic: license-file

[中文](README.zh-CN.md)|English

# Ensemble Pitch Extractor

[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
[![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)

Ensemble Pitch Extractor is a singing-voice F0 extractor that combines FCPE test-time augmentation with a pYIN high-frequency candidate in a dynamic programming decoder. It is designed for ordinary singing, high notes, and whistle-register material where a single extractor may fail.

The package provides:

- a Python API for extracting F0 from waveforms or audio files;
- a command-line interface that saves `.npy` F0 tracks and optional `.png` diagnostic plots;
- an ensemble decoder that selects a smooth candidate path instead of averaging incompatible F0 estimates.

## Demonstration

*Audio samples sourced from the internet.*

<table>
<tr>
  <td><img src="assets/胸转哨.png" alt="Chest-to-Whistle"></td>
  <td><img src="assets/大颤音.png" alt="Large Vibrato"></td>
</tr>
<tr>
  <td><img src="assets/带噪声高音.png" alt="Noisy High Notes"></td>
  <td><img src="assets/低音.png" alt="Low Notes"></td>
</tr>
</table>

## Installation

```bash
pip install ensemble-pitch-extractor
```

For local development:

```bash
uv sync
uv run ensemble-pitch-extractor --help
```

Python 3.12 or newer is required.

## Command Line

Extract F0 from one audio file:

```bash
ensemble-pitch-extractor input.wav -o f0_out
```

This writes:

```text
f0_out/input.npy
```

The `.npy` file is a one-dimensional `float32` array in Hz. Unvoiced frames are stored as `0`.

Save a plot of F0 overlaid on a mel spectrogram:

```bash
ensemble-pitch-extractor input.wav -o f0_out --plot
```

Process all supported audio files in a directory:

```bash
ensemble-pitch-extractor audio_dir -o f0_out --plot
```

CUDA is auto-detected by default. To force a specific device:

```bash
ensemble-pitch-extractor audio_dir -o f0_out --device cpu
```

Control GPU memory with `--max-batch-length` (default 480000 samples ≈ 30s):

```bash
ensemble-pitch-extractor audio_dir -o f0_out --max-batch-length 200000
```

Disable pYIN and use FCPE TTA only:

```bash
ensemble-pitch-extractor input.wav -o f0_out --no-pyin
```

Useful options:

```text
--f0-min 80
--f0-max 4000
--max-batch-length 480000
--device auto
--pyin-priority-min-f0 1300
--pyin-fcpe-close-semitones 1.0
--interp-uv
--recursive
```

## Python API

```python
from ensemble_pitch_extractor import extract_f0_from_file, load_model

model = load_model()  # auto-detects CUDA, or pass device="cpu"
result = extract_f0_from_file(
    "input.wav",
    model=model,
    save_npy="f0_out/input.npy",
    save_plot="f0_out/input.png",
    f0_min=80,
    f0_max=4000,
)

f0 = result.f0          # Hz, shape: (frames,)
times = result.times    # seconds, shape: (frames,)
```

For audio already in memory:

```python
import librosa
from ensemble_pitch_extractor import extract_f0, load_model

model = load_model()
sr = model.get_model_sr()
audio, _ = librosa.load("input.wav", sr=sr, mono=True)
f0 = extract_f0(audio, sr, model, f0_min=80, f0_max=4000)
```

For torch tensor input (padded batch or concatenated, supports GPU):

```python
import torch
from ensemble_pitch_extractor import extract_f0_from_tensor, load_model

model = load_model("cuda")

# padded batch: (batch=4, samples) with fixed-length clips
wav = torch.randn(4, 16000, device="cuda")
f0 = extract_f0_from_tensor(wav, sr=16000, model=model)  # (4, frames)

# concatenated: clips of different lengths, no padding waste
wavs = [torch.randn(8000, device="cuda"), torch.randn(12000, device="cuda")]
lengths = [len(w) for w in wavs]
concat = torch.cat(wavs)
f0 = extract_f0_from_tensor(concat, sr=16000, model=model, lengths=lengths,
                            max_batch_length=20000)  # (2, max_frames), NaN padded
```

## Method Overview

The decoder treats each extractor output as a candidate trajectory. Current candidates are:

```text
FCPE key shift = 0
FCPE key shift = -12
FCPE key shift = +12
pYIN
```

For an FCPE candidate with key shift $s$, the model output is mapped back to the original pitch space before fusion:

$$
\hat f_{t,s} = \frac{f_{t,s}}{2^{s/12}} .
$$

pYIN is included as an ultra-high frequency candidate. By default it only searches 1300–4000 Hz, and frames below 1300 Hz are discarded. This prevents pYIN from replacing FCPE in normal ranges where FCPE usually captures finer detail.

All candidates are converted to MIDI note space:

$$
n_{t,k}=69+12\log_2\frac{f_{t,k}}{440}.
$$

The final path is selected by dynamic programming:

$$
\pi^*=\arg\min_\pi \sum_t U_t(\pi_t)+\sum_{t=1}^{T-1} C_t(\pi_{t-1},\pi_t).
$$

Here $U_t(k)$ is a per-frame candidate prior and $C_t(i,k)$ is a transition cost. This formulation avoids averaging octave errors, half-frequency errors, and algorithm-specific mistakes into spurious intermediate pitches.

## Heuristics as Priors

The implementation uses the following structured priors:

- MIDI-space costs make equal musical intervals comparable across frequency ranges.
- UV penalty discourages fragmented voiced/unvoiced paths.
- Octave-aware jump cost allows one-, two-, and three-octave transitions, which are important for chest-to-whistle jumps.
- FCPE `+12` receives a low-pitch prior below E2.
- FCPE `-12` receives a high-pitch prior above D5.
- pYIN receives a high-frequency prior only when it is above 1300 Hz and more than one semitone away from every FCPE candidate.
- RMS energy gating removes false voiced output during silence after decoding.

The default candidate order is `FCPE 0`, `FCPE -12`, `FCPE +12`, `pYIN`, so that exact ties prefer FCPE over pYIN.

## Build and Publish

```bash
uv lock --python 3.12
uv build
uv publish
```

With a PyPI token:

```bash
uv publish --token "pypi-..."
```
