Metadata-Version: 2.4
Name: canapy
Version: 0.1.2
Summary: canapy is a user friendly auto annotator for animal vocalizations using reservoir computing
License: BSD-3-Clause
License-File: LICENSE
Author: Axel Arnaud
Author-email: axel.arnaud@inria.fr
Requires-Python: >=3.11,<3.12
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Requires-Dist: Levenshtein (==0.21.1)
Requires-Dist: PyYAML (==6.0.3)
Requires-Dist: attrs (==23.2.0)
Requires-Dist: audioread (==3.1.0)
Requires-Dist: bokeh (==3.2.2)
Requires-Dist: click (==8.3.1)
Requires-Dist: crowsetta (==5.1.0)
Requires-Dist: hyperopt (==0.2.7)
Requires-Dist: ipykernel (==6.29.4)
Requires-Dist: joblib (==1.5.2)
Requires-Dist: jupyter (==1.1.1)
Requires-Dist: librosa (==0.10.2.post1)
Requires-Dist: matplotlib (==3.10.7)
Requires-Dist: multimethod (<2.0)
Requires-Dist: numba (>=0.58,<0.59)
Requires-Dist: numpy (==1.26)
Requires-Dist: pandas (==2.3.3)
Requires-Dist: pandera (==0.15.2)
Requires-Dist: panel (==1.2.3)
Requires-Dist: param (==1.13.0)
Requires-Dist: pillow (==12.0.0)
Requires-Dist: py4j (==0.10.9.9)
Requires-Dist: pydantic (==1.10.24)
Requires-Dist: pydub (==0.25.1)
Requires-Dist: requests (==2.32.5)
Requires-Dist: reservoirpy (>=0.4.0,<0.5.0)
Requires-Dist: scikit-learn (==1.7.2)
Requires-Dist: scipy (==1.15.3)
Requires-Dist: setuptools (==68.2.2)
Requires-Dist: soundfile (==0.13.1)
Requires-Dist: toml (==0.10.2)
Requires-Dist: tqdm (==4.67.1)
Description-Content-Type: text/markdown

# Canapy

**Automatic audio annotation tools for animal vocalizations**

--------

Canapy trains automatic annotators for animal vocalizations using [Reservoir Computing](https://reservoirpy.readthedocs.io/) (Echo State Networks). It comes with an interactive dashboard to guide you through the full pipeline: dataset preparation, model training, evaluation, and annotation.

> For the full reference documentation, see [README_extended.md](README_extended.md).

## Installation

```bash
git clone git@github.com:birds-canopy/canapy.git
pip install -e canapy/.
```

## Quick start

### 1. Prepare your dataset

You need hand-labeled audio recordings. Annotations should be `.csv` files in **marron1csv** format (columns: `wave`, `start`, `end`, `syll`). Audio must be mono WAV files.

Recommended structure:

```text
song_dataset/
├── annotations/
│   ├── song1.csv
│   └── song2.csv
└── audio/
    ├── song1.wav
    └── song2.wav
```

or

```text
song_dataset/
├── song1.wav
├── song1.csv
├── song2.wav
└── song2.csv
```

Aim for 30 min–1 hour of annotated data (10 min can already give good results on canary songs).

### 2. Launch the dashboard

```bash
canapy dash
```

The dashboard opens automatically at [localhost:9321](http://localhost:9321).

You can also pass paths directly as arguments:

```bash
canapy dash -a song_dataset/annotations -s song_dataset/audio -o output
```

---

## Using the dashboard

The dashboard is organized around a sidebar giving access to all pages. The typical workflow for training a new annotator is:

```
Load data → Preprocess → Train → Eval → (iterate) → Export → Annotate
```

### Home

The **Home** page gives an overview of Canapy and its pipelines, with a FAQ section and links to the GitHub repository and the Mnemosyne INRIA team.

### Load data

On the **Load data** page, specify where your data lives:

- **Source selection**: choose *combined folders* if audio and annotations are in the same directory, or *separate folders* otherwise.
- **Model directory**: if you want to annotate unlabeled data, point to a folder containing already-trained models (exported after using the training pipeline).
- **Output directory**: where models and annotations will be saved (defaults to `output/`).
- **Annotation format** and **audio extension**: set these to match your files.
- The **sampling rate** is auto-detected from your audio files. Enable *Downsample* if you want audio resampled to that rate at load time.

### Preprocess (Edit dataset)

Use this page to clean your dataset before training. It has two sections:

Two collapsible modules are always accessible at the top of the page, regardless of the active section:

- **Class spectrograms** — displays one representative mel spectrogram per class (the sample closest to the median duration). Useful for quickly spotting acoustically similar classes before merging.
- **Trim silences** *(Preprocess only)* — balances the proportion of silence in the dataset. Set the target silence ratio (e.g. 20%) and Canapy center-crops silence segments that exceed it. Trimmed files are saved to `output/audio_trimmed/` and `output/annots_trimmed/`.

**Class merge** — listen to each annotation class, compare them side by side, and rename or merge classes that are acoustically too similar. Type the new label in the text field and click *Apply*.

**Sample correction** — review individual samples per class. Select a class to listen to its samples one by one, view their spectrogram, and correct any mislabeled ones via the text input. Click *Save all* when done.

You can export corrected annotations at any time using the *Export Annotations* button at the top of the page.

### Train

**Hyperparameter search (optional):** Before training, you can run an automatic HP search to find better ESN parameters for your dataset. It uses TPE (sequential) or parallel random search. Key settings (on the **Settings** page): `opt_max_evals`, `opt_n_jobs`, `opt_max_percentage`. Optimized parameters are preserved across training iterations — no need to re-run the search each time.

Click *Start Training*. Three ESN-based models are trained:

| Model | Description |
|-------|-------------|
| `syn` | Trained on complete songs in order — uses sequential/syntactic context |
| `nsyn` | Trained on randomly shuffled, class-balanced samples — context-free |
| `ensemble` | Combines `syn` and `nsyn` by majority vote |

### Eval

After training, the **Eval** page shows:

- **Confusion matrix** — which classes are being confused with each other.
- **Per-class metrics table** — precision, recall, F1-score for each class (1.0 = perfect).
- **Class merge / sample correction** — same tools as Preprocessing, but focused on misclassified samples from the model's output.

If results are not satisfying, correct samples or merge classes, then retrain. **3–4 iterations** of train → eval is typically enough to converge. All corrections are preserved across iterations.

### Export

When you are satisfied with the model's performance, go to the **Export** page to save the trained models to the output folder. These exported models are then used in the **Annotate** pipeline. You can also export the current configuration (ESN and species parameters) from this page — or from the **Settings** page — to reuse it as a personal preset in future sessions.

### Annotate unlabeled data

Load your unlabeled audio and trained models via the **Load data** page, or pass them directly at launch:

```bash
canapy dash -d song_dataset/audio -c output/model -o output/annotations
```

Select which model(s) to use (*Syn-ESN*, *NSyn-ESN*, *Ensemble*), click *Start annotation*, then *Export annotation* when done. The export folder is named with a timestamp (`YYYY-MM-DD_HHhMMminSS`).

> If you trained with multiple iterations, you can pick a specific one: load `output/model/3` instead of `output/model`.

### Settings

The **Settings** page lets you configure the parameters used by Canapy:

- **Species parameters**: `fmin`, `fmax` (frequency range), `win_length`, `hop_length`, `n_fft` (spectrogram).
- **ESN parameters**: `sr` (spectral radius), `leak` (leak rate), `iss`, `isd`, `isd2` (input scalings), `ridge` (regularization).
- **HP search parameters**: `opt_max_evals`, `opt_n_jobs`, `opt_max_percentage`.

Click *Validate* to apply changes to the current session.

**Presets** — the Settings page provides pre-configured profiles for specific species (canary, Bengalese finch, zebra finch, mouse, infant marmoset). The canary preset is loaded by default. You can also load your own configuration file via *Load config*. If you have tuned Canapy for a new species, feel free to send us your configuration so we can add it to the preset library.

---

## Quick commands (no dashboard)

```bash
# Train once and export models
canapy train -d song_dataset/data -o output

# Annotate unlabeled audio with trained models
canapy annotate -d song_dataset/audio -c models_folder -o output
```

## Support

Contact Axel Arnaud or Xavier Hinaut at Inria Mnemosyne:
<axel.arnaud@inria.fr> — <xavier.hinaut@inria.fr>

