Metadata-Version: 2.4
Name: DeePAQ
Version: 0.1.0
Summary: MERT-based audio quality embeddings and reference evaluation
Author: Guanxin Jiang, Fraunhofer IIS
License-Expression: MIT
Project-URL: Homepage, https://git.audiolabs.uni-erlangen.de/guanxin.jiang/audio-quality-assessment-dnn
Keywords: audio,quality-assessment,MERT
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Operating System :: OS Independent
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: MIT_License.txt
Requires-Dist: numpy
Requires-Dist: pandas
Requires-Dist: scipy
Requires-Dist: matplotlib
Requires-Dist: seaborn
Requires-Dist: tqdm
Requires-Dist: pyyaml
Requires-Dist: click
Requires-Dist: torch>=2.5.1
Requires-Dist: torchaudio>=2.5.1
Requires-Dist: transformers==4.57.1
Requires-Dist: peft==0.17.1
Requires-Dist: nnAudio==0.3.4
Dynamic: license-file

# DeePAQ

DeePAQ uses pre-trained MERT to extract embeddings of size 256. The provided `training_mapping.py` maps embeddings to a MUSHRA-like scale (0–100).

## Table of Contents

* [Installation](#installation)
* [Dataset](#dataset)
* [Usage](#usage)
* [Training](#training)
* [Evaluation](#evaluation)
* [Contributing](#contributing)
* [License](#license)

## Installation
DeePAQ is built and tested on PyTorch 2.5.1, TorchVision 0.20.1, TorchAudio 2.5.1 and CUDA 12.1 runtime
It can be either installed with pip or git.

### pip
Checkpoints are not part of the PyPI package. Download the DeePAQ checkpoints, and the neural mapping checkpoint if you use `mapping: nn`, and point `model_path` / `mapping_model_path` at those files.

```bash
pip install DeePAQ
```

From a local build of this repository:

```bash
pip install dist/deepaq-0.1.0-py3-none-any.whl
```

### git
```bash
git clone https://git.audiolabs.uni-erlangen.de/guanxin.jiang/audio-quality-assessment-dnn.git
cd audio-quality-assessment-dnn
git checkout DeePAQ
conda env create -f environment.yml
conda activate DeePAQ-env
```

## Dataset

For evaluation the project requires signals under test, matched reference signals, and corresponding subjective scores in CSV format. Each CSV should include columns:

* `filepath_deg` — path to the degraded/test signal
* `filepath_ref` — (optional) path to the matched reference signal
* `SG` — (optional) subjective rating. If SG column exists in your csv file, then the script will automatically detect it and then calculate the overall Spearman and Pearson correlation between the mapped results and subjective ratings

Place your dataset files in the `data/` folder.

## Usage

### Training

Run the main script with your training config:

```bash
python main.py --config_file src/config/train1_1.yaml
```

### Evaluation
#### Extract Embeddings
If you pip install deepaq 

```python
import deepaq
wav_path='/path/test.wav'
checkpoint='/path/best_model.pt'
embs = deepaq.compute_embedding(wav_path, checkpoint, model_type='MERT', mapping_type='polynomial')
```

#### Full Reference

```bash
python main.py --config_file src/config/fr_eval.yaml
```
or if you pip install deepaq
```bash
deepaq --config_file src/config/fr_eval.yaml
```

In fr_eval.yaml config file
* `model` — foundation model 
* `model_path` — checkpoint has been provided in out-models folder. For full reference, 2 checkpoints are provided, which are MERT-FR-2f and MERT-FR-best
* `mapping` - choose the mapping functions: polynomial uses 3rd order polynomial mapping; spline uses quadratic B-spline (degree 2, order 3) with three internal knots; nn uses a neural network trained to map Euclidean distance to MUSHRA scale.
* `mapping_model_path`: ./out-models/Dist2SG_Mapping/best_model.pt (This is optional if polynomial or spline is used for mapping)
* `test_files_csv` — csv file containing the data for testing
* `test_root_wav` — folder where your test data stored
* `emb_out_dir` - folder where you want to save your evalution results
* `emb_out_file` — csv file saves the evaluation results
* `plot_out_file` — png file saves scatterplot of the evaluation results

#### Non-matching Reference

```bash
python main.py --config_file src/config/nmr_eval.yaml
```
In nmr_eval.yaml config file 
* `model` — foundation model 
* `model_path` — checkpoint has been provided in out-models folder.
* `mapping` - choose the mapping functions: polynomial uses 3rd order polynomial mapping; spline uses quadratic B-spline (degree 2, order 3) with three internal knots; nn uses a neural network trained to map Euclidean distance to MUSHRA scale.
* `mapping_model_path`: ./out-models/Dist2SG_Mapping/best_model.pt (This is optional if polynomial or spline is used for mapping)
* `test_files_csv` — csv file containing the data for testing
* `test_root_wav` — folder where your test data stored
* `emb_out_dir` - folder where you want to save your evalution results
* `emb_out_file` — csv file saves the evaluation results
* `plot_out_file` — png file saves scatterplot of the evaluation results
* `non_matching_Mref_root` — folder where contains the non-matching reference with music content
* `non_matching_Sref_root` — folder where contains the non-matching reference with speech content

## Notation
model wav2vec is for development. If users want to use DeePAQ for inference, wav2vec could be skipped; otherwise need to install fairseq for wav2vec


