Metadata-Version: 2.4
Name: auditeeg
Version: 0.0.1
Summary: Audits for EEG decoding benchmarks: paired clean/leaky protocols, toolchain checks and reporting audits
Author-email: Rolly Maulana Awangga <awangga@gmail.com>
License: BSD-3-Clause
Project-URL: Homepage, https://auditeeg.github.io/
Keywords: eeg,bci,neural-decoding,benchmark,audit,data-leakage,reproducibility,moabb
Classifier: Development Status :: 2 - Pre-Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: BSD License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Medical Science Apps.
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.25
Requires-Dist: scipy>=1.10
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Dynamic: license-file

# auditeeg

Audits for EEG decoding benchmarks: how much of a reported score comes from the evaluation rather than
from the decoder.

Each audit compares a leaky evaluation with its clean twin on the same participants and reports the
inflation, how many participants show it, a one-sided paired Wilcoxon test, a rank-biserial effect
size and Holm correction across a family of audits.

**Status: early release.** Version 0.0.1 contains the paired statistics (`auditeeg.stats`). The
protocol audits (segment leakage, supervised spatial filters, transductive alignment, test-set
selection of preprocessing, epochs and seeds), the toolchain checks and the reporting audits will be
released with the accompanying paper.

## Install

```bash
pip install auditeeg
```

## Example

```python
import numpy as np
from auditeeg import audit_leakage_mechanism, audit_family

clean = np.array([0.62, 0.55, 0.71, 0.48, 0.66, 0.59, 0.73, 0.52, 0.68])  # one score per participant
leaky = clean + np.array([0.05, 0.09, 0.02, 0.11, 0.04, 0.07, 0.03, 0.08, 0.06])

audits = [audit_leakage_mechanism(leaky, clean, name="segment leakage")]
audit_family(audits)  # adds Holm-adjusted p-values
print(audits[0]["inflation_margin"], audits[0]["subject_consistency"], audits[0]["p_holm"])
```

## Links

- Project page: https://auditeeg.github.io/

## License

BSD-3-Clause.
