moabbr

MOABB interface to tombolo for analysis of machine learning benchmarks.

Requires Docker with the tombolo image pulled:

docker pull ethandavisecd/tombolo:latest

 1"""[MOABB](https://neurotechx.github.io/moabb/) interface to [tombolo](https://pypi.org/project/tombolo/) for analysis of machine learning benchmarks.
 2
 3Requires Docker with the tombolo image pulled:
 4
 5```
 6docker pull ethandavisecd/tombolo:latest
 7```
 8"""
 9
10from .run import bnma, nma
11from . import plots
12
13__all__ = ["nma", "bnma", "plots"]
def nma(results: pandas.DataFrame, greater_is_better: bool = True) -> dict: 
16def nma(results: pd.DataFrame, greater_is_better: bool = True) -> dict:
17    """Run a frequentist random-effects network meta-analysis on MOABB evaluation results.
18
19    Parameters:
20    - `results`: MOABB evaluation DataFrame with columns `dataset`, `pipeline`, `subject`, `score`.
21    - `greater_is_better`: If `True`, higher scores rank better (e.g. accuracy).
22      If `False`, lower scores rank better (e.g. error rate).
23
24    Each dataset is treated as an independent study and each pipeline as a treatment.
25    Pairwise mean differences are computed per subject, then aggregated per dataset.
26
27    Returns the result dict from `tombolo.nma`. See [tombolo](https://pypi.org/project/tombolo/) for the full return structure.
28    """
29    return tombolo.nma(_sql("nma", results), greater_is_better)

Run a frequentist random-effects network meta-analysis on MOABB evaluation results.

Parameters:

  • results: MOABB evaluation DataFrame with columns dataset, pipeline, subject, score.
  • greater_is_better: If True, higher scores rank better (e.g. accuracy). If False, lower scores rank better (e.g. error rate).

Each dataset is treated as an independent study and each pipeline as a treatment. Pairwise mean differences are computed per subject, then aggregated per dataset.

Returns the result dict from tombolo.nma. See tombolo for the full return structure.

def bnma(results: pandas.DataFrame, greater_is_better: bool = True) -> dict: 
32def bnma(results: pd.DataFrame, greater_is_better: bool = True) -> dict:
33    """Run a Bayesian random-effects network meta-analysis on MOABB evaluation results.
34
35    Parameters:
36    - `results`: MOABB evaluation DataFrame with columns `dataset`, `pipeline`, `subject`, `score`.
37    - `greater_is_better`: If `True`, higher scores rank better (e.g. accuracy).
38      If `False`, lower scores rank better (e.g. error rate).
39
40    Each dataset is treated as an independent study and each pipeline as a treatment.
41    Per-subject scores are averaged per dataset before being passed to the model.
42
43    Returns the result dict from `tombolo.bnma`. See [tombolo](https://pypi.org/project/tombolo/) for the full return structure.
44    """
45    return tombolo.bnma(_sql("bnma", results), greater_is_better)

Run a Bayesian random-effects network meta-analysis on MOABB evaluation results.

Parameters:

  • results: MOABB evaluation DataFrame with columns dataset, pipeline, subject, score.
  • greater_is_better: If True, higher scores rank better (e.g. accuracy). If False, lower scores rank better (e.g. error rate).

Each dataset is treated as an independent study and each pipeline as a treatment. Per-subject scores are averaged per dataset before being passed to the model.

Returns the result dict from tombolo.bnma. See tombolo for the full return structure.