Metadata-Version: 2.4
Name: nn-stat
Version: 2.2.7.1
Summary: Neural Network Performance Analysis
Author-email: ABrain One and contributors <AI@ABrain.one>
License: MIT License
        
        Copyright (c) 2025- ABrain One and contributors
        All rights reserved.
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in all
        copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
        SOFTWARE.
        
Project-URL: Homepage, https://ABrain.one
Project-URL: Repository, https://github.com/ABrain-One/nn-stat
Keywords: neural networks,statistical analysis,machine learning,python,artificial intelligence,data science,AI tools
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Software Development :: Libraries
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: openpyxl
Requires-Dist: matplotlib
Requires-Dist: pandas
Requires-Dist: numpy
Requires-Dist: seaborn
Provides-Extra: dataset
Requires-Dist: nn-dataset; extra == "dataset"
Dynamic: license-file

# <img src='https://abrain.one/img/lemur-nn-icon-64x64.png' width='32px'/> Neural Network Performance Analysis
<sub><a href='https://pypi.python.org/pypi/nn-stat'><img src='https://img.shields.io/pypi/v/nn-stat.svg'/></a> <a href="https://pepy.tech/project/nn-stat"><img alt="GitHub release" src="https://static.pepy.tech/badge/nn-stat"></a><br/>
short alias <a href='https://pypi.python.org/pypi/lmurs'>lmurs</a></sub>

The original version of the NN Stat project was created by <strong>Waleed Khalid</strong> at the Computer Vision Laboratory, University of Würzburg, Germany.

<img src='https://abrain.one/img/lemur-nn-stat-whit.jpg' width='25%'/>

<h3>Overview 📖</h3>

<p>Automated conversion of <a href="https://github.com/ABrain-One/nn-dataset" target="_blank" rel="noopener noreferrer">LEMUR</a> data into Excel format with statistical visualizations. It is developed to support the <a href="https://github.com/ABrain-One/nn-dataset">NN Dataset</a> and <a href="https://github.com/ABrain-One/nn-gpt">NNGPT</a> projects.</p>

## Create and Activate a Virtual Environment (recommended)
For Linux/Mac:
   ```bash
   python3 -m venv .venv
   source .venv/bin/activate
   ```
For Windows:
   ```bash
   python3 -m venv .venv
   .venv\Scripts\activate
   ```

It is assumed that CUDA 13.0 is installed. If you have a different version, please replace 'cu130' with the appropriate version number.

## Environment for NN Stat Contributors

Run the following command to install all the project dependencies:
```bash
python -m pip install --upgrade pip
pip install -r requirements.txt --extra-index-url https://download.pytorch.org/whl/cu130
```

## Installation with the LEMUR Dataset

```bash
pip install nn-stat[dataset]
```

## Usage

```bash
python -m ab.stat.export
```
Data and statistics are stored in the <strong>stat</strong> directory in Excel files and PNG/SVG plots.

To use 'ab/stat/nn_analytics.ipynb' install jupyter:

```bash
pip install jupyter
```

and run jupyter notebook:

```bash
jupyter notebook --notebook-dir=.
```

## Update of NN Dataset
Install from GitHub to get the most recent code and statistics updates:
```bash
rm -rf db
pip install --no-cache-dir git+https://github.com/ABrain-One/nn-dataset --upgrade --force --extra-index-url https://download.pytorch.org/whl/cu130
```
Installing the stable version:
```bash
rm -rf db
pip install nn-dataset --upgrade --extra-index-url https://download.pytorch.org/whl/cu130
```


### Docker
All versions of this project are compatible with <a href='https://hub.docker.com/r/abrainone/ai-linux' target='_blank'>AI Linux</a> and can be seamlessly executed within the AI Linux Docker container:
```bash
docker run -v /a/mm:. abrainone/ai-linux bash -c "PYTHONPATH=/a/mm python -m ab.stat.export"
```
Some recently added dependencies might be missing in the <b>AI Linux</b>. In this case, you can create a container from the Docker image ```abrainone/ai-linux```, install the missing packages (preferably using ```pip install <package name>```), and then create a new image from the container using ```docker commit <container name> <new image name>```. You can use this new image locally or push it to the registry for deployment on the computer cluster.

## Tasks & datasets
[Tasks & datasets](https://github.com/ABrain-One/nn-stat/tree/main/ab/stat/docs/tasks_datasets_table.md)

## Current LEMUR Statistics

### Image Classification
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/img-classification_top5_metric_vs_epoch_grid.png'/>

### Image Captioning, Image Segmentation, Text Generation
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/Figure_A_accuracy_vs_epoch.png'/>


### Accuracy VS Duration Best Models
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/img-classification_accuracy_vs_duration_top5_models_grid.png'/>
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/Figure_D_acc_vs_duration.png'/>


### Best Per Run Distribution
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/Figure_B_best_metric_distributions.png'/>


### Best Per Run VS Duration
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/Figure_C_pareto_frontiers.png'/>


### Model Rank Heatmap
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/img-classification_rank_heatmap_top10.png'/>
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/img-segmentation_iou_rank_heatmap_top10.png'/>
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/img-captioning_bleu_rank_heatmap_top10.png'/>

### The best models for image classification, image segmentation and text generation tasks across all the datasets.
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/Figure_E.png'/>

### Model performance and variability across runs


#### The bars show the average result for each model, while the error bars indicate how much those results vary across different runs when enough data is available.
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/Figure_F.png'/>

#### The plot shows how performance develops over time. Error bars reflect how much the results change across different runs with different settings, and they are only included when enough data is available.
<img src='https://raw.githubusercontent.com/ABrain-One/nn-stat/main/ab/stat/docs/figures/Figure_G.png'/>

#### The confidence intervals show how much results vary across different runs. They are not meant to compare models statistically or indicate which model is significantly better.

#### The idea and leadership of Dr. Ignatov
