Metadata-Version: 2.4
Name: dead-relu-viz
Version: 0.1.0
Summary: Visualize dead ReLU neurons directly on a PyTorch model's architecture graph
Author-email: Safa Agouram <safaagouram28@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/safa-agouram/dead-relu-viz
Project-URL: Repository, https://github.com/safa-agouram/dead-relu-viz
Project-URL: Issues, https://github.com/safa-agouram/dead-relu-viz/issues
Keywords: pytorch,deep-learning,neural-network,visualization,relu,dead-neurons,torchview
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Scientific/Engineering :: Visualization
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Operating System :: OS Independent
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=1.10
Requires-Dist: torchview>=0.2.6
Dynamic: license-file

# dead-relu-viz

Visualize **dead ReLU neurons** directly on a PyTorch model's architecture
graph.

A dead neuron is one whose activation never leaves zero across an entire
dataset — a common failure mode after a bad weight init or too high a
learning rate. `dead-relu-viz` propagates your data through your model,
measures which `Conv2d`/`Linear` units never activate, and overlays the
result on the model's architecture graph (built with
[torchview](https://github.com/mert-kurttutan/torchview)), so you can spot
where dead neurons are concentrated at a glance.

Works with any PyTorch `nn.Module` and any `DataLoader` — not tied to a
specific model or dataset.

## Installation

```bash
pip install dead-relu-viz
```

[Graphviz](https://graphviz.org/) must also be installed on your system
(it's what torchview uses to render the graph):

```bash
apt install graphviz      # Debian/Ubuntu
brew install graphviz     # macOS
```

## Usage

In a Jupyter notebook:

```python
from dead_relu_viz import visualization

# model: any PyTorch nn.Module
# data_loader: any DataLoader
visualization(model, data_loader, with_activation=True, dpi=300)
```

This displays the model's architecture graph inline, with each
`Conv2d`/`Linear` layer that contains dead neurons highlighted in red and
annotated with the count and percentage of dead units.

Call it with `with_activation=False` (the default) to just render the
plain architecture graph, without running any data through the model.

### Arguments

- `model` — any PyTorch model (`nn.Module`).
- `data` — a `DataLoader` used to measure activations.
- `with_activation` — `False` (default): plain torchview graph. `True`:
  graph annotated with dead neurons in red.
- `device` — device used to run the model when measuring activations
  (defaults to CUDA if available, else CPU).
- `dpi` — if set, overrides the graph's rendering resolution (e.g.
  `dpi=300`).
- `exclude_from_dead_check` — layer names (as in `model.named_modules()`)
  to skip when coloring dead neurons — still measured and printed, just
  not flagged red. Useful for layers where "dead" isn't meaningful (e.g. a
  layer followed by a sigmoid instead of a ReLU).

## Example

See [cnn-activations](https://github.com/safa-agouram/cnn-activations) for
two worked examples (a small CNN trained on CIFAR-10, and
[GMIC](https://github.com/nyukat/GMIC), a breast-cancer screening model)
using this tool.

## License

MIT
