Metadata-Version: 2.4
Name: slowai
Version: 0.3.0
Summary: One command to find why your PyTorch model is slow — and fix it.
Author-email: Rico Allen <ricardojallen37@gmail.com>
License-Expression: MIT
Project-URL: Homepage, https://github.com/ricojallen37-sketch/slowai
Project-URL: Repository, https://github.com/ricojallen37-sketch/slowai
Project-URL: Issues, https://github.com/ricojallen37-sketch/slowai/issues
Keywords: pytorch,profiling,performance,gpu,edge-ai,optimization,cuda,jetson
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Benchmark
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.1
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: ruff>=0.1; extra == "dev"
Requires-Dist: mypy>=1.0; extra == "dev"
Dynamic: license-file

# slowai

**One command to find why your PyTorch model is slow — and fix it.**

slowai diagnoses which performance regime your workload is stuck in (compute-bound, memory-bound, or overhead-bound), prescribes the right fix, auto-applies it, and proves the speedup with before/after measurements. No guesswork. No manual profiler interpretation.

```
$ slowai fix model.py

==============================================================
  BASELINE: model.py: COMPUTE_BOUND (confidence: 0.85)
  wall time: 7.523s
==============================================================

  Tried 4 remedies:

  1. [10.00x] bf16_autocast  ** BEST **
     Run under bfloat16 automatic mixed precision
     7.523s >>> 0.752s
     regime: compute (confidence: 0.85)

  2. [6.32x] tf32_tensor_cores
     Enable TF32 tensor cores (~2x matmul throughput on Ampere+)
     7.523s >>> 1.191s

  3. [6.22x] high_matmul_precision
     Set float32 matmul precision to 'high'
     7.523s >>> 1.210s

  4. [1.31x] cudnn_benchmark
     Enable cuDNN auto-tuner for conv kernels
     7.523s >>> 5.719s

--------------------------------------------------------------
  WINNER: bf16_autocast
  7.523s >>> 0.752s  (10.00x, +900% faster)
  How: Run under bfloat16 automatic mixed precision
--------------------------------------------------------------
```

## Why this exists

Every deep learning workload is stuck in one of three performance regimes ([Horace He, 2022](https://horace.io/brrr_intro.html)):

| Regime | What's happening | Wrong fix = no speedup |
|--------|-----------------|----------------------|
| **Compute-bound** | GPU is saturated doing math (matmuls, convolutions) | Fusing ops won't help — the math itself is the bottleneck |
| **Memory-bound** | GPU is waiting for data (pointwise ops, activations) | Smaller model won't help — you need less data movement |
| **Overhead-bound** | GPU is idle waiting for Python/dispatcher (tiny ops) | Lower precision won't help — you need fewer, bigger ops |

The fix for each regime is different. Applying a compute-bound fix to a memory-bound workload does nothing. Engineers waste hours in profiler UIs figuring this out manually.

slowai does it in one command.

## How it works

```bash
slowai diagnose model.py    # Classify the regime + prescribe fixes
slowai fix model.py          # ^ plus auto-apply fixes and measure speedup
```

Under the hood:

1. **Profile** — Runs your workload under `torch.profiler` with CUDA timing, warmup pass, and op-level statistics
2. **Classify** — A heuristic classifier analyzes op shares (matmul, normalization, pointwise, tiny-op fraction) to determine the dominant regime
3. **Prescribe** — Returns a ranked list of fixes for that regime, cheapest first
4. **Remediate** — Auto-applies each applicable fix (TF32 tensor cores, bf16/fp16 autocast, cuDNN benchmark, matmul precision), re-profiles, and ranks by measured speedup

No code changes required. Remedies are environment-level transforms — they modify PyTorch's runtime settings, not your model code.

## Benchmarks

Tested on NVIDIA Jetson Orin Nano Super (Ampere GPU, 1024 CUDA cores, JetPack 6.2, PyTorch 2.8.0):

| Workload | Regime | Baseline | Best remedy | After | Speedup |
|----------|--------|----------|-------------|-------|---------|
| Dense GEMM (4096x4096) | Compute | 7.523s | bf16_autocast | 0.752s | **10.00x** |
| Pointwise chain (8192x8192) | Memory | 2.400s | tf32_tensor_cores | 0.575s | **4.17x** |
| Tiny ops (5000 micro-ops) | Overhead | 3.281s | tf32_tensor_cores | 1.141s | **2.88x** |
| MobileNetV2 inference | Compute | 1.778s | cudnn_benchmark | 1.681s | **1.06x** |
| ResNet-50 inference | Compute | 2.105s | bf16_autocast | 1.969s | **1.07x** |

Production models (MobileNet, ResNet) show modest gains because PyTorch already optimizes common architectures well. The real value is that slowai **finds the right fix automatically** — cuDNN benchmark wins for convolution-heavy MobileNet, bf16 autocast wins for matmul-heavy ResNet. Different models, different winners, zero guesswork.

## Installation

```bash
git clone https://github.com/ricojallen37-sketch/slowai.git
cd slowai
pip install -e .
```

Requires Python 3.10+ and PyTorch >= 2.1 with CUDA support.

## Writing a workload

slowai profiles any Python script that exposes a `main()` function:

```python
# my_model.py
import torch
from torchvision.models import resnet50

model = resnet50().cuda().eval()
data = torch.randn(8, 3, 224, 224, device="cuda")

def main():
    with torch.no_grad():
        for _ in range(30):
            model(data)
```

```bash
slowai fix my_model.py
```

## Architecture

```
slowai/
  schema.py      # Regime enum, Diagnosis dataclass — the product thesis in types
  profiler.py    # torch.profiler wrapper → ProfileResult (op stats + wall time)
  diagnose.py    # Heuristic classifier → Diagnosis (regime + confidence + prescriptions)
  remediate.py   # Auto-fix engine → FixReport (before/after speedup per remedy)
  cli.py         # CLI entry points: diagnose, fix
```

The classifier is a pure function of profiler output — no torch dependency, fully unit-testable. The remediate engine applies fixes as environment transforms (global flags, autocast context managers) so it never modifies user code.

## What's different

Other tools in this space are profiler UIs that show you data and leave interpretation to you. slowai is the only tool that goes from raw workload to regime classification to ranked prescriptions to auto-applied fixes to measured speedup in a single CLI command.

| Tool | Profiles | Classifies regime | Prescribes fixes | Auto-applies | Measures speedup |
|------|----------|------------------|-----------------|-------------|-----------------|
| PyTorch Profiler | Yes | No | No | No | No |
| NVIDIA Nsight | Yes | No | No | No | No |
| torch.utils.bottleneck | Yes | No | No | No | No |
| DeepSpeed Flops Profiler | Yes | No | No | No | No |
| **slowai** | **Yes** | **Yes** | **Yes** | **Yes** | **Yes** |

## Roadmap

- **V1** (shipped) — Profile + classify regime for synthetic workloads
- **V2** (shipped) — Noise filtering (sync ops, init ops), normalization-aware classification, real model support
- **V3** (shipped) — Auto-remediate: apply fixes and measure before/after speedup
- **V4** (next) — torch.compile integration, channels_last auto-transform, batch size search, export optimized config

## Built by

Rico Allen — [@ricojallen37-sketch](https://github.com/ricojallen37-sketch)

Built and tested on NVIDIA Jetson Orin Nano Super Developer Kit.
