Metadata-Version: 2.5
Name: tokenwatt-check
Version: 0.1.0
Summary: Check whether the GPU you rented performs like it should: power cap, clocks under load, compute, memory bandwidth, PCIe, disk and network.
Project-URL: Homepage, https://www.tokenwatt.io
Project-URL: Methodology, https://www.tokenwatt.io/methodology.html
Author-email: TokenWatt <hello@tokenwatt.io>
License-Expression: MIT
License-File: LICENSE
Keywords: benchmark,cloud,gpu,h100,inference,llm,nvidia,runpod,vast.ai
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: System :: Benchmark
Classifier: Topic :: System :: Hardware
Requires-Python: >=3.9
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == 'dev'
Provides-Extra: torch
Requires-Dist: torch>=2.1; extra == 'torch'
Description-Content-Type: text/markdown

# tokenwatt-check

**Is the GPU you rented performing like it should?**

GPU listings tell you the model — "H100 SXM, verified, 99.5% reliability" — not the card you get. When we
[benchmarked four rented H100s](https://www.tokenwatt.io/blog/runpod-vs-vast-vs-lambda-h100.html), two had their
power limit lowered by the host and one ran at **787 MHz instead of ~1,900** under load: every token took 2.9× longer.
Nothing on the listing showed it.

`tokenwatt-check` finds that in about two minutes:

```
pip install tokenwatt-check torch    # torch is usually already there on GPU images
tokenwatt-check
```

```
  OVERALL: ❌ FAIL

  ⚠️  power limit            GPU power limit is 500 W of a possible 700 W (71%).
  ❌ clocks under load      SM clock during the matmul averaged 560 MHz — 41% of a healthy H100 (1360 MHz).
                            → Far below normal. Possible causes: aggressive power cap, poor cooling or
                              power delivery, a degraded card. Ask for another machine.
  ❌ BF16 matmul            280 TFLOPS — 41% of the healthy H100 reference (680 TFLOPS).
  ❌ memory copy bandwidth  2.2 TB/s — 73% of the healthy H100 reference (3.0 TB/s).
  ✅ PCIe link              Gen5 x16.
  ✅ disk                   Sequential write 1,616 MB/s.
  ✅ download               2,100 Mbit/s from Hugging Face, one stream (~6.3 min per 100 GB of weights).
```

(Illustrative output modeled on a real under-performing node — see `src/tokenwatt_check/demo.py`. Try `tokenwatt-check --demo throttled-h100`, `capped-h100` or `healthy-h100` without a GPU.)

## What it checks

| Check | How | Why it matters |
|---|---|---|
| Power limit | `nvidia-smi` power.limit vs power.max_limit | Hosts lower it to save power; throughput drops under load |
| Clocks under load | SM clock sampled every second during a 30 s BF16 matmul, compared with a healthy card of the same model in the same test | The clearest sign of a capped, hot or degraded card |
| Throttle reasons | nvidia-smi clock event reasons during load | Hardware or thermal slowdown means the card is protecting itself |
| Compute | Sustained BF16 GEMM (8192²) in TFLOPS | Prompt processing and large batches scale with it |
| Memory bandwidth | Device-to-device copy in TB/s | LLM token generation follows it closely |
| PCIe link | Generation and width under load vs the card's maximum | Slow links slow model loading |
| Memory errors | Uncorrected ECC errors, pending retired pages | Unreliable memory |
| Disk | 2 GB sequential write and read | How long weights take to load from disk |
| Download | 20 s from the Hugging Face CDN | You pay for the minutes spent downloading weights |

Results: `PASS`, `WARN` or `FAIL` per check with a reason and what to do, plus a JSON file with every reading
(`--json path`) and an optional Markdown report (`--md path`). Exit code 2 on `FAIL`, for scripts.

## Options

```
tokenwatt-check --gpu 1            # another GPU
tokenwatt-check --seconds 60       # longer load test
tokenwatt-check --no-load          # inventory, disk and network only (no PyTorch needed)
tokenwatt-check --no-net --no-disk
```

## Reference values

Compute, bandwidth and clock-under-load are compared with a healthy card of the same model (`src/tokenwatt_check/verdict.py`):

| GPU | BF16 matmul | Clock during matmul | Copy bandwidth | Source |
|---|---|---|---|---|
| H100 SXM | 680 TFLOPS | 1,360 MHz | 3.0 TB/s | measured, RunPod Secure Cloud, 700 W |
| H200 | 660 TFLOPS | 1,460 MHz | 4.3 TB/s | measured, RunPod Secure Cloud, 700 W |
| others | | | | provisional (vendor specs, public measurements) |

A dense matmul pulls a healthy card to its power limit, so its clock sits well below the maximum — a healthy H100 holds
about 69% of max clock in this test. That's why the clock check compares against a healthy card rather than the maximum.
If a healthy card scores low on your machine, please open an issue with your JSON so we can improve the references.

## Privacy

Everything runs locally. Nothing is sent anywhere except the 20-second download test from huggingface.co.

## Supported

NVIDIA GPUs with `nvidia-smi` (H100, H200, B200, A100, L40S, RTX 4090/5090 and others). AMD Instinct support is planned.

## About

Built by [TokenWatt](https://www.tokenwatt.io), which publishes vendor-neutral measurements of LLM inference —
SLO goodput, tokens per joule and cost per million tokens — on NVIDIA and AMD. MIT licensed.
