Metadata-Version: 2.5
Name: smelt-gpu
Version: 0.2.2
Summary: This-host GPU efficiency profiler for vLLM: $/hour → $/1M tokens, then keep or revert one flag.
Project-URL: Homepage, https://github.com/Exciton-Research/smelt
Project-URL: Documentation, https://github.com/Exciton-Research/smelt/tree/main/docs
Project-URL: Repository, https://github.com/Exciton-Research/smelt
Author: Exciton Research
License-Expression: MIT
License-File: LICENSE
Keywords: efficiency,gpu,inference,nvidia,slo,vllm
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.11
Requires-Dist: fastapi>=0.115
Requires-Dist: httpx>=0.28
Requires-Dist: pydantic>=2.10
Requires-Dist: rich>=13.9
Requires-Dist: typer>=0.15
Requires-Dist: uvicorn[standard]>=0.34
Provides-Extra: dev
Requires-Dist: black>=25.0; extra == 'dev'
Requires-Dist: build>=1.2; extra == 'dev'
Requires-Dist: flake8>=7.0; extra == 'dev'
Requires-Dist: hatch>=1.14; extra == 'dev'
Requires-Dist: isort>=6.0; extra == 'dev'
Requires-Dist: pre-commit>=4.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Requires-Dist: twine>=5.0; extra == 'dev'
Description-Content-Type: text/markdown

# smelt

This-host **GPU efficiency profiler for vLLM**. One model, one server.

Objective: minimize **$/1M tokens** subject to your p95 latency SLO.

You already know the GPU-hour price. Smelt turns a live `nvidia-smi` +
vLLM `/metrics` sample into `$/hour → tok/s → $/1M tokens`, names waste
it can measure, and tells you whether the last flag change won.

```
pip install smelt-gpu && smelt analyze --demo --gpu-hourly 3.89
make run                       # this repo: install from PyPI + scan/analyze/optimize
make install                   # editable checkout + dashboard
smelt scan --vllm-url http://127.0.0.1:8000
smelt analyze --gpu-hourly 3.89
smelt experiment save --name before
# change one vLLM flag, reload
smelt experiment compare --baseline before --slo-ms 500
make dev                       # API :8741 + dashboard :3010
```

PyPI name is `smelt-gpu`; the CLI stays `smelt`. `smelt --version` is the version.

Smelt does **not** restart vLLM or edit its flags. You apply the change;
it records and compares. Keep the candidate only if `$/1M` dropped (or
same cost, more tok/s) and p95 still meets the SLO.

If nothing is live, `analyze` falls back to a sample fleet
(96×H100 + 32×H200). `--demo` forces that.

The wheel is the CLI + `smelt serve`. The Next.js dashboard is this repo
only (`make dev`). Full notes: [docs/](docs/README.md).

## Use it next to your vLLM

```
pip install smelt-gpu
smelt analyze --vllm-url http://127.0.0.1:8000 --gpu-hourly 3.89
```

In Python:

```python
from smelt.integrations.scan import collect_host
from smelt.snapshot import fleet_from_snapshot, host_summary

snap = collect_host(vllm_url="http://127.0.0.1:8000")
fleet = fleet_from_snapshot(snap, gpu_hourly=3.89)
print(host_summary(fleet))
```

Or `smelt serve` and `GET /api/fleet?live=true`. How to wire this into a
job or another service: [docs/use.md](docs/use.md).

## Improve it

```
make install && make hooks
make test
make test-live          # this GPU + a server
make verify
```

New GPU prices → `classify_gpu` / `GPU_HOURLY_USD`. New `/metrics`
fields → `smelt/integrations/vllm.py` plus a fixture under
`tests/fixtures/`. Waste rules → `_opportunities()` in `snapshot.py`.
Details: [docs/use.md](docs/use.md).
