Metadata-Version: 2.4
Name: ridgepoint
Version: 0.0.0
Summary: LLM inference sizing: engine-aware VRAM capacity, KV cache, and roofline intervals. Placeholder — under active development.
Author: Iskariot
License: MIT
Project-URL: Homepage, https://github.com/Isk4R1oT/ridgepoint
Project-URL: Repository, https://github.com/Isk4R1oT/ridgepoint
Keywords: llm,inference,vram,kv-cache,roofline,gpu,sizing,vllm
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# ridgepoint

**LLM inference sizing that models the engine, not just the weights.**

`ridgepoint` answers the question every VRAM calculator gets wrong: *after your serving
engine (vLLM, SGLang, …) grabs its memory pool, how many concurrent tokens of KV cache
actually fit — and where does throughput land on the latency/throughput frontier?*

The name is the **ridge point** of the roofline model: the arithmetic intensity where
decode stops being memory-bandwidth-bound. It's the single number this tool reasons about.

> 🚧 **Early development.** This `0.0.0` release reserves the name. v0 (Rust core + Python
> bindings via PyO3/maturin, one code path for both a CLI and `pip install ridgepoint`) is
> in progress.

## Why another calculator?

Most "will it fit?" tools compute `weights + KV < VRAM`. That is the wrong model for paged
engines: vLLM pre-reserves `gpu_memory_utilization × VRAM` (default 90%) at startup and
pages KV into that pool. ridgepoint models what actually decides the answer:

- **Engine-aware capacity** — usable KV pool *after* the pre-grab → max concurrent tokens.
- **Intervals, not points** — throughput sits on a Pareto frontier set by your latency budget.
- **MLA vs GQA KV geometry** — DeepSeek-style MLA is ~50–100× smaller than the GQA formula predicts.
- **Real quant bpw** — Q4_K_M is ~4.8 bits/weight, not the label's 4.5.
- **OOM honesty** — never silently under-predict memory.

## License

MIT
