Metadata-Version: 2.5
Name: llamastat
Version: 0.1.0
Summary: nvtop for llama.cpp: an interactive terminal dashboard for llama-server inference monitoring
Project-URL: Homepage, https://github.com/bryanvine/llamastat
Project-URL: Issues, https://github.com/bryanvine/llamastat/issues
Author-email: Bryan Vine <bryan@bryanvine.com>
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: gguf,llama-server,llama.cpp,llm,monitoring,tui
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: System :: Monitoring
Requires-Python: >=3.10
Requires-Dist: httpx>=0.27
Requires-Dist: prometheus-client>=0.20
Requires-Dist: textual>=0.60
Provides-Extra: dev
Requires-Dist: pyright>=1.1; extra == 'dev'
Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.5; extra == 'dev'
Description-Content-Type: text/markdown

# llamastat

**nvtop for llama.cpp**: a zero-infrastructure terminal dashboard for `llama-server` inference. Point it at a running server and watch slots, throughput, and context usage live.

![llamastat](https://raw.githubusercontent.com/bryanvine/llamastat/main/docs/screenshot.svg)

---

## Why llamastat?

`llama-server` tells you what it is doing, but only in raw JSON. llamastat turns that into a live picture:

- **Per-slot activity**: which slots are processing prompts, which are generating, and how full each context window is.
- **Throughput without `--metrics`**: llamastat diffs slot token counters between polls, so you get generation and prompt tokens/s even on servers started without the Prometheus endpoint.
- **Zero infrastructure**: no Prometheus, no Grafana, no agents. One `pip install`, one command, works over any HTTP reachable server (including an SSH port-forward to a production box).
- **Degrades gracefully**: if the server flaps, is still loading the model, or has `/slots` disabled, llamastat shows what it can instead of crashing.

Sibling project: [vllmstat](https://github.com/bryanvine/vllmstat) does the same for vLLM.

## Install

```bash
pip install llamastat
```

## Usage

```bash
llamastat                          # local server on the default port
llamastat -u http://host:8080      # remote server
llamastat -u http://host:8080 -i 0.5
llamastat --once --json            # one snapshot as JSON, for scripting
llamastat --mock                   # synthetic data, no server needed
```

### Key bindings

| Key | Action |
| --- | ------ |
| `q` | Quit |
| `p` | Pause/resume polling |
| `+` | Poll faster (halve interval, min 0.1s) |
| `-` | Poll slower (double interval, max 10s) |

### Flags

| Flag | Default | Meaning |
| ---- | ------- | ------- |
| `-u, --url` | `http://localhost:8080` | llama-server base URL |
| `-i, --interval` | `1.0` | poll interval in seconds |
| `--api-key` | `$LLAMACPP_API_KEY` | bearer token for servers started with `--api-key` |
| `--metrics-path` | `/metrics` | Prometheus metrics path |
| `--once` | off | poll once, print, exit |
| `--json` | off | with `--once`, print machine-readable JSON |
| `--mock` | off | run against synthetic data |

## What it shows

- **Header**: model alias, quantization, parameter count, file size, context per slot, training context, server build, health.
- **Slots**: one row per slot with phase (`prompt`/`gen`/`idle`), task id, prompt progress, generated tokens, per-slot tokens/s, and a context-usage bar.
- **Throughput**: EWMA generation and prompt tokens/s with session peaks and totals, plus a braille history plot.
- **Requests**: running/peak concurrency, completed tasks, aggregate context in use.
- **Server metrics**: when the server was started with `--metrics`, llamastat also shows the server-side Prometheus counters (total tokens, deferred requests, context high-water mark, busy slots per decode). The panel hides itself otherwise.
- **Sampling defaults**: temperature, top_p, top_k, min_p, max_tokens from `/props`.

llamastat only issues read-only GET requests (`/health`, `/slots`, `/props`, `/v1/models`, and optionally `/metrics`), so it is safe to point at a production server.

## Requirements

- Python 3.10+
- A llama.cpp `llama-server` new enough to expose `/slots` (enabled by default; some deployments disable it, in which case llamastat falls back to header and metrics panels)

## Development

```bash
git clone https://github.com/bryanvine/llamastat
cd llamastat
pip install -e ".[dev]"
pytest -q
```

See [CONTRIBUTING.md](CONTRIBUTING.md).

## License

Apache-2.0. See [LICENSE](LICENSE).
