Metadata-Version: 2.5
Name: workingset
Version: 0.1.0
Summary: How many agentic-coding users a local LLM deployment serves: capacity model, CLI and live-endpoint hypothesis tests
Project-URL: Homepage, https://workingset.tomvaucourt.com
Project-URL: Repository, https://github.com/T0mSIlver/working-set
Author: Tom Vaucourt
License-Expression: MIT
License-File: LICENSE
Keywords: benchmark,capacity-planning,kv-cache,llm,serving,vllm
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: System :: Benchmark
Requires-Python: >=3.11
Requires-Dist: httpx>=0.27
Requires-Dist: numpy>=2.0
Provides-Extra: study
Requires-Dist: matplotlib>=3.9; extra == 'study'
Requires-Dist: pandas>=2.0; extra == 'study'
Requires-Dist: scipy>=1.13; extra == 'study'
Description-Content-Type: text/markdown

# Working Set

Tools for scaling **local LLM deployments**: how many concurrent users or
agents a given GPU configuration can keep warm, where KV cache, decode
bandwidth, and prefill compute each become the binding constraint, and which
knob (topology, dtypes, `max_num_seqs`, prompt caching) buys the most headroom
for agentic coding workloads.

**Start with the [interactive explorer](https://workingset.tomvaucourt.com/)** —
live sliders for the workload, model (Qwen3.8-27B / 35B-A3B /
Mistral-Medium-3.5 / GLM-5.3 / DeepSeek-V4-Flash / Qwen3.8-Flash-Next /
GLM-5.3-Flash), GPU (H200 / B300), weight & KV dtypes, and DP × TP
topology. It answers as a decision tool: a binding-constraint verdict, a
deploy recipe (vLLM flags), the **bill** (€/GPU-hour and €/kWh sliders:
hardware plus a duty-cycle power model), a **sensitivity panel** showing which assumption
would flip the decision, the **steady-state decode point** (how many
sessions are actually decoding at your load, and how fast each one runs —
Little's law, not the all-warm stress test), **shareable links** that
encode the whole configuration, and a **"Test these hypotheses" button**
that hands out the configuration on screen as a `workingset.toml` — feed it to
`ws test` below and measure the real limits on a live vLLM endpoint.

## The `workingset` package

The model behind the explorer is a Python package (`src/workingset/`, the
source of truth; the explorer's JS mirrors it). It ships a CLI:

```bash
uv run ws init --model Q38FN --gpu B300 --tp 8 --weight-dtype nvfp4   # writes workingset.toml
uv run ws predict workingset.toml        # the four ceilings, which one binds, the operating point
uv run ws predict workingset.toml --json # the same as a run record
uv run ws hypotheses                     # the H-* and what each one needs
uv run ws test workingset.toml --dry-run # the plan, the sampler self-check, no requests
uv run ws test workingset.toml --exclusive --out run.json   # measure it
uv run ws report run.json                # re-print the verdicts
uv run ws models                         # model / GPU keys
uv run pytest                            # self-checks + config round-trips
```

No checkout needed — the explorer's `workingset.toml` runs straight from git
(`--from` carries the package because `workingset` publishes one console
script, `ws`):

```bash
uvx --from git+https://github.com/T0mSIlver/working-set ws predict workingset.toml
uvx --from git+https://github.com/T0mSIlver/working-set ws test workingset.toml --dry-run
uvx --from git+https://github.com/T0mSIlver/working-set ws test workingset.toml --all --exclusive --out run.json
```

After the PyPI release the same commands shorten to `uvx --from workingset
ws …`.

Predictions live in no file: `ws predict` recomputes them from the config every
time, so a config can never carry a number the code did not produce. A harness
`.py` downloaded from the explorer before the package existed still loads (its
CONFIG block is extracted).

`ws test` puts the predictions to a live endpoint, one falsifiable hypothesis
at a time. Without `--exclusive` it runs only the hypotheses that need a
handful of requests (miss TTFT, the inter-token gap distribution, the steady
decode point) and lists the rest as skipped — a hypothesis that has to
generate its own population is never measured against someone else's load.
With `--exclusive` it drives the geometric load ladder once, and every ceiling
reads from it. `--burst N` adds the correlated-flush probe (B*).

## Contents

- [docs/writeup.md](docs/writeup.md) — baseline study: KV-cache capacity and
  the prompt-caching / offload / `max_num_seqs` trade-offs.
- [docs/scenarios.md](docs/scenarios.md) — extended scenario model: multi-GPU
  topologies, MoE vs dense, subagent workloads, the cost of a cache miss, and
  cold-spike tolerance.
- [scripts/](scripts/) — everything is reproducible:

  ```bash
  uv run ws selfcheck                # the shared model's self-checks (src/workingset/model.py)
  uv run scripts/scenarios.py        # renders the scenario figures
  uv run scripts/tables.py           # regenerates every number in docs/scenarios.md
  ```

- [research/](research/) — sourced constants for each model and GPU.
- [interactive/](interactive/) — the explorer, a dependency-free page
  mirroring the Python model: `index.html` holds the markup and styles,
  `src/*.js` the model and the charts as ES modules (`src/main.js` is the
  entry and lists the layering). Browsers refuse module scripts from
  `file://`, so serve the folder to open it locally:

  ```sh
  python3 -m http.server 8000 --directory interactive   # then http://localhost:8000
  ```

Method, calibration, and caveats are laid out in the docs above.

MIT licensed; see [LICENSE](LICENSE).
