# vllmstat v0.3 — Fleet / multi-instance — Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:subagent-driven-development. TDD, checkbox steps. Spec: `docs/superpowers/specs/2026-06-14-fleet-design.md`.

**Goal:** Monitor many vLLM instances (local Docker + remote) from one process, with an nvtop-style
overview + drill-in to the existing per-instance dashboard. Single-instance UX is unchanged.

**Architecture:** N × `InstanceRuntime` (VllmProvider+MetricsEngine+History) polled concurrently
with `asyncio.gather`; instances resolved from TOML config + `--discover-docker` + repeatable
`--url`; one host `GpuProvider` sliced per local instance; Textual app swaps between a pure-rendered
overview and the existing detail panels.

**Conventions:** Python ≥3.10, `from __future__ import annotations`, dataclasses, src layout, ruff +
pyright clean, pytest with `tmp_path`/stub seams (no network/docker/GPU in tests). Activate `.venv`
before pytest/ruff/pyright. Commit after each task.

---

## Task 1: `Instance` + `FleetSnapshot` dataclasses

**Files:** Modify `src/vllmstat/core/state.py`; Test `tests/test_state.py`.

- [ ] **Step 1 — failing tests** (append to `tests/test_state.py`):

```python
def test_instance_defaults():
    from vllmstat.core.state import Instance
    i = Instance(name="a", url="http://localhost:8000")
    assert (i.metrics_path, i.api_key, i.gpus, i.locality) == ("/metrics", None, (), "local")


def test_fleet_snapshot_defaults():
    from vllmstat.core.state import FleetSnapshot
    fs = FleetSnapshot(ts=1.0)
    assert fs.items == [] and fs.gpu.available is False
```

- [ ] **Step 2 — run, expect ImportError/fail:** `pytest tests/test_state.py -q`
- [ ] **Step 3 — implement** (add to `core/state.py`, after `GpuSnapshot`):

```python
@dataclass
class Instance:
    name: str
    url: str
    metrics_path: str = "/metrics"
    api_key: str | None = None
    gpus: tuple[int, ...] = ()
    locality: str = "local"  # "local" | "remote"


@dataclass
class FleetSnapshot:
    ts: float
    items: list[tuple["Instance", "Snapshot"]] = field(default_factory=list)
    gpu: GpuSnapshot = field(default_factory=GpuSnapshot)
```

- [ ] **Step 4 — run, expect pass.** `pytest tests/test_state.py -q`
- [ ] **Step 5 — commit:** `feat(fleet): Instance and FleetSnapshot dataclasses`

---

## Task 2: Config — repeatable `--url`, `--config`, `--discover-docker`

**Files:** Modify `src/vllmstat/config.py`; Test `tests/test_config.py`.

`Config` keeps single-instance scalars as defaults; adds `urls`, `config_path`, `discover_docker`,
and a resolved `instances` list (populated later, not in `from_sources`). `url` becomes a back-compat
property.

- [ ] **Step 1 — failing tests** (append):

```python
def test_urls_repeatable():
    c = Config.from_sources(["--url", "http://a:8000", "-u", "http://b:8001"], {})
    assert c.urls == ["http://a:8000", "http://b:8001"]
    assert c.url == "http://a:8000"          # back-compat: first


def test_url_default_when_none():
    c = Config.from_sources([], {})
    assert c.urls == [] and c.url == "http://localhost:8000"


def test_config_and_discover_flags():
    c = Config.from_sources(["--config", "/tmp/x.toml", "--discover-docker"], {})
    assert c.config_path == "/tmp/x.toml" and c.discover_docker is True
```

Update any existing test that constructs `Config(url=...)` to use `urls=[...]` (and read `.url`).

- [ ] **Step 2 — run, expect fail.**
- [ ] **Step 3 — implement** (`config.py`):

```python
from dataclasses import dataclass, field
from vllmstat.core.state import Instance

@dataclass
class Config:
    urls: list[str] = field(default_factory=list)
    metrics_path: str = "/metrics"
    interval: float = 1.0
    api_key: str | None = None
    gpu: bool = True
    mock: bool = False
    once: bool = False
    json: bool = False
    config_path: str | None = None
    discover_docker: bool = False
    instances: list[Instance] = field(default_factory=list)

    @property
    def url(self) -> str:
        return self.urls[0] if self.urls else "http://localhost:8000"
```

In `build_parser`, replace the `--url` line and add two flags:

```python
        p.add_argument("-u", "--url", action="append", dest="urls", default=None, metavar="URL")
        p.add_argument("--config", dest="config_path", default=None)
        p.add_argument("--discover-docker", dest="discover_docker", action="store_true", default=False)
```

In `from_sources`, build with `urls=list(ns.urls or [])`, `config_path=ns.config_path`,
`discover_docker=ns.discover_docker` (drop the old `url=`).

- [ ] **Step 4 — run, expect pass** (whole suite may have ripples; fix `Config(url=...)` call sites,
  e.g. `app.py`/`cli.py` read `cfg.url` — the property covers them). `pytest tests/test_config.py -q`
- [ ] **Step 5 — commit:** `feat(fleet): repeatable --url, --config, --discover-docker flags`

---

## Task 3: Config file loader + fleet resolution

**Files:** Create `src/vllmstat/core/config_file.py`, `src/vllmstat/core/resolve.py`; Modify
`pyproject.toml` (add `tomli` for py<3.11); Test `tests/test_config_file.py`, `tests/test_resolve.py`.

- [ ] **Step 1 — failing tests** `tests/test_config_file.py`:

```python
import pytest
from vllmstat.core.config_file import parse_config, find_config

def test_parse_config_instances_and_globals():
    insts, g = parse_config('interval = 2.0\n[[instance]]\nname="a"\nurl="http://a:8000"\ngpus=[0]\n')
    assert g["interval"] == 2.0
    assert insts == [{"name": "a", "url": "http://a:8000", "gpus": [0]}]

def test_parse_config_bad_toml_raises():
    with pytest.raises(ValueError):
        parse_config("this is = = not toml")

def test_parse_config_instance_must_be_array():
    with pytest.raises(ValueError):
        parse_config('instance = "oops"')

def test_find_config_precedence():
    seen = {"/explicit.toml", "./vllmstat.toml"}
    assert find_config("/explicit.toml", {}, exists=lambda p: p in seen) == "/explicit.toml"
    assert find_config(None, {"VLLMSTAT_CONFIG": "/env.toml"}, exists=lambda p: False) == "/env.toml"
    assert find_config(None, {}, exists=lambda p: p in seen) == "./vllmstat.toml"
    assert find_config(None, {}, exists=lambda p: False) is None
```

`tests/test_resolve.py`:

```python
from vllmstat.core.resolve import (
    normalize_url, derive_name, classify_locality, instance_from_dict, resolve_fleet,
)
LN = {"localhost", "127.0.0.1"}

def test_normalize_url():
    assert normalize_url("LocalHost:8000/") == "http://localhost:8000"
    assert normalize_url("http://h:9/metrics/") == "http://h:9/metrics"

def test_classify_locality():
    assert classify_locality("http://localhost:8000", LN) == "local"
    assert classify_locality("http://gpu-box:8000", LN) == "remote"

def test_instance_from_dict_gpus_and_name():
    i = instance_from_dict({"url": "http://localhost:8000", "gpus": [0, 1]}, local_names=LN)
    assert i.gpus == (0, 1) and i.name == "localhost:8000" and i.locality == "local"

def test_instance_from_dict_local_override():
    i = instance_from_dict({"url": "http://gpu-box:8000", "local": True}, local_names=LN)
    assert i.locality == "local"

def test_resolve_fleet_merges_and_dedupes():
    cfg = [instance_from_dict({"url": "http://localhost:8000", "name": "a"}, local_names=LN)]
    fleet = resolve_fleet(cfg, [], ["http://localhost:8000", "http://b:8001"], local_names=LN)
    urls = [i.url for i in fleet]
    assert urls == ["http://localhost:8000", "http://b:8001"]   # dup dropped, order kept

def test_resolve_fleet_default_when_empty():
    fleet = resolve_fleet([], [], [], local_names=LN)
    assert len(fleet) == 1 and fleet[0].url == "http://localhost:8000"

def test_resolve_fleet_dedupes_names():
    a = instance_from_dict({"url": "http://a:8000", "name": "x"}, local_names=LN)
    b = instance_from_dict({"url": "http://b:8000", "name": "x"}, local_names=LN)
    out = resolve_fleet([a, b], [], [], local_names=LN)
    assert [i.name for i in out] == ["x", "x#2"]
```

- [ ] **Step 2 — run, expect fail.**
- [ ] **Step 3 — implement `core/config_file.py`:**

```python
from __future__ import annotations
import sys
from pathlib import Path

if sys.version_info >= (3, 11):
    import tomllib
else:
    import tomli as tomllib


def parse_config(text: str) -> tuple[list[dict], dict]:
    try:
        data = tomllib.loads(text)
    except tomllib.TOMLDecodeError as e:
        raise ValueError(f"invalid TOML: {e}") from e
    if not isinstance(data, dict):
        raise ValueError("config root must be a table")
    raw = data.get("instance", [])
    if not isinstance(raw, list):
        raise ValueError("'instance' must be an array of tables ([[instance]])")
    globals_ = {k: v for k, v in data.items() if k != "instance"}
    return raw, globals_


def load_config(path: str) -> tuple[list[dict], dict]:
    return parse_config(Path(path).expanduser().read_text())


def find_config(explicit, env, *, candidates=None, exists=None) -> str | None:
    exists = exists or (lambda p: Path(p).expanduser().is_file())
    if explicit:
        return explicit
    if env.get("VLLMSTAT_CONFIG"):
        return env["VLLMSTAT_CONFIG"]
    for c in candidates or ["./vllmstat.toml", "~/.config/vllmstat/config.toml"]:
        if exists(c):
            return c
    return None
```

Implement `core/resolve.py`:

```python
from __future__ import annotations
import socket
from dataclasses import replace
from urllib.parse import urlparse
from vllmstat.core.state import Instance

_LOCAL = {"localhost", "127.0.0.1", "::1", "0.0.0.0", ""}


def normalize_url(url: str) -> str:
    u = url.strip()
    if "://" not in u:
        u = "http://" + u
    p = urlparse(u)
    host = (p.hostname or "").lower()
    port = f":{p.port}" if p.port else ""
    return f"{p.scheme}://{host}{port}{p.path.rstrip('/')}"


def derive_name(url: str) -> str:
    p = urlparse(normalize_url(url))
    return f"{p.hostname}:{p.port}" if p.port else (p.hostname or url)


def classify_locality(url: str, local_names: set[str]) -> str:
    host = (urlparse(normalize_url(url)).hostname or "").lower()
    return "local" if host in local_names else "remote"


def local_hostnames() -> set[str]:
    names = set(_LOCAL)
    try:
        h = socket.gethostname()
        names |= {h.lower(), socket.getfqdn().lower()}
        for info in socket.getaddrinfo(h, None):
            names.add(str(info[4][0]).lower())
    except OSError:
        pass
    return names


def instance_from_dict(raw, *, defaults_api_key=None, defaults_metrics_path="/metrics", local_names) -> Instance:
    url = raw.get("url")
    if not url:
        raise ValueError("instance is missing required 'url'")
    locality = ("local" if raw["local"] else "remote") if "local" in raw else classify_locality(url, local_names)
    return Instance(
        name=raw.get("name") or derive_name(url),
        url=normalize_url(url),
        metrics_path=raw.get("metrics_path", defaults_metrics_path),
        api_key=raw.get("api_key", defaults_api_key),
        gpus=tuple(int(g) for g in raw.get("gpus", [])),
        locality=locality,
    )


def instance_from_url(url, **kw) -> Instance:
    return instance_from_dict({"url": url}, **kw)


def resolve_fleet(config_instances, docker_instances, url_flags, *,
                  defaults_api_key=None, defaults_metrics_path="/metrics",
                  default_url="http://localhost:8000", local_names) -> list[Instance]:
    by_url, order = {}, []

    def add(inst):
        key = normalize_url(inst.url)
        if key not in by_url:
            order.append(key)
            by_url[key] = inst

    for i in config_instances:
        add(i)
    for i in docker_instances:
        add(i)
    for u in url_flags:
        add(instance_from_url(u, defaults_api_key=defaults_api_key,
                              defaults_metrics_path=defaults_metrics_path, local_names=local_names))
    if not order:
        add(instance_from_url(default_url, defaults_api_key=defaults_api_key,
                              defaults_metrics_path=defaults_metrics_path, local_names=local_names))
    return _dedupe_names([by_url[k] for k in order])


def _dedupe_names(instances):
    seen, out = {}, []
    for inst in instances:
        if inst.name in seen:
            seen[inst.name] += 1
            inst = replace(inst, name=f"{inst.name}#{seen[inst.name]}")
        else:
            seen[inst.name] = 1
        out.append(inst)
    return out
```

- [ ] **Step 4 — pyproject:** add to `[project].dependencies`: `'tomli; python_version < "3.11"'`.
- [ ] **Step 5 — run, expect pass.** `pytest tests/test_config_file.py tests/test_resolve.py -q`
- [ ] **Step 6 — commit:** `feat(fleet): TOML config loader + fleet resolution (merge/locality/dedupe)`

---

## Task 4: Docker discovery

**Files:** Create `src/vllmstat/providers/discover_docker.py`; Test `tests/test_discover_docker.py`.

All parsing is pure over captured JSON; only `discover_docker(run=...)` shells out (injectable).

- [ ] **Step 1 — failing tests** `tests/test_discover_docker.py`:

```python
from vllmstat.providers.discover_docker import (
    is_vllm_container, parse_ps, host_port_from_inspect, gpus_from_inspect, build_instances,
    discover_docker,
)

def test_is_vllm_container():
    assert is_vllm_container({"Image": "vllm/vllm-openai:latest", "Command": "python -m x"})
    assert not is_vllm_container({"Image": "nginx", "Command": "nginx -g"})

def test_parse_ps_jsonl():
    rows = parse_ps('{"ID":"abc","Image":"vllm/vllm-openai"}\n\n{"ID":"def","Image":"nginx"}\n')
    assert [r["ID"] for r in rows] == ["abc", "def"]

def test_host_port_prefers_8000():
    insp = {"NetworkSettings": {"Ports": {
        "8000/tcp": [{"HostPort": "9001"}], "22/tcp": [{"HostPort": "2222"}]}}}
    assert host_port_from_inspect(insp) == 9001

def test_gpus_device_requests():
    insp = {"HostConfig": {"DeviceRequests": [
        {"Capabilities": [["gpu"]], "DeviceIDs": ["0", "1"]}]}}
    assert gpus_from_inspect(insp) == (0, 1)

def test_gpus_visible_devices_env():
    insp = {"Config": {"Env": ["FOO=1", "NVIDIA_VISIBLE_DEVICES=0,2"]}}
    assert gpus_from_inspect(insp) == (0, 2)

def test_gpus_all_uses_host_count():
    insp = {"HostConfig": {"DeviceRequests": [{"Capabilities": [["gpu"]], "Count": -1}]}}
    assert gpus_from_inspect(insp, host_gpu_count=2) == (0, 1)

def test_build_instances_filters_and_maps():
    ps = [{"ID": "abc", "Image": "vllm/vllm-openai", "Command": "python -m vllm"},
          {"ID": "def", "Image": "nginx", "Command": "nginx"}]
    insp = {"abc": {"Name": "/qwen", "NetworkSettings": {"Ports": {"8000/tcp": [{"HostPort": "9000"}]}},
                    "HostConfig": {"DeviceRequests": [{"Capabilities": [["gpu"]], "DeviceIDs": ["0"]}]}}}
    out = build_instances(ps, insp)
    assert len(out) == 1
    assert out[0].name == "qwen" and out[0].url == "http://localhost:9000"
    assert out[0].gpus == (0,) and out[0].locality == "local"

def test_discover_docker_with_stub_run():
    calls = []
    def run(cmd):
        calls.append(cmd)
        if cmd[1] == "ps":
            return '{"ID":"abc","Image":"vllm/vllm-openai","Command":"python -m vllm"}\n'
        return ('[{"Name":"/qwen","NetworkSettings":{"Ports":{"8000/tcp":[{"HostPort":"9000"}]}},'
                '"HostConfig":{"DeviceRequests":[{"Capabilities":[["gpu"]],"DeviceIDs":["0"]}]}}]')
    out = discover_docker(run=run)
    assert len(out) == 1 and out[0].url == "http://localhost:9000"

def test_discover_docker_failure_returns_empty():
    def run(cmd):
        raise FileNotFoundError("docker missing")
    assert discover_docker(run=run) == []
```

- [ ] **Step 2 — run, expect fail.**
- [ ] **Step 3 — implement `providers/discover_docker.py`** (signatures matching the tests):

```python
from __future__ import annotations
import json
import shutil
import subprocess
from vllmstat.core.resolve import normalize_url
from vllmstat.core.state import Instance

VLLM_MARKERS = ("vllm", "api_server", "openai.api_server")


def is_vllm_container(row) -> bool:
    blob = f"{row.get('Image', '')} {row.get('Command', '')}".lower()
    return any(m in blob for m in VLLM_MARKERS)


def parse_ps(out: str) -> list[dict]:
    rows = []
    for line in out.splitlines():
        line = line.strip()
        if line:
            try:
                rows.append(json.loads(line))
            except json.JSONDecodeError:
                continue
    return rows


def host_port_from_inspect(insp, *, container_port="8000") -> int | None:
    ports = (insp.get("NetworkSettings") or {}).get("Ports") or {}
    binds = ports.get(f"{container_port}/tcp")
    if binds:
        return int(binds[0]["HostPort"])
    for key, b in ports.items():
        if key.endswith("/tcp") and b:
            return int(b[0]["HostPort"])
    return None


def gpus_from_inspect(insp, *, host_gpu_count=None) -> tuple[int, ...]:
    hc = insp.get("HostConfig") or {}
    for req in hc.get("DeviceRequests") or []:
        caps = req.get("Capabilities") or []
        if any("gpu" in c for grp in caps for c in grp):
            ids = req.get("DeviceIDs") or []
            got = tuple(int(i) for i in ids if str(i).isdigit())
            if got:
                return got
            if req.get("Count") in (-1, None) and host_gpu_count:
                return tuple(range(host_gpu_count))
    for e in (insp.get("Config") or {}).get("Env") or []:
        if e.startswith("NVIDIA_VISIBLE_DEVICES="):
            val = e.split("=", 1)[1]
            if val in ("all", "") and host_gpu_count:
                return tuple(range(host_gpu_count))
            got = tuple(int(t) for t in val.split(",") if t.strip().isdigit())
            if got:
                return got
    return ()


def _name(insp, row) -> str:
    n = (insp.get("Name") or "").lstrip("/")
    return n or row.get("Names") or (row.get("ID") or "")[:12]


def build_instances(ps_rows, inspect_by_id, *, host_gpu_count=None, default_metrics_path="/metrics"):
    out = []
    for row in ps_rows:
        if not is_vllm_container(row):
            continue
        cid = row.get("ID") or row.get("Id") or ""
        insp = inspect_by_id.get(cid) or {}
        port = host_port_from_inspect(insp)
        if not port:
            continue
        out.append(Instance(
            name=_name(insp, row),
            url=normalize_url(f"http://localhost:{port}"),
            metrics_path=default_metrics_path,
            gpus=gpus_from_inspect(insp, host_gpu_count=host_gpu_count),
            locality="local",
        ))
    return out


def discover_docker(*, run=None, host_gpu_count=None):
    run = run or _default_run
    try:
        rows = parse_ps(run(["docker", "ps", "--no-trunc", "--format", "{{json .}}"]))
        inspect_by_id = {}
        for r in rows:
            if not is_vllm_container(r):
                continue
            cid = r.get("ID") or r.get("Id") or ""
            if not cid:
                continue
            data = json.loads(run(["docker", "inspect", cid]))
            if isinstance(data, list) and data:
                inspect_by_id[cid] = data[0]
        return build_instances(rows, inspect_by_id, host_gpu_count=host_gpu_count)
    except Exception:  # noqa: BLE001 - discovery is best-effort, never crash the app
        return []


def _default_run(cmd):
    if not shutil.which("docker"):
        raise FileNotFoundError("docker not found")
    return subprocess.run(cmd, capture_output=True, text=True, timeout=10, check=True).stdout
```

- [ ] **Step 4 — run, expect pass.** `pytest tests/test_discover_docker.py -q`
- [ ] **Step 5 — commit:** `feat(fleet): best-effort Docker discovery of vLLM containers`

---

## Task 5: Fleet runtime (`InstanceRuntime`, `Fleet`, GPU slicing) + mock fleet

**Files:** Create `src/vllmstat/core/fleet.py`; Modify `src/vllmstat/providers/mock.py`; Test
`tests/test_fleet.py`.

- [ ] **Step 1 — failing tests** `tests/test_fleet.py`:

```python
import asyncio
from vllmstat.core.fleet import InstanceRuntime, Fleet, slice_gpu, build_fleet
from vllmstat.core.state import GpuSample, GpuSnapshot, Instance
from vllmstat.providers.vllm import ModelInfo, RawText

HOST = GpuSnapshot(available=True, source="multi", gpus=[
    GpuSample(index=0, name="A", vendor="intel", util_gpu=100.0),
    GpuSample(index=1, name="B", vendor="nvidia", util_gpu=50.0)])

def test_slice_gpu_subset():
    sub = slice_gpu(HOST, (1,))
    assert sub.available and [g.index for g in sub.gpus] == [1]

def test_slice_gpu_empty_mapping():
    assert slice_gpu(HOST, ()).available is False

class FakeProvider:
    def __init__(self, text, ok=True):
        self._text, self._ok = text, ok
        self.closed = False
    async def fetch_metrics(self):
        if self._text is None:
            raise RuntimeError("boom")
        return RawText(text=self._text, fetched_ok=self._ok, error=None if self._ok else "down")
    async def fetch_model_info(self):
        return ModelInfo(model_names=["m"], max_model_len=None, root=None)
    async def aclose(self):
        self.closed = True

METRICS = ('# TYPE vllm:num_requests_running gauge\n'
           'vllm:num_requests_running{model_name="m"} 3.0\n')

def test_runtime_poll_ok():
    inst = Instance(name="a", url="http://localhost:8000")
    rt = InstanceRuntime(inst, provider=FakeProvider(METRICS))
    snap = asyncio.run(rt.poll(1.0))
    assert snap.connected and snap.running == 3.0
    assert rt.history.series("running").values[-1] == 3.0

def test_fleet_poll_isolates_failures_and_slices_gpu():
    a = InstanceRuntime(Instance("a", "http://localhost:8000", gpus=(0,), locality="local"),
                        provider=FakeProvider(METRICS))
    b = InstanceRuntime(Instance("b", "http://gpu-box:8000", locality="remote"),
                        provider=FakeProvider(None))     # raises
    fleet = Fleet([], runtimes=[a, b])
    fs = asyncio.run(fleet.poll(HOST, 1.0))
    assert fs.items[0][1].connected is True
    assert [g.index for g in fs.items[0][1].gpu.gpus] == [0]   # local sliced
    assert fs.items[1][1].connected is False                   # failure isolated
    assert fs.items[1][1].gpu.available is False               # remote → no gpu

def test_build_fleet_from_instances():
    fleet = build_fleet([Instance("a", "http://localhost:8000")])
    assert len(fleet.runtimes) == 1
```

- [ ] **Step 2 — run, expect fail.**
- [ ] **Step 3 — implement `core/fleet.py`** (use the code in the spec §6/§7; key points: `poll`
  uses `asyncio.gather(..., return_exceptions=True)`; local → `slice_gpu(host, gpus)`, remote →
  `GpuSnapshot(available=False, source="remote")`; `InstanceRuntime` mirrors today's `_ensure_dims`
  + `_tick_body` + `_push_history`):

```python
from __future__ import annotations
import asyncio
from vllmstat.core.history import History
from vllmstat.core.metrics import MetricsEngine
from vllmstat.core.parse import parse_metrics
from vllmstat.core.state import FleetSnapshot, GpuSnapshot, Instance, Snapshot
from vllmstat.model_dims import load_model_dims
from vllmstat.providers.vllm import VllmProvider


def slice_gpu(host: GpuSnapshot, gpus: tuple[int, ...]) -> GpuSnapshot:
    if not host.available or not gpus:
        return GpuSnapshot(available=False, source=host.source)
    want = set(gpus)
    sub = [g for g in host.gpus if g.index in want]
    return GpuSnapshot(available=bool(sub), source=host.source, gpus=sub)


class InstanceRuntime:
    def __init__(self, instance, *, provider=None):
        self.instance = instance
        self._provider = provider or VllmProvider(
            base_url=instance.url, metrics_path=instance.metrics_path, api_key=instance.api_key)
        self._engine = MetricsEngine()
        self.history = History()
        self.snapshot: Snapshot | None = None
        self.model_names: list[str] = []
        self._dims_loaded = False

    async def _ensure_dims(self):
        if self._dims_loaded:
            return
        self._dims_loaded = True
        info = await self._provider.fetch_model_info()
        md = load_model_dims(info.root, info.max_model_len)
        self._engine = MetricsEngine(dims=md.dims, max_model_len=md.max_model_len)
        self.model_names = info.model_names

    async def poll(self, now):
        await self._ensure_dims()
        raw = await self._provider.fetch_metrics()
        if raw.fetched_ok and raw.text:
            snap = self._engine.derive(parse_metrics(raw.text), now=now)
        else:
            snap = self.snapshot or Snapshot(ts=now, connected=False, error=raw.error)
            snap.connected = False
            snap.error = raw.error
        self.snapshot = snap
        self._push_history(snap)
        return snap

    def _push_history(self, s):
        self.history.push("running", s.running)
        self.history.push("waiting", s.waiting)
        self.history.push("gen_tps", s.gen_tps)
        self.history.push("prompt_tps", s.prompt_tps)
        if s.prefix_hit_window is not None:
            self.history.push("prefix_hit", s.prefix_hit_window)

    def reset_session(self):
        self._engine.reset_session()

    async def aclose(self):
        await self._provider.aclose()


class Fleet:
    def __init__(self, instances, *, runtimes=None):
        self.runtimes = runtimes if runtimes is not None else [InstanceRuntime(i) for i in instances]

    async def poll(self, host_gpu, now):
        results = await asyncio.gather(*(rt.poll(now) for rt in self.runtimes), return_exceptions=True)
        items = []
        for rt, res in zip(self.runtimes, results):
            if isinstance(res, BaseException):
                res = rt.snapshot or Snapshot(ts=now, connected=False, error=str(res))
                res.connected = False
            if rt.instance.locality == "local":
                res.gpu = slice_gpu(host_gpu, rt.instance.gpus)
            else:
                res.gpu = GpuSnapshot(available=False, source="remote")
            items.append((rt.instance, res))
        return FleetSnapshot(ts=now, items=items, gpu=host_gpu)

    async def aclose(self):
        await asyncio.gather(*(rt.aclose() for rt in self.runtimes), return_exceptions=True)


def build_fleet(instances) -> Fleet:
    return Fleet(instances)
```

- [ ] **Step 4 — mock fleet** in `providers/mock.py` (inspect the existing `MockProvider`/
  `mock_gpu_snapshot` first; ensure the mock GPU snapshot can yield ≥3 GPUs — extend its signature
  with a `count`/`n` param if needed, keeping existing callers working):

```python
from vllmstat.core.fleet import Fleet, InstanceRuntime
from vllmstat.core.state import Instance
from vllmstat.providers.vllm import ModelInfo, RawText

class MockVllmProvider:
    def __init__(self, mock: MockProvider):
        self._mock = mock
    async def fetch_metrics(self):
        return RawText(text=self._mock.metrics_text(), fetched_ok=True)
    async def fetch_model_info(self):
        return ModelInfo(model_names=["mock-model"], max_model_len=None, root=None)
    async def aclose(self):
        pass

def mock_fleet(names=("qwen-30b", "llama-70b", "mixtral")) -> Fleet:
    rts = []
    for i, name in enumerate(names):
        inst = Instance(name=name, url=f"http://localhost:{8000 + i}", gpus=(i,), locality="local")
        rts.append(InstanceRuntime(inst, provider=MockVllmProvider(MockProvider())))
    return Fleet([], runtimes=rts)
```

  Add a test in `tests/test_fleet.py` (or `test_provider_mock.py`): `mock_fleet()` has 3 runtimes and
  `asyncio.run(mock_fleet().poll(mock_gpu_snapshot(1, ...), 1.0))` returns 3 connected items.

- [ ] **Step 5 — run, expect pass.** `pytest tests/test_fleet.py -q`
- [ ] **Step 6 — commit:** `feat(fleet): concurrent InstanceRuntime/Fleet polling + GPU slicing + mock fleet`

---

## Task 6: Overview render

**Files:** Modify `src/vllmstat/render.py`; Test `tests/test_render.py`.

- [ ] **Step 1 — failing tests** (append to `tests/test_render.py`):

```python
def _fleet_fixture():
    from vllmstat.core.state import FleetSnapshot, GpuSnapshot, GpuSample, Instance, Snapshot, Quantiles
    up = Snapshot(ts=1.0, connected=True, running=12, waiting=3, gen_tps=1400, kv_usage=0.63,
                  ttft=Quantiles(p50=0.142),
                  gpu=GpuSnapshot(available=True, source="x",
                                  gpus=[GpuSample(index=0, name="Arc", vendor="intel", util_gpu=100.0)]))
    down = Snapshot(ts=1.0, connected=False)
    return FleetSnapshot(ts=1.0, items=[
        (Instance("qwen-30b", "http://localhost:8000", gpus=(0,), locality="local"), up),
        (Instance("remote-a", "http://gpu-box:8000", locality="remote"), down)])

def test_fleet_overview_rows_and_cursor():
    from vllmstat import render
    out = render.fleet_overview(_fleet_fixture(), 0, width=80, uptime="0h03m")
    lines = out.splitlines()
    assert "fleet" in lines[0]
    assert "▸" in [l for l in lines if "qwen-30b" in l][0]      # selected cursor
    remote_line = [l for l in lines if "remote-a" in l][0]
    assert "✗" in remote_line and "(remote)" in remote_line     # down + remote gpu cell
    assert "intel" in [l for l in lines if "qwen-30b" in l][0]   # local gpu cell

def test_fleet_overview_none_safe_selection_out_of_range():
    from vllmstat import render
    render.fleet_overview(_fleet_fixture(), 99, width=80)        # must not raise
```

- [ ] **Step 2 — run, expect fail.**
- [ ] **Step 3 — implement** in `render.py` (add `Instance, FleetSnapshot` to the `state` import):

```python
def _gpu_cell(s: Snapshot, inst: Instance) -> str:
    if inst.locality == "remote":
        return "(remote)"
    if not s.gpu.available or not s.gpu.gpus:
        return "—"
    idxs = ",".join(str(g.index) for g in s.gpu.gpus)
    g0 = s.gpu.gpus[0]
    util = f"{g0.util_gpu:.0f}%" if g0.util_gpu is not None else "—"
    return f"G{idxs} {g0.vendor} {util}".strip()


def fleet_overview(fleet, selected, *, width=None, uptime="", interval=1.0, show_gpu=True) -> str:
    n = len(fleet.items)
    head = f"vllmstat  fleet · {n} instance{'' if n == 1 else 's'}  up {uptime}  {interval:.1f}s"
    cols = f" {'NAME':<14} {'ST':<2} {'RUN/WAIT':>9} {'GEN t/s':>8} {'KV%':>5} {'p50':>7}"
    if show_gpu:
        cols += "  GPU"
    lines = [head, cols]
    for i, (inst, s) in enumerate(fleet.items):
        cur = "▸" if i == selected else " "
        st = "●" if s.connected else "✗"
        rw = f"{s.running:.0f}/{s.waiting:.0f}" if s.connected else "—"
        gen = fmt_si(s.gen_tps) if s.connected else "—"
        kv = fmt_pct(s.kv_usage) if s.connected else "—"
        p50 = fmt_dur(s.ttft.p50) if (s.connected and s.ttft.p50 is not None) else "—"
        row = f"{cur}{inst.name:<14.14} {st:<2} {rw:>9} {gen:>8} {kv:>5} {p50:>7}"
        if show_gpu:
            row += f"  {_gpu_cell(s, inst)}"
        lines.append(row)
    return "\n".join(lines)


def detail_header(inst: Instance, s: Snapshot, *, interval: float, uptime: str) -> str:
    state = "● connected" if s.connected else "✗ down"
    return f"‹ fleet / {inst.name} @ {inst.url}   esc back    {state}  up {uptime}  {interval:.1f}s"
```

- [ ] **Step 4 — run, expect pass.** `pytest tests/test_render.py -q`
- [ ] **Step 5 — commit:** `feat(fleet): overview table + detail breadcrumb renderers`

---

## Task 7: App — overview/detail modes + navigation

**Files:** Modify `src/vllmstat/app.py`; Test `tests/test_app.py`.

Refactor `VllmStatApp` to own a `Fleet` + host `GpuProvider`, poll all instances each tick, and swap
between the overview Panel and the existing detail panels. **Single-instance mode mounts detail
directly and behaves exactly as before.**

- [ ] **Step 1 — failing tests** (extend `tests/test_app.py`; use Textual's `run_test()` harness as
  the existing tests do). Add:

```python
import pytest
from vllmstat.app import VllmStatApp
from vllmstat.config import Config
from vllmstat.core.state import Instance

def _cfg(insts, **kw):
    c = Config(mock=True, **kw)
    c.instances = insts
    return c

@pytest.mark.asyncio
async def test_single_instance_mounts_detail():
    app = VllmStatApp(_cfg([Instance("a", "http://localhost:8000")]))
    async with app.run_test() as pilot:
        await pilot.pause()
        assert app.is_fleet is False
        assert app.query_one("#detail").display is True
        assert app.p_overview.display is False

@pytest.mark.asyncio
async def test_fleet_mounts_overview_and_drills_in():
    insts = [Instance("a", "http://localhost:8000"), Instance("b", "http://localhost:8001")]
    app = VllmStatApp(_cfg(insts))
    async with app.run_test() as pilot:
        await pilot.pause()
        assert app.is_fleet is True
        assert app.p_overview.display is True
        await pilot.press("down")
        assert app.selected == 1
        await pilot.press("enter")
        assert app.in_detail is True
        assert app.query_one("#detail").display is True
        await pilot.press("escape")
        assert app.in_detail is False
        assert app.p_overview.display is True
```

(If the existing tests don't use `pytest.mark.asyncio`, follow their established harness pattern
instead — match the file's existing style for running a Textual app in tests. In `--mock` fleet, the
app must build its fleet via `mock_fleet(...)`; ensure `cfg.instances` of length ≥2 triggers fleet
mode even under mock.)

- [ ] **Step 2 — run, expect fail.**
- [ ] **Step 3 — implement.** Rework `app.py` per spec §8. Key structure (preserve existing CSS for
  the detail panels; add `#overview`):

  - `__init__`: `self.is_fleet = len(cfg.instances) > 1`; `self.in_detail = not self.is_fleet`;
    `self.selected = 0`; build the fleet — `mock_fleet(names...)` when `cfg.mock` (length matching
    `cfg.instances` if given, else default 3) else `build_fleet(cfg.instances)`; keep `self._gpu =
    GpuProvider(enabled=cfg.gpu)`, `self._mock`, timer/tick scaffolding from today.
  - `compose`: yield `self.p_overview = Panel(id="overview")`, then the existing detail panels wrapped
    in `with Vertical(id="detail"):`, then `Footer()`.
  - `on_mount`: `self._apply_mode()`; start the interval timer; `call_later(self.tick)`.
  - `_apply_mode`: `self.p_overview.display = self.is_fleet and not self.in_detail`;
    `self.query_one("#detail").display = self.in_detail`.
  - `_tick_body`: `now=time.monotonic()`; `host_gpu = mock_gpu_snapshot(self._tick_n, n=…)` when mock
    else `self._gpu.sample()` if enabled else `GpuSnapshot()`; `fs = await self.fleet.poll(host_gpu,
    now)`; store `self.fleet_snapshot = fs`; `self._refresh()`.
  - `_refresh`: if `self.is_fleet and not self.in_detail` → update `p_overview` via
    `render.fleet_overview(fs, self.selected, width=self._panel_width(self.p_overview),
    uptime=self._uptime(), interval=self.cfg.interval, show_gpu=self._gpu.enabled)`; else
    `self._refresh_detail(*self._current())`.
  - `_current()`: returns `(inst, snap, history)` for `self.selected` from `fleet_snapshot.items` and
    `fleet.runtimes[selected].history` (guard `fleet_snapshot is None`).
  - `_refresh_detail(inst, snap, hist)`: same body as today's `_refresh` but reads `snap`/`hist`; the
    header uses `render.detail_header(inst, snap, …)` in fleet mode, else `render.header(snap,
    url=inst.url, …)`.
  - Actions: `cursor_up`/`cursor_down` (clamp, only in overview) adjust `self.selected` + refresh;
    `drill_in` (overview only) sets `in_detail=True` + `_apply_mode` + refresh; `back` (fleet detail
    only) sets `in_detail=False` + `_apply_mode` + refresh; `reset_session` resets
    `self.fleet.runtimes[self.selected]`; `toggle_gpu` toggles `self._gpu.enabled`; keep
    `toggle_pause`, `faster`, `slower`, `_reschedule`.
  - BINDINGS: add `("up,k","cursor_up","Up")`, `("down,j","cursor_down","Down")`,
    `("enter","drill_in","Open")`, `("escape","back","Back")` to the existing set.

- [ ] **Step 4 — run, expect pass.** `pytest tests/test_app.py -q` then full `pytest -q`.
- [ ] **Step 5 — commit:** `feat(fleet): overview/detail navigation in the TUI`

---

## Task 8: `--once --json` fleet output

**Files:** Modify `src/vllmstat/cli.py`; possibly `src/vllmstat/snapshot_json.py`; Test
`tests/test_cli_once.py`.

Single instance → unchanged single object. Fleet (≥2 instances) → JSON **array**, each element
`{"name", "url", "locality", "snapshot": {...}}`.

- [ ] **Step 1 — failing test** (append to `tests/test_cli_once.py`, following its existing style for
  invoking `run_once_json`/`main` and capturing stdout): assert that with two `--url` flags +
  `--mock`, the printed JSON is a list of length 2 whose elements carry `name` and `snapshot.running`.
- [ ] **Step 2 — run, expect fail.**
- [ ] **Step 3 — implement:** in `cli.py`, after resolving instances, branch: if `len(instances) <= 1`
  keep today's single-object path; else build a `Fleet`, poll twice (now=0.0 then now=interval to get
  rates, mirroring the existing two-sample approach), and print
  `json.dumps([{ "name": inst.name, "url": inst.url, "locality": inst.locality,
  "snapshot": snapshot_to_dict(snap)} for inst, snap in fs.items], default=str)`. Reuse `mock_fleet`
  under `--mock`.
- [ ] **Step 4 — wire resolution:** add a small `resolve_config_instances(cfg, env)` helper (in
  `cli.py` or `core/resolve.py`) that runs `find_config`/`load_config` → `instance_from_dict`,
  optional `discover_docker()`, then `resolve_fleet(...)`, and sets `cfg.instances`. Call it in
  `main()` before dispatching to `run_app`/`run_once_json`. Guard all file/docker IO so a missing
  config or docker never crashes (fall back to flags/default).
- [ ] **Step 5 — run, expect pass.** `pytest tests/test_cli_once.py -q` then full `pytest -q`.
- [ ] **Step 6 — commit:** `feat(fleet): --once --json emits a per-instance array in fleet mode`

---

## Task 9 (controller): docs, version, screenshot, release

Handled by the controller after the final review (not a TDD subagent task):

- [ ] `pyproject.toml` + `src/vllmstat/__init__.py` → `0.3.0`.
- [ ] README: new **Fleet / multi-instance** section — config example, `--url` (repeatable),
  `--config`, `--discover-docker`, GPU mapping (`gpus=[...]`), local-vs-remote behaviour, overview +
  drill-in key bindings; update the key-binding and flags tables.
- [ ] New fleet-overview screenshot (`vllmstat --mock` with ≥2 instances → SVG → `docs/screenshot.png`
  via librsvg), or add a second image for the overview.
- [ ] Full `ruff check . && ruff format --check . && pyright && pytest -q` green; final code review;
  merge `feat/v0.3-fleet` → `main`; build + `twine check` + upload `vllmstat 0.3.0`.

---

## Self-review notes (controller)

- **Spec coverage:** instances (T3) ✓, config file (T3) ✓, Docker discovery (T4) ✓, concurrent poll
  + isolation (T5) ✓, GPU slicing (T5) ✓, overview + drill-in (T6/T7) ✓, CLI flags (T2) ✓, json
  fleet (T8) ✓, backward-compat single mode (T2 property, T7 mounts detail) ✓, docs/version (T9) ✓.
- **Type consistency:** `Instance.gpus` is `tuple[int,...]` everywhere (state, resolve, discovery,
  slice_gpu); `Fleet.poll(host_gpu, now) -> FleetSnapshot`; `fleet.items: list[(Instance, Snapshot)]`;
  `render.fleet_overview(fleet, selected, ...)`. All call sites match.
- **No new hard dep except** `tomli` (py<3.11 only; stdlib `tomllib` on ≥3.11).
