Which setup earns the slot

One request, one user

Nobody waiting behind you. The score collapses to tok/s × size, so it asks one question: how much model can the rig push at interactive speed. Both rigs answer with a mixture-of-experts model — few active params to compute, many total params to count. srv2's answer is North-Mini-Code-1.0, added to the evidence base on 2026-08-27: 30B total on 3B active, whole model on the card at IQ2_M, 90.3 tok/s. It beats the incumbent GPT-OSS-20B by 36%.

Many requests, many users

Each setup is taken at the concurrency that maximises its own score, over the measured ladder {1 … 384}. This is where the two rigs stop resembling each other: srv1's best fleet row is a 35B MoE crawling at 4.0 tok/s per stream, srv2's is a 1.5B on vLLM serving 384 streams at once.

Why n is counted twice

The formula is right. The number it is fed is the wrong one.

Written as tok/s × size × n with tok/s meaning one stream's rate, the key is exactly correct — it is model-mass delivered per second across the fleet. The record stores the aggregate instead, which already contains n. Feeding the aggregate in makes the score quadratic in width, so it does not rank setups: it ranks how many slots each one will accept.

It matters. Under the quadratic score the argmax always drifts to the widest level that has not yet collapsed; under the corrected one it lands where aggregate throughput actually peaks. Below is the fleet top ten re-ranked, each setup taken at its own new argmax n.

srv2's answer changes hands entirely. The 1.5B that wins on the quadratic score falls to fourth; North-Mini-Code-1.0 at n=16 takes it on 518.4 tok/s × 30B, and Qwen2.5-Coder-7B AWQ at n=256 — the setup the 08-28 record picked before the driver capped it at n=128 — comes second. A 30B MoE that never accepted more than 16 streams was invisible under the quadratic score and is the best fleet setup on the rig under the corrected one. On srv1 the mixture-of-experts models climb three to seven places each: they were being punished for not accepting 128 streams, which was never the question.

What a draft model actually buys

Nine measured target/draft pairs, each against its own no-draft baseline from the same run and the same build. These are not on the 475-token protocol — reply lengths were 60, 150 and 256 tokens — so the ratio transfers to the ladder above, the absolute rates do not.

The split is by engine, not by rig. llama.cpp gains: +11% on the srv2 7B at n=1, still +6% at n=4. vLLM loses under load: every batched pairing measured is negative, down to 0.59× on srv1 — the draft steals the compute the batch was using. At n=1 on srv2 vLLM it is a wash at 1.02×.

How long before it starts talking

TTFT was never recorded under the 475-token protocol — neither the llama.cpp nor the vLLM driver logs it, and it cannot be recovered from p50 and wall alone. It exists in exactly one place: 334 per-request ollama records in the 08-24 engine sweep, which report prefill, decode and total separately. TTFT here is derived as total − decode, so it includes queue wait — which is the honest number, because that is what the caller feels.

At n=1 srv2 answers in 20.8 ms against srv1's 42.1 ms — the same 2× that shows up everywhere else between these two rigs. The shape changes with width: through n=32 prefill dominates and TTFT tracks the model, but at n=128 on srv2 queue wait is 523 ms of the 809 ms. Past that point TTFT stops measuring the engine and starts measuring the backlog, which is precisely the cost the fleet tables charge nothing for.

Six things the tables do not say

What was dropped, and why

2,308 measured rows came out of the August directories. 1,301 were eligible to rank. The gap is not noise — it is rows that answer a different question.

Empty cells, with cause

A refusal is a recorded result. 195 of them are in the dataset with the engine's own sentence; these are the ones that shape the tables above.