{% if error %}
{{ error }}
{% else %}

{{ "{:,}".format(e.isl) }} in, {{ "{:,}".format(e.osl) }} out estimated from this run's request-size sweep

{% if not e.fits_context %}
That's longer than the server's context window ({{ "{:,}".format(e.max_model_len) }} tokens), so the server would reject it.
{% endif %} {% if e.loaded_tpot_ms is not none %} {% endif %} {% if e.fits %} {% endif %}
First token, one user{{ ms(e.ttft_ms) }}without the network; add your round trip
Per token, one user{{ ms(e.tpot_ms) }}
Whole answer, one user{{ ms(e.e2e_ms) }}
Per token at {{ e.loaded_concurrency }} users{{ ms(e.loaded_tpot_ms) }}median, at the load the shapes were compared at
Requests the cache holds at once{{ "{:,}".format(e.fits) }}at full length
{% if e.extrapolated %}

The {{ e.extrapolated | join(" and ") }} length is outside what this run measured, so the estimate extends the measured straight line; treat it as rough. Long prompts usually cost more than the line says.

{% endif %}
{% endif %}