{% if error %}
{{ "{:,}".format(e.isl) }} in, {{ "{:,}".format(e.osl) }} out estimated from this run's request-size sweep
{% if not e.fits_context %}| First token, one user | {{ ms(e.ttft_ms) }} | without the network; add your round trip |
| Per token, one user | {{ ms(e.tpot_ms) }} | |
| Whole answer, one user | {{ ms(e.e2e_ms) }} | |
| Per token at {{ e.loaded_concurrency }} users | {{ ms(e.loaded_tpot_ms) }} | median, at the load the shapes were compared at |
| Requests the cache holds at once | {{ "{:,}".format(e.fits) }} | at full length |
The {{ e.extrapolated | join(" and ") }} length is outside what this run measured, so the estimate extends the measured straight line; treat it as rough. Long prompts usually cost more than the line says.
{% endif %}