{% extends "base.html" %} {% block title %}Dashboard — FreeWeight{% endblock %} {% block content %}

Dashboard

Every figure on this page is one stored measurement from one named run — the latest completed run of that suite for that model. Nothing here is averaged across runs, and every number links to the samples that produced it.

{% if error %} {% endif %} {% if dashboard %}
Clear
{% if dashboard.is_empty %}

Nothing measured yet

No completed run matches this scope. Start one from the Run page, or on the command line:

freeweight run start --model <model> --suite native.echo

native.echo is the harness self-test: it proves the machine can run a benchmark at all, in under a minute, before you spend an hour on a real one.

{% else %}

Summary

Completed runs
{{ dashboard.cards.completed_runs }}

In this scope.

Models measured
{{ dashboard.cards.models_measured }}

Across {{ dashboard.cards.suites_run }} suites.

Samples stored
{{ dashboard.cards.samples_stored }}

Every figure below drills to one of these.

Unsupported measurements
{{ dashboard.cards.unsupported_metrics }}

Things this machine could not measure. Shown as — everywhere, never 0.

Machines
{{ dashboard.cards.machines }}

{% if dashboard.cards.machines > 1 %} More than one: speed, memory and energy figures below are not comparable across them. {% else %} One machine, so hardware figures are comparable. {% endif %}

Latest run
{{ dashboard.cards.latest_run_at | timestamp }}

Completed.

Comparison heatmap

One headline metric per suite, from the latest completed run. An empty cell means that model has never completed that suite — not that it scored zero.

{% if dashboard.heatmap.separated %}
Separated: these cells span more than one machine or more than one version of a suite. Read them down a column, not across a row; FreeWeight will not merge them into one figure.
{% endif %} {% if dashboard.heatmap.models %}
{% for suite in dashboard.heatmap.suites %} {% endfor %} {% for model in dashboard.heatmap.models %} {% for suite in dashboard.heatmap.suites %} {% set cell = dashboard.heatmap.cell(model, suite) %} {% endfor %} {% endfor %}
{{ dashboard.heatmap.models | length }} models × {{ dashboard.heatmap.suites | length }} suites.
Model {{ suite }}
{{ dashboard.heatmap.headline_metric[suite] }}
{{ model }} {% if cell is none %} {% elif cell.value is none %} {{ cell.unavailable_reason }} {% else %} {{ '%.4g' % cell.value }} {{ cell.unit }} {% endif %}
{% else %}

No suite in this scope has a declared headline metric yet. The panels below still show everything that was measured.

{% endif %}

Trade-offs

{% with points = dashboard.quality_vs_speed, chart_id = 'quality-vs-speed', title = 'Quality against decode speed', x_label = 'decode tokens/s', y_label = 'quality' %} {% include "partials/_scatter.html" %} {% endwith %} {% with points = dashboard.quality_vs_vram, chart_id = 'quality-vs-vram', title = 'Quality against peak VRAM', x_label = 'peak VRAM bytes', y_label = 'quality' %} {% include "partials/_scatter.html" %} {% endwith %} {% for panel in dashboard.panels %}

{{ panel.title }}

{{ panel.description }}

{% if panel.is_empty %}

Nothing measured for this panel in this scope yet.

{% else %}
{% for row in panel.rows %} {% endfor %}
{{ panel.rows | length }} figures, each from one completed run.
Model Suite Metric Value Unit n Excluded Raw source
{{ row.model_canonical_id }} {{ row.suite_key }} {{ row.metric_key }} {% if row.value is none %} {% else %}{{ '%.6g' % row.value }}{% endif %} {{ row.unit }} {{ row.sample_count }} {{ row.excluded_count }} {{ row.run_test_key or 'run' }} {{ row.run_id[:8] }}
{% endif %} {% endfor %} {% endif %} {% endif %} {% endblock %} {% block scripts %} {% endblock %}