{% include "_head.html" %} {{ run_id }} · AI Code Quality Auditor {% with subtitle = run_id, nav_meta = conditions|length ~ ' conditions · ' ~ metrics|length ~ ' metrics' %} {% include "_topbar.html" %} {% endwith %}

Experiment report

{{ run_id }}

{% if banner_text %} {% endif %} {#- A warning recorded in the provenance is a disclosure about what the data is. It is rendered whatever else is or is not present, because the one situation where it must not be dropped is the one where the rest of the page is thin. -#} {% if provenance.warning %} {% endif %}
Conditions
{{ conditions|length }}
one human baseline, {{ conditions|length - 1 }} agents
Metrics
{{ metrics|length }}
one of them needs the specification
Runs recorded
{{ reps_per_condition.values()|sum }}
{% for c, n in reps_per_condition.items() %}{{ n }}{% if not loop.last %} · {% endif %}{% endfor %}
{% if leaderboard %}
Best overall
{{ leaderboard[0].condition }}
won {{ leaderboard[0].wins }} of {{ metrics|length }} metrics
{% endif %}

{% if spec_summary %}Spec set: {{ spec_summary }}.{% endif %} {% if provenance.note %} {{ provenance.note }}{% endif %}

{% if summary %}

Best and worst on each metric

At a glance

{% for s in summary %}
{{ s.metric }}
{{ '%.2f'|format(s.best_value) }} {{ s.unit }}
Best {{ s.best_condition }}
Worst {{ s.worst_condition }} ({{ '%.2f'|format(s.worst_value) }})
{% endfor %}
{% endif %} {% if leaderboard %}

Across every metric

Ranking

Each condition is ranked on every metric, and the ranks are summed. A low average rank means a condition was rarely the worst, which is a different claim from being the best at any one thing.

{% for row in leaderboard %}
{{ row.overall_rank }}
{{ row.condition }}
avg {{ row.avg_rank }} · {{ row.wins }} win{{ '' if row.wins == 1 else 's' }}
{% endfor %}
{% endif %}

All five metrics at once

Shape of each condition

Every metric normalised so that further from the centre is better.

Calibrated scores are framed around practical decision thresholds; raw values preserve the underlying measurement scale. Neither replaces reading the figures in the tables below.

Condition against metric

Heatmap

Every figure of record

Metric by metric

{% for m in metrics %}

{{ m }}

unit: {{ units.get(m, '—') }}
{% if blurbs.get(m) %}

{{ blurbs[m] }}

{% endif %} {% if metric_guidance.get(m, {}).get('axis') %}

{{ metric_guidance[m]['axis'] }}

{% endif %} {% if metric_guidance.get(m, {}).get('adoption') %}

Adoption implication · {{ metric_guidance[m]['adoption'] }}

{% endif %}
{% for c in conditions %} {% endfor %}
Condition Value Rank Runs
{{ c }} {% if pivot[m].get(c) is not none %}{{ '%.2f'|format(pivot[m][c]) }}{% else %}—{% endif %} {% if ranks[m].get(c) %} {% if ranks[m][c] == 1 %}1st{% else %}{{ ranks[m][c] }}{% endif %} {% else %}—{% endif %} {{ reps_per_condition.get(c, '—') }}
{% endfor %}

This page renders an immutable CSV and adds nothing to it. Where a published figure was later corrected, the superseded figure is kept in the repository beside the corrected one, under docs/ERRATUM_*.md.