{% extends "base.html" %} {% block title %}Compare - Scenario Result Viewer{% endblock %} {% block breadcrumb %} {% endblock %} {% macro outcome_badge(r) %} {% if r is none %}not run {% elif r.outcome == "pass" %}PASS {% elif r.outcome == "fail" %}FAIL {% elif r.outcome == "error" %}ERROR {% else %}N/A{% endif %} {% endmacro %} {% macro pct(value) %}{% if value is none %}-{% else %}{{ "%.0f" | format(value * 100) }}%{% endif %}{% endmacro %} {% block content %} {% if oo_styles %}{% endif %}
Cases are matched by what they ran (their sweep overrides, else the ODD values they were drawn at), not by their position. The oldest session picked is the baseline.
{% if comparison %} {% set last = comparison.sessions | length - 1 %}The buckets each case was drawn in. Each cell shows the pass rate of every session (baseline first); red: lower in the latest, green: higher.
| {{ comparison.y or "" }} \ {{ comparison.x }} | {% for bx in comparison.x_labels %}{{ bx }} | {% endfor %}|
|---|---|---|
| {{ by }} | {% for bx in comparison.x_labels %} {% set cell = comparison.cells.get((bx, by)) %} {% if cell %} {% set d = cell.delta %}
{% for t in cell.tallies %}
{% if t[1] %}{{ t[0] }}/{{ t[1] }}{% else %}-{% endif %}
{% endfor %}
|
{% else %}
{% endif %} {% endfor %} |
No coverage files.
{% endif %}| Change | Case | {% for ref in comparison.sessions %}{{ ref.split('/', 1)[1] }} | {% endfor %} {% for key in comparison.metrics %}Δ {{ metric_names[key][0] }} | {% endfor %}|
|---|---|---|---|---|
| {% if change == "regression" %}pass → fail {% elif change == "fixed" %}fail → pass {% elif change == "flaky" %}flaky {% else %}{{ change }}{% endif %} |
{{ c.first.label }}
{% for name, bucket in c.first.buckets.items() %}{% if name in (comparison.x, comparison.y) %}{{ name.split('.')[-1] }} {{ bucket }}{% endif %}{% endfor %}
|
{% for r in c.runs %}
{% if r %}{{ outcome_badge(r) }}{% else %}{{ outcome_badge(none) }}{% endif %} | {% endfor %} {% for key in comparison.metrics %} {% set d = c.delta(key) %} {% set better = metric_names[key][1] %}{% if d is none %}-{% else %}{{ "%+.2f" | format(d) }}{% endif %} | {% endfor %}{% set ns = namespace(q=[]) %} {% for r in c.runs %}{% if r %}{% set ns.q = ns.q + ["run=" ~ (comparison.sessions[loop.index0] ~ "/" ~ r.index) | urlencode] %}{% endif %}{% endfor %} {% if ns.q | length > 1 %}side by side{% endif %} |