DemoScope
Demonstration data diagnostics

Robot demonstration datasets fail silently. Two operators solve the same task two different ways, a regression policy learns the average of both, and the training loss falls the entire time. This reads the data and tells you which policy class will break on it — before you train anything.

Inject faults

{% for key, name, desc in faults %} {% endfor %}

Leave every switch off to scan a clean dataset. A quiet report on clean data is the control — check that first.

{% if not report %}
No scan yet.
Pick your faults on the left and run one.
Takes about a second — nothing is trained.
{% else %} {% set c = report.counts %}
{% if c.critical or c.warning %} {{ c.critical }} critical · {{ c.warning }} warning {% else %} No warnings {% endif %} {{ report.n_episodes }} episodes · {{ "{:,}".format(report.n_frames) }} frames · nothing trained
{% if c.critical or c.warning %} {% if c.critical %}{% endif %} {% if c.warning %}{% endif %} {% if c.info %}{% endif %} {% else %}{% endif %}

Findings

{% if not report.findings %}

Nothing trips a detector. Train with confidence.

{% endif %} {% for f in report.findings %}
{{ f.id }}{{ f.severity }}
{{ f.where }}

{{ f.what }}

{{ f.so_what }}

{% endfor %}

Diagnostic figure

Four diagnostic panels: approach strategies, task-condition coverage, conditional action entropy by phase, and episode feature space.
Top left is the one that matters. Thin lines are individual demonstrations, coloured by the strategy the tool recovered. The thick red line is their average — which is what a regression policy converges to.

Measurements

{% for a in report.averaging %} {% endfor %}
PhaseStrategiesAveraging hazardH(a|s)
{{ a.phase }} {{ a.n_modes }} 1 and a.hazard_ratio > 1.0 %}class="bad"{% endif %}> {{ "%.2f"|format(a.hazard_ratio) }} SD {{ "%.2f"|format(report.entropy.per_phase.get(a.phase, 0)) }}

Condition-space gap {{ "%.0f%%"|format(report.coverage.gap_ratio * 100) }} · intrinsic dimensionality {{ "%.2f"|format(report.coverage.intrinsic_dim) }} · {{ report.shards.n_shards }} feature shard(s), CKA {{ "%.2f"|format(report.shards.cka_between) }}

{% if validation %}

Did the warning come true?

Two policy classes trained on this exact configuration and rolled out 40 times each. Measured ahead of time — training takes minutes, so it does not run in the browser.

{% for name, succ, coll in validation.rows %} {% endfor %}
PolicySuccessCollision
{{ name }} {{ succ }} {{ coll }}
{% endif %} {% endif %}