fieldtrial report

Study {{ s.name }} · design {{ s.design_hash[:12] }} · {{ s.design_type | replace("_", " ") }} · {{ s.n_conditions }} conditions × {{ s.replicates }} replicate{{ "s" if s.replicates != 1 }} · seed {{ s.seed }} · generated {{ r.provenance.generated_at.strftime("%Y-%m-%d %H:%M") }} UTC

Summary

{{ sentence }}
–

Deviations from the plan

{% if r.deviations %}

{{ d.kind | replace("_", " ") }}: {{ d.message }}

{% else %}

None.

{% endif %}

Success rate per arm

{% if charts.rates %}

{{ charts.rates }}{% endif %}

{% for a in r.arms %} {% endfor %}

Arm	Label	Code	Successes	Completed	Rate	95% CI	Invalid
{{ a.arm }}	{{ a.label or "" }}	`{{ a.blind_code }}`	{{ a.successes }}	{{ a.completed }}	{{ a.rate \| rate }}	{{ a.ci \| rate_ci }}	{{ a.invalid }}

Primary analysis

{% set p = r.primary %}

Method	Used	Estimate	95% CI	p	α	Alternative	H0
{{ method_label }}	{{ p.n_used }}	{% if p.estimate is none %}– {% elif p.estimate_kind == "difference" %}{{ p.estimate \| pp }} {% elif p.estimate_kind == "odds_ratio" %}odds ratio {{ "%.2f" \| format(p.estimate) }} {% else %}{{ p.estimate \| rate }}{% endif %}	{% if p.ci is none %}– {% elif p.estimate_kind == "difference" %}{{ p.ci \| diff_ci }} {% elif p.estimate_kind == "odds_ratio" %}{{ "%.2f" \| format(p.ci.low) }} to {{ "%.2f" \| format(p.ci.high) }} {% else %}{{ p.ci \| rate_ci }}{% endif %}	{{ p.pvalue \| p }}	{{ p.alpha }}	{{ p.alternative }}	{{ "rejected" if p.rejected else "not rejected" }}

{% if p.pairwise | length > 1 %}

{% for w in p.pairwise %} {% endfor %}

Arm	vs	Pairs	b	c	Difference	95% CI	p	Adjusted p
{{ w.treatment }}	{{ w.control }}	{{ w.n_pairs }}	{{ w.b }}	{{ w.c }}	{{ w.difference \| pp }}	{{ w.ci \| diff_ci }}	{{ w.pvalue \| p }}	{{ w.adjusted_pvalue \| p }}

{% elif p.pairwise %} {% set w = p.pairwise[0] %}

Discordant blocks: {{ w.b }} where only {{ w.treatment }} succeeded, {{ w.c }} where only {{ w.control }} did, out of {{ w.n_pairs }}.

{% endif %} {% if charts.forest %}

Black: the paired primary analysis. Grey: arms treated as independent samples (sensitivity analysis).

{% endif %}

{% if r.sensitivity %}

Sensitivity analysis: arms as independent samples

{% for c in r.sensitivity %} {% endfor %}

Comparison	Counts	Difference	Newcombe 95% CI	Boschloo p	Fisher p
{{ c.treatment }} vs {{ c.control }}	{{ c.k1 }}/{{ c.n1 }} vs {{ c.k2 }}/{{ c.n2 }}	{{ c.difference \| pp }}	{{ c.ci \| diff_ci }}	{{ c.boschloo_p \| p }}	{{ c.fisher_p \| p }}

{% endif %} {% set staged = r.stages | selectattr("n") | list %} {% if staged %}

Progress stages

Furthest stage reached. Success means reaching {{ s.success_stage }}.

{% if charts.stages %}

{{ charts.stages }}{% endif %} {% if charts.funnel %}

Share of all trials that reached each stage. The dotted line marks the success stage.

{% endif %}

{% for st in staged %}{% endfor %} {% for name in s.stages %} {% set i = loop.index0 %} {% for st in staged %}{% set step = st.funnel[i] %}{% endfor %} {% endfor %}

Stage	{{ st.arm }}: reached / entered
{{ name }}	{{ step.reached }}/{{ step.entered }} ({{ step.conversion \| rate }})

{% if r.stage_comparisons %}

{% for c in r.stage_comparisons %}{% endfor %}

Brunner–Munzel on stage	p	Pre-registered
{{ c.treatment }} vs {{ c.control }}	{{ c.pvalue \| p }}	{{ "yes" if c.preregistered else "no" }}

{% endif %}

Time to success

{% if charts.timing %}

Share of all trials that had succeeded by each time. Every trial is observed until it ends, so there is no censoring.

{% endif %}

{% for t in r.timing %} {% endfor %}

Arm	Successes	Median (s)	95% CI (bootstrap)
{{ t.arm }}	{{ t.n_successes }}	{{ "%.1f" \| format(t.median_s) if t.median_s is not none else "–" }}	{{ ("%.1f–%.1f" \| format(t.ci.low, t.ci.high)) if t.ci else "–" }}

Outcomes per condition

{% if charts.conditions %}

Success rate per condition and arm; grey cells have no completed trial.

{% endif %}

Successes / completed trials per condition and arm

{% for arm in s.arms %}{% endfor %}{% for row in r.conditions %}{% for c in row.cells %}{% endfor %}{% endfor %}

Condition	{{ arm }}
{{ row.condition }}	{{ c.successes }}/{{ c.completed }}

Sessions and validity checks

{% if charts.sessions %}

{{ charts.sessions }}{% endif %}

{% for arm in s.arms %}{% endfor %}{% for row in r.sessions %}{% for c in row.cells %}{% endfor %}{% endfor %}

Started (UTC)	Operator	Rig	{{ arm }}
{{ row.started_at.strftime("%Y-%m-%d %H:%M") }}	{{ row.operator }}	{{ row.rig }}	{{ c.successes }}/{{ c.completed }}

{% if r.drift %}

{% for d in r.drift %}{% endfor %}

Check	Arm	Groups	Method	p	Flag
across {{ d.grouping }}s	{{ d.arm }}	{{ d.groups }}	{{ d.method }}	{{ d.pvalue \| p }}	{{ "flagged" if d.flagged else "" }}

{% else %}

Drift checks need at least two sessions.

{% endif %} {% set inv = r.invalid %}

Invalid trials (invalid/attempts): {% for a, n in inv.per_arm.items() %}{{ a }} {{ n }}/{{ inv.attempts[a] }}{{ ", " if not loop.last }}{% endfor %}{% if inv.method %}; {{ inv.method }} p = {{ inv.pvalue | p }}{% endif %}{% if inv.flagged %} (flagged){% endif %}.

Provenance

{% set pv = r.provenance %}

Design hash {{ pv.design_hash }} (seed {{ pv.seed }})
Locked with fieldtrial {{ pv.locked_with_version }}; analyzed with {{ pv.fieldtrial_version }} (Python {{ pv.python }}, numpy {{ pv.numpy }}, scipy {{ pv.scipy }})
Arm fingerprints: {% for a, h in pv.policy_fingerprints.items() %}{{ a }} {{ h }}{{ ", " if not loop.last }}{% endfor %}
{{ pv.sessions }} sessions; operators: {{ pv.operators | join(", ") or "–" }}; rigs: {{ pv.rigs | join(", ") or "–" }}
Results schema version {{ r.schema_version }}