Experiment Report

{% for report in reports_by_score %}

Score Target: {{ report.score_target }}

Context Result Statistics

{% for model in report.score_described %} {% endfor %}
Context Name Mean Std Median Min Max Mode
{{ model.context_name }} {{ '%.4f' | format(model.mean) }} {{ '%.4f' | format(model.std) }} {{ '%.4f' | format(model.median) }} {{ '%.4f' | format(model.minimum) }} {{ '%.4f' | format(model.maximum) }} {{ '%.4f' | format(model.mode) }}

A/B Test Pipeline

{% if "check_normality_with_shapiro" in report.ab_tests.pipeline_track %}

Shapiro-Wilk

{% for test in report.ab_tests.shapirowilk %} {% if test.stat is none and test.p_value is none%} {% else %} {% endif %} {% endfor %}
Context Stat p-value Normal?
{{ test.context }} NaN NaN {{ 'Yes' if test.is_normal else 'No' }}{{ test.context }} {{ '%.4f' | format(test.stat) }} {{ '%.4e' | format(test.p_value) }} {{ 'Yes' if test.is_normal else 'No' }}
{% endif %} {% if "check_homocedasticity_with_levene" in report.ab_tests.pipeline_track %}

Levene

{% if report.ab_tests.levene.stat is none and report.ab_tests.levene.p_value is none%}

Stat: NaN | p-value: NaN | Homoscedastic: {{ 'Yes' if report.ab_tests.levene.is_homoscedastic else 'No' }}

{% else %}

Stat: {{ '%.2f' | format(report.ab_tests.levene.stat) }} | p-value: {{ '%.4e' | format(report.ab_tests.levene.p_value) }} | Homoscedastic: {{ 'Yes' if report.ab_tests.levene.is_homoscedastic else 'No' }}

{% endif %} {% endif %} {% if "perform_bartlett" in report.ab_tests.pipeline_track %}

Bartlett

{% if report.ab_tests.bartlett.stat is none and report.ab_tests.bartlett.p_value is none%}

Stat: NaN | p-value: NaN | Homoscedastic: {{ 'Yes' if report.ab_tests.bartlett.is_homoscedastic else 'No' }}

{% else %}

Stat: {{ '%.2f' | format(report.ab_tests.bartlett.stat) }} | p-value: {{ '%.4e' | format(report.ab_tests.bartlett.p_value) }} | Homoscedastic: {{ 'Yes' if report.ab_tests.bartlett.is_homoscedastic else 'No' }}

{% endif %} {% endif %} {% if "perform_kurskalwallis" in report.ab_tests.pipeline_track %}

Kruskal-Wallis

{% if report.ab_tests.kurskalwallis.stat is none or report.ab_tests.kurskalwallis.p_value is none%}

Stat: NaN | p-value: NaN | Significant: {{ 'Yes' if report.ab_tests.kurskalwallis.is_significant else 'No' }}

{% else %}

Stat: {{ '%.2f' | format(report.ab_tests.kurskalwallis.stat) }} | p-value: {{ '%.4e' | format(report.ab_tests.kurskalwallis.p_value) }} | Significant: {{ 'Yes' if report.ab_tests.kurskalwallis.is_significant else 'No' }}

{% endif %} {% endif %} {% if "perform_mannwhitney" in report.ab_tests.pipeline_track %}

Mann-Whitney

{% for test in report.ab_tests.mannwhitney %} {% endfor %}
Comparison Stat p-value Significant?
{{ test.context }} {{ '%.2f' | format(test.stat) }} {{ '%.4e' | format(test.p_value) }} {{ 'Yes' if test.is_significant else 'No' }}
{% endif %} {% endfor %}

Significance Conclusions

Best Context