{% extends "report_base.html" %} {# The report of the credibility assessment of a model, see `pkpdsim.credibility.report.render_html`. It extends the base of the reports of sbmlsim and fills its blocks. Sections: the overview, one card per context of use, the limitations, the unclaimed validation data and the provenance. A verdict, a coverage and the result of a rule are words with a colour class (`verdict`, `state`), never only a colour. Nothing loads from the network. #} {% macro verdict(v) %}{{ v.label }}{% endmacro %} {% macro limitation_context(e) %}{% if e.context %}{{ e.context }}{% else %}the model{% endif %}{% endmacro %} {% macro level(l) %}{{ l.value }} {{ l.label }}{% endmacro %} {% block title %}Credibility of {{ provenance.title }}{% endblock %} {% block items %}contexts of use{% endblock %} {% block heading %}Credibility{% endblock %} {% block subtitle %}{{ provenance.title }} {{ provenance.model }}{% endblock %} {% block placeholder %}Search contexts of use, evidence, studies …{% endblock %} {% block badges %}contexts of use {{ contexts | length }}{% for v in verdicts %}{{ v.label }} {{ verdict_counts[v] }}{% endfor %}{% endblock %} {% block head %} {% endblock %} {% block nav %} Overview Contexts of use Limitations Unclaimed data Provenance {% endblock %} {% block chips %} {% for v in verdicts %} {% endfor %} {% endblock %} {% block footer %}created by pkpdsim {{ provenance.versions.get("pkpdsim", "") }} on {{ provenance.date }}{% endblock %} {% block content %}

Overview

The contexts of use of the model, their model risk from the influence of the model on the decision and the consequence of a wrong decision, and their verdict on the decisive evidence (decided on): the calibration data, the data the model was fitted to, for a description (C1), else the validation data. The evaluable points are the decisive points which the fold criteria can evaluate.

{% for view in contexts %} {% set c = view.result %} {% endfor %}
context of useverdict decided on title levelrisk decisive points evaluable points
{{ c.id }} {{ verdict(c.verdict) }} {{ c.decisive_role.value }} {{ c.title }} {{ level(c.level) }} {{ c.risk.value }} {{ c.n_decisive }} {{ c.n_evaluable }}

Limitations of the model

{% if model_limitations %} {% else %}

The model declares no limitations.

{% endif %}
{% if assessment.notes %}

Notes

{{ assessment.notes }}

{% endif %}

Contexts of use

{% for view in contexts %} {% set c = view.result %}
{{ c.id }} {{ c.title }} {{ verdict(c.verdict) }}

{{ c.question }}

level
{{ level(c.level) }}
influence
{{ c.influence.value }}
consequence
{{ c.consequence.value }}
model risk
{{ c.risk.value }}
decided on
{{ c.decisive_role.value }} evidence
decisive points
{{ c.n_decisive }}
evaluable points
{{ c.n_evaluable }}
{% if c.rationale %}
rationale
{{ c.rationale }}
{% endif %}

Quantities of interest

{% for q in c.quantities %} {% endfor %}
quantitydescriptionkind
{{ q.id }}{{ q.description }}{% if q.ratio %}ratio, assessed in step 3{% else %}absolute{% endif %}

Claimed domain

{% if c.domain_statement %}

{{ c.domain_statement }} statement, not assessed

{% endif %} {% if view.coverage %}
{% for e in view.coverage %} {% endfor %}
fieldvalue coveragepoints
{{ e.field }}{{ e.value }} {% if e.coverage is none %}outside the claim{% else %}{{ e.coverage.value }}{% endif %} {{ e.n_points }}
{% elif not c.domain_statement %}

The domain claims no values of the metadata.

{% endif %}

Evidence

{% if view.evidence %}
{% for e in view.evidence %} {% endfor %}
evidencerole selection pointsmappings studies outliers dropped limitation subtracted
{{ e.name }}{% if e.description %}
{{ e.description }}{% endif %}		 {{ e.role.value }} {{ e.selection }} {{ e.n_points }}{{ e.n_mappings }} {{ e.n_studies }} {{ e.outliers_dropped }} {{ e.limitation_subtracted }}

Metrics of the evidence

The fold errors (GMFE, AFE) and the percentage of the points within a fold or a number of standard deviations of the data, n/a without an evaluable point.

{% for name in view.metrics %}{% endfor %} {% for e in view.evidence %} {% for name in view.metrics %}{% endfor %} {% endfor %}
evidence{{ name }}
{{ e.name }}{{ e.metrics[name] | number if name in e.metrics else "" }}
{% else %}

No evidence.

{% endif %}

Acceptance

{% if c.acceptance %}
{% for a in c.acceptance %} {% endfor %}
rulevaluelimitresult
{{ a.name }} {{ a.value | number }} {{ a.limit }} {% if a.met is none %}not evaluable{% elif a.met %}met{% else %}not met{% endif %}
{% else %}

No acceptance rules.

{% endif %}

Factors of V&V 40

{% for f in c.factors %} {% endfor %}
statusfactor
{{ f.status | replace("_", " ") }}{% if f.step is not none %}
in step {{ f.step }}{% endif %}		 {{ f.name }}
{{ f.statement }}

Extrapolation

{% if c.extrapolation %}

{{ c.extrapolation }}

{% else %}

No extrapolation is argued.

{% endif %}
{% endfor %}

Limitations

The data the model does not describe, of the model and of a context of use. Their metrics are reported for information and are outside every acceptance; their points are subtracted from the evidence.

{% if assessment.limitations %}
{% for e in assessment.limitations %} {% endfor %}
context of uselimitation selection pointsmappings studies
{{ limitation_context(e) }} {{ e.name }}{% if e.description %}
{{ e.description }}{% endif %}		 {{ e.selection }} {{ e.n_points }}{{ e.n_mappings }} {{ e.n_studies }}

Metrics of the limitations

{% for name in limitation_metrics %}{% endfor %} {% for e in assessment.limitations %} {% for name in limitation_metrics %}{% endfor %} {% endfor %}
context of uselimitation{{ name }}
{{ limitation_context(e) }} {{ e.name }}{{ e.metrics[name] | number if name in e.metrics else "" }}
{% else %}

No limitations are declared.

{% endif %}

Unclaimed validation data

The validation points of the adopted fits which no evidence and no limitation selects.

{% if assessment.unclaimed %} {% for u in assessment.unclaimed %} {% endfor %}
fitstudy mappingpoints
{{ u.fit }}{{ u.experiment }}{{ u.mapping }}{{ u.n }}
{% else %}

Every validation point is claimed.

{% endif %}

Provenance

model
{{ provenance.model }}
title
{{ provenance.title }}
adopted fits
{{ provenance.fits | join("\n") if provenance.fits else "none" }}
{% for package, version in provenance.versions.items() %}
{{ package }}
{{ version }}
{% endfor %}
commit
{% if provenance.commit %}{{ provenance.commit }}{% if provenance.dirty %} with uncommitted changes{% endif %}{% else %}outside a repository{% endif %}
date
{{ provenance.date }}
{% endblock %}