{{ c.question }}
- level
- {{ level(c.level) }}
- influence
- {{ c.influence.value }}
- consequence
- {{ c.consequence.value }}
- model risk
- {{ c.risk.value }}
- decided on
- {{ c.decisive_role.value }} evidence
- decisive points
- {{ c.n_decisive }}
- evaluable points
- {{ c.n_evaluable }}{% if c.minimum_points %} (minimum {{ c.minimum_points }}){% if c.n_evaluable < c.minimum_points %} below the minimum{% endif %}{% endif %} {% if c.rationale %}
- rationale
- {{ c.rationale }} {% endif %}
Quantities of interest
A quantity is covered by the evaluable decisive points of its observable inside the claimed domain, a ratio by its evaluable pairs whose numerator lies inside the claimed domain.
| quantity | description | kind | coverage | points |
|---|---|---|---|---|
| {{ q.id }} | {{ q.description }} | {% if q.ratio and q.coverage %}ratio{% elif q.ratio %}ratio, no pairing declared{% else %}absolute{% endif %} | {% if q.coverage %}{{ q.coverage }}{% else %}n/a{% endif %} | {% if q.coverage %}{{ q.n_points }}{% else %}n/a{% endif %} |
Claimed domain
{% if c.domain_statement %}{{ c.domain_statement }} statement, not assessed
{% endif %} {% if view.coverage %}| field | value | coverage | points |
|---|---|---|---|
| {{ e.field }} | {{ e.value }} | {% if e.coverage is none %}outside the claim{% else %}{{ e.coverage.value }}{% endif %} | {{ e.n_points }} |
The domain claims no values of the metadata.
{% endif %}Evidence
{% if view.evidence %}| evidence | role | selection | points | mappings | studies | outliers dropped | limitation subtracted |
|---|---|---|---|---|---|---|---|
| {{ e.name }}{% if e.description %} {{ e.description }}{% endif %} |
{{ e.role.value }} | {{ e.selection }} | {{ e.n_points }} | {{ e.n_mappings }} | {{ e.n_studies }} | {{ e.outliers_dropped }} | {{ e.limitation_subtracted }} |
Metrics of the evidence
The fold errors (GMFE, AFE) and the percentage of the points within a fold or a number of standard deviations of the data, n/a without an evaluable point.
| evidence | {% for name in view.metrics %}{{ name }} | {% endfor %}
|---|---|
| {{ e.name }} | {% for name in view.metrics %}{{ e.metrics[name] | number if name in e.metrics else "" }} | {% endfor %}
No evidence.
{% endif %}Acceptance
{% if c.acceptance %}| rule | value | limit | result |
|---|---|---|---|
| {{ a.name }} | {{ a.value | number }} | {{ a.limit }} | {% if a.met is none %}not evaluable{% elif a.met %}met{% else %}not met{% endif %} |
| {{ a.name }} ratio pairs | {{ a.value | number }} | {{ a.limit }} | {% if a.met is none %}not evaluable{% elif a.met %}met{% else %}not met{% endif %} |
No acceptance rules.
{% endif %} {% if c.ratio_acceptance %}Ratio pairs
The ratio of the summary of an arm over the one of its reference arm in the same study, of the data and of the predictions; a pair is within when the predicted ratio lies within the limit of Guest et al. (2011) of the observed ratio, 1.25 at a ratio of 1 and towards 2 for a large ratio.
{% if c.ratios %}| quantity | fit | study | arms | observed | predicted | limit | result |
|---|---|---|---|---|---|---|---|
| {{ r.quantity }} | {{ r.fit }} | {{ r.experiment }} | {{ r.numerator }} / {{ r.denominator }} | {{ r.observed | number }} | {{ r.predicted | number }} | {{ r.limit | number }} | {% if r.within is none %}not evaluable{% elif r.within %}within{% else %}outside{% endif %} |
No study has both arms of a ratio in the decisive points.
{% endif %} {% endif %} {% if c.temporal is not none %}{% set t = c.temporal %}Temporal validation
The decisive points of the studies the model collected after the day its fit ran, data which did not exist for the model when it was fitted; the day of the collection is the record of the study in the data of the model. The points are part of the acceptance above, the verdict does not change.
{% if t.studies %}| rule | value | limit | result |
|---|---|---|---|
| {{ a.name }} new studies | {{ a.value | number }} | {{ a.limit }} | {% if a.met is none %}not evaluable{% elif a.met %}met{% else %}not met{% endif %} |
{{ t.n_evaluable }} evaluable points of the new studies, the minimum of the verdict is {{ c.minimum_points }}; {{ t.n_known }} decisive {{ "study was" if t.n_known == 1 else "studies were" }} collected before {{ "its" if t.n_known == 1 else "their" }} fit.
| study | fit | fit ran | collected | released | points | evaluable |
|---|---|---|---|---|---|---|
| {{ s.study }} | {{ s.fit }} | {{ s.fit_date }} | {{ s.collected }} | {% if s.released %}{{ s.released }} {{ s.pkdb_id }}{% else %}not released{% endif %} | {{ s.n_points }} | {{ s.n_evaluable }} |
No decisive study was collected after its fit: {{ t.n_known }} collected before{% if t.undated %}, {{ t.undated | length }} without a date{% endif %}.
{% endif %} {% if t.undated %}Without a day of collection (collected before the records of the studies existed, or without a record): {{ t.undated | join(", ") }}.
{% endif %} {% endif %} {% if c.uncertainty is not none %}{% set u = c.uncertainty %} {% set states = {"accepted": "met", "rejected": "not_met", "equivalent": "not_evaluable", "undecided": "not_evaluable", "not_evaluable": "outside"} %}Parameter uncertainty
The adopted fits evaluated at {{ u.n_draws }} draws of their fitted parameters (the profile or Fisher uncertainty of their records{% if u.fits | selectattr("correlated") | list %}, with the correlation of the Fisher information{% endif %}). A rule is robust when it is met with a probability of at least {{ u.threshold }} over the draws, decided at the level {{ u.level }} for the rules of the context together; the verdict is the acceptance at the adopted parameters and does not change.
{% if u.rules %}| rule | limit | P(met) | sequence | draws | decision |
|---|---|---|---|---|---|
| {{ r.name }} | {{ r.limit }} | {{ r.estimate | number }} | {% if r.low is not none %}{{ r.low | number }} to {{ r.high | number }}{% endif %} | {{ r.n }}{% if r.n_failed %} +{{ r.n_failed }} failed{% endif %} | {{ r.decision | replace("_", " ") }}{% if r.n_needed %} about {{ r.n_needed }} draws{% endif %} |
The {{ ((u.quantiles[1] - u.quantiles[0]) * 100) | round | int }} % bands of the predictions hold {{ "n/a" if u.coverage is none else ((u.coverage * 100) | round | int) ~ " %" }} of the {{ u.n_points }} evaluable decisive observations; a band of the parameter uncertainty holds the prediction of the mean, the variability between the subjects and the error of the measurements lie outside it.
{% if u.quantities %}| quantity | points | band fold | coverage % |
|---|---|---|---|
| {{ q.quantity }} | {{ q.n_points }} | {{ q.band_fold | number }} | {{ "n/a" if q.coverage is none else (q.coverage * 100) | round | int }} |
The parameters which carry the variance of a quantity (the mean log ratio of its predictions to the adopted ones) and of the value of a rule over the draws: the first-order indices, estimated from the draws without further simulations (Plischke 2013), up to three of at least 0.05 per target; the index of a parameter includes the effect of the parameters it is correlated with.
| target | leading parameters |
|---|---|
| {{ target }}{% if group[0].kind == "rule" %} rule{% endif %} | {% for i in leading[:3] %}{{ i.fit }}.{{ i.parameter }} {{ i.index | number }}{% if not loop.last %}, {% endif %}{% else %}none of at least 0.05{% endfor %} |
{{ f.fit }}: {{ f.n_parameters }} {{ "parameter" if f.n_parameters == 1 else "parameters" }}, {{ f.note }}{% if f.n_failed %}, {{ f.n_failed }} failed draws{% endif %}.
{% endfor %} {% endif %} {% if c.inputs is not none and c.inputs.n_checked %}{% set inp = c.inputs %}Inputs
The simulated covariates of the subjects and the doses of {{ inp.n_checked }} of the {{ inp.n_mappings }} decisive mappings against the groups, individuals and interventions their studies report. A covariate is equivalent within 10 % of the reported mean (the value of an individual) and in the range within the reported minimum and maximum (else two standard deviations around the mean), a dose is equivalent within 2 % of a reported dose; the simulated range is what the evidence covers.
| input | mappings | simulated | equivalent | in range | differ | not reported |
|---|---|---|---|---|---|---|
| {{ s.target }}{% if s.kind == "dose" %} dose{% endif %} | {{ s.n_mappings }} | {% if s.low == s.high %}{{ s.low | number }}{% else %}{{ s.low | number }} to {{ s.high | number }}{% endif %} {{ s.unit }} | {{ s.n_equivalent }} | {{ s.n_in_range }} | {% if s.n_differs %}{{ s.n_differs }}{% else %}0{% endif %} | {{ s.n_not_reported }} |
The inputs which differ from the reported ones, the ratio is the simulated over the reported value{% if inp.differs | selectattr("source", "equalto", "model") | list %}; model marks the value of the model, which the simulation does not set{% endif %}:
| mapping | input | simulated | reported | reported by | ratio |
|---|---|---|---|---|---|
| {{ d.experiment }}/{{ d.mapping }} | {{ d.target }}{% if d.source == "model" %} model{% endif %} | {{ d.simulated | number }} {{ d.unit }} | {{ d.reported | number }} {{ d.unit }}{% if d.low is not none %}, {{ d.low | number }} to {{ d.high | number }}{% endif %} | {{ d.reported_by | join(", ") }} | {{ d.ratio | number }} |
Variability
The visual predictive check of the decisive points with the variability of the subjects the model declares ({% for t, d in v.targets.items() %}{{ t }} {{ d }}{% if not loop.last %}, {% endif %}{% endfor %}){% if v.between %} and between the studies ({% for t, d in v.between.items() %}{{ t }} {{ d }}{% if not loop.last %}, {% endif %}{% endfor %}){% endif %}: {% if v.between %}{{ v.studies }} studies of {% endif %}{{ v.subjects }} subjects {% if v.vectors > 1 %}nested in {{ v.vectors }} draws of the parameters{% else %}at the adopted parameters{% endif %}, the subjects of every study group varying around its inputs. A mean of a group of N subjects is compared with the means of {{ v.groups }} virtual groups of N subjects{% if v.between %} of a study{% endif %}, an individual with the subjects; the verdict does not change.
{% if v.coverage is not none %}| points | means of groups | coverage % | interval % | expected % | agreement |
|---|---|---|---|---|---|
| {{ v.n_points }} | {{ v.n_groups }} | {{ (v.coverage * 100) | round | int }} | {{ (v.low * 100) | round | int }} to {{ (v.high * 100) | round | int }} | {{ (v.expected * 100) | round | int }} | {{ v.agreement }}{% if v.failed %} {{ v.failed }} failed subjects{% endif %} |
The interval is the Wilson interval of the coverage at the level {{ v.level }}. Narrow: the bands hold fewer observations than expected, the declared variability with the uncertainty of the parameters does not explain the scatter of the data, or the typical predictions are biased; wide: they hold more, the variability is larger than the one of the data.
{% if v.quantities %}| quantity | points | coverage % |
|---|---|---|
| {{ q.quantity }} | {{ q.n_points }} | {{ "n/a" if q.coverage is none else (q.coverage * 100) | round | int }} |
No evaluable decisive point has a band.
{% endif %} {% endif %}Verification
{% if c.verification %}| check | factor | points | largest difference | at | acceptance | result |
|---|---|---|---|---|---|---|
| {{ check_labels[r.check] }}{% for problem in r.problems %} {{ problem }}{% endfor %} |
{{ r.factor }} | {{ r.n_points }} | {{ r.max_difference | number if r.acceptance_changed is not none else "" }} | {{ r.at }} | {% if r.acceptance_changed is none %}{% elif r.acceptance_changed %}changed{% else %}unchanged{% endif %} | {{ "passed" if r.passed else "failed" }} |
| battery of the builds, {{ battery | length }} checks{% for r in failed %} {{ check_labels[r.check] }} failed{% endfor %} |
{{ battery | map(attribute="factor") | unique | join(", ") }} | {{ "failed" if failed else "passed" }} |
The assessment was not verified.
{% endif %}Factors of V&V 40
| status | factor |
|---|---|
| {{ f.status | replace("_", " ") }}{% if f.step is not none %} in step {{ f.step }}{% endif %} |
{{ f.name }} {{ f.statement }} |
Extrapolation
{% if c.extrapolation %}{{ c.extrapolation }}
{% else %}No extrapolation is argued.
{% endif %}