simcal_validate — SIMCAL check op

• Datenarten: calresult × simmodel → calresult

• Aufruf: import fullseye as fs; fs.ledger.simcal_validate(result=None, holdout=None, locked=None, tau=None, dims=None, n_boot=200, alpha=0.001, seed=0) -> 'dict' (die Implementierung direkt: import simcalcheck; simcalcheck.simcal_validate(result=None, holdout=None, locked=None, tau=None, dims=None, n_boot=200, alpha=0.001, seed=0) -> 'dict'; aus dem Register: opssimcal.get("simcal_validate"))

Verwendung

Überanpassungs-Prüfungen auf zurückgehaltenen Bildern; schlägt eine fehl, wird die Kalibrierung verworfen (Schätzwerte zurückgezogen).

> Die ausführliche Beschreibung unten ist der Originaltext — Zusammenfassung und Überschriften sind übersetzt.

holdout (and locked): observed simmodels of the same knobs (simcal_observe with split='holdout' / 'locked') or image lists for the fitted model's conditions. Errors are reduced chi^2 (1 = the residuals are exactly the noise). Gates, each recorded in validation.gates with its value and limit:

1. beats_initial — holdout error at the estimate < holdout error at the initial knobs;

2. holdout_over_train — holdout / train <= 1 + tau, tau = the bootstrap 95 % relative spread of the train error over its pixels (n_boot resamples) unless given;

3. beats_gain_offset — better than the initial simulation re-scaled by a fitted gain and offset only (the cheapest model that could explain the images);

4. estimate_plausible — every estimate inside its range, not within 1 % of an end, and within 3 sd of its prior when one is given;

5. knobs_within_dims — free knobs <= output directions above noise (dims, or computed from a Latin hypercube over the knobs with sensitivity.mv_residual_pca);

6. fit_consistent_with_noise — the absolute gate: on the train images (nu = pixels − free knobs) and on the holdout images (nu = pixels), chi^2 of the noise-normalised residuals has an upper-tail probability P(chi^2_nu >= chi^2) of at least alpha (default 1e-3). Gates 1-5 are relative comparisons and pass a fit that is 60 times the noise everywhere (a thin film in the wrong fringe order, measured); this one does not. Judged on p, not on chi^2/nu, because chi^2/nu concentrates on 1 for large nu (both are returned). The variance is the noise model's plus, for a stochastic model, its sample variance. A failure means: the fit is bad for this noise model — a local minimum, a model discrepancy, or a noise estimate that is too small. ★It does NOT detect a noise model that is too large: over-estimating the variance by 4x drives p to 1 and the gate passes (measured) — that direction is visible only as chi^2/nu << 1.

locked is scored once (errors.locked) and is not used by any gate; a result that already carries a LOCKED score is refused (the robust.py discipline: the untouched number is untouched only once).

Ausführlicher Anwendungsleitfaden

• Leitfaden zur Familie sim_to_real_calibration

Referenzen (Beispieldaten, Literatur)

• Katalog der Beispieldaten (Download-URLs / Lizenzen) — 2-D nutzt skimage.data (BSD/Public Domain) plus synthetische Bilder, 3-D nennt Download-URLs echter Datenquellen (Stanford, PDS, …).

• Herkunft und Literatur der Operatoren — die Quellen der Forschung/Verfahren, auf denen diese Operatorfamilie beruht.

• Der kanonische Algorithmus (Autor, Jahr) und seine Anwendungen stehen im Familienleitfaden oben.

Ausführbare Beispiele (verifizierte Samples, die diesen Operator wirklich aufrufen)

• poc_pocket_interreflection — py -3.11 examples/poc_pocket_interreflection.py

• poc_simcal_thin_film — py -3.11 examples/poc_simcal_thin_film.py

• simcal_checks_tour — py -3.11 examples/simcal_checks_tour.py

Typkompatible Folge-Operatoren (nehmen calresult als Eingabe)

simcal_identifiability · simcal_uncertainty · simcal_discrepancy · simcal_report

Gleiche Kategorie (check)

simcal_identifiability · simcal_uncertainty · simcal_discrepancy


*Provenance: simcalcheck.py — SIMCAL Operator-Registry. Diese Notiz wird von tools/opdocs.py md erzeugt (nicht von Hand bearbeiten).*

© 2026 Kazufumi Furuse — Fullseye operator documentation. Licensed under Apache-2.0.