anomaly_eval op• Data kinds: signal × signal → table
• Call: import fullseye as fs; fs.ledger.anomaly_operating_point(scores, labels, recall=0.95, confidence=0.95) (to call the implementation directly, import anomeval; anomeval.anomaly_operating_point(scores, labels, recall=0.95, confidence=0.95); from the registry, opsspc.get("anomaly_operating_point"))
Pick the highest threshold whose defect recall is guaranteed to be at least recall at confidence (table).
scores (1-D, higher = more anomalous) and labels (1 / True = defective) are image scores of a held-out set that was NOT used to fit the detector. Detecting k of the n defective images, the exact one-sided Clopper–Pearson lower limit of the recall at level confidence is computed; the smallest k whose limit reaches recall fixes the threshold at the k-th highest defect score (>= rule: the highest threshold that still carries the guarantee, so the fewest false alarms). The guarantee holds for defects drawn like the held-out ones; it is a statement about a sample, not about every future defect. Returns threshold, recall_observed, recall_lower (the limit), detected, n_pos, n_neg, fpr (fraction of good images at or above the threshold), fpr_upper (its one-sided Clopper–Pearson upper limit at the same level) and n_required (the smallest number of defective images for which the guarantee is attainable at all:
ceil(ln(1 - confidence) / ln(recall))).
Raises ValueError when the guarantee is unattainable with so few defective images (n_pos < n_required): it never returns a threshold it cannot back. Also on empty / mismatched / non-finite input, non-binary labels, no good or no defective image, or recall / confidence outside (0, 1).
• Sample-data catalog (download URLs / licences) — 2-D uses skimage.data (BSD/public domain) plus synthetic images; 3-D lists download URLs for real data sources (Stanford, PDS, …).
• Operator provenance and references — the sources of the research/methods this op family came from.
• The canonical algorithm (author, year) and its uses are named in the family usage guide above.
• poc_patch_anomaly — py -3.11 examples/poc_patch_anomaly.py
table as input)msa_anova_table · msa_gauge_rr · msa_bias_linearity · msa_attribute_agreement · gum_standard_uncertainty · gum_propagate · gum_expanded · gum_monte_carlo
anomaly_eval)anomaly_image_auroc · anomaly_pixel_auroc · anomaly_pro · anomaly_region_overlap · anomaly_pixel_f1_max · anomaly_image_f1_max
*Provenance: anomeval.py — SPC operator registry. This per-op note is generated by tools/opdocs.py md (do not hand-edit).*
© 2026 Kazufumi Furuse — Fullseye operator documentation. Licensed under Apache-2.0.