evaluate op• Data kinds: signal × signal × signal × signal → table
• Call: import fullseye as fs; fs.ledger.holdout_threshold(train_pos, train_neg, test_pos, test_neg, target_fpr: 'float' = 0.05) -> 'dict' (to call the implementation directly, import emproof; emproof.holdout_threshold(train_pos, train_neg, test_pos, test_neg, target_fpr: 'float' = 0.05) -> 'dict'; from the registry, opsemproof.get("holdout_threshold"))
> This operator's description has not been translated yet. The original text follows as it is.
閾値を訓練側で選び、評価は別の側で測る(score が高いほど「疑わしい」)。
`tau = 訓練の負例(仕込んでいない成分・対)の偽陽性率が target_fpr` 以下になる最小の閾値
(負例スコアの上位 `target_fpr 分位)。返すのは tau` と、訓練・評価それぞれの AUC・TPR・FPR、件数。
評価側の数字だけを主張に使う ―― 閾値を選んだ側で測った TPR は必ず楽観する(evolve の holdout 規律を
op に切り出したもの)。AUC は Mann–Whitney(同点 0.5)。
>>> r = holdout_threshold(tr_pos, tr_neg, te_pos, te_neg, target_fpr=0.05)
>>> r["tau"], r["test_auc"], r["test_tpr"], r["test_fpr"]
• Sample-data catalog (download URLs / licences) — 2-D uses skimage.data (BSD/public domain) plus synthetic images; 3-D lists download URLs for real data sources (Stanford, PDS, …).
• Operator provenance and references — the sources of the research/methods this op family came from.
• The canonical algorithm (author, year) and its uses are named in the family usage guide above.
• poc_em_second_opinion — py -3.11 examples/poc_em_second_opinion.py
table as input)—
evaluate)—
*Provenance: emproof.py — EMPROOF operator registry. This per-op note is generated by tools/opdocs.py md (do not hand-edit).*
© 2026 Kazufumi Furuse — Fullseye operator documentation. Licensed under Apache-2.0.