Exact predictions
2/15
Meets the current exact-hit gate and matches the prior precision run.
Exact component targeting benchmark
Exact targeting cleared the coverage gate; policy drift is the remaining blocker.
Vaner prepared locally with qwen3.5:35b. Claude Sonnet generated
cold and warm answers; Claude Opus judged prediction relevance and answer quality.
The run improved related coverage from 5/15 to 8/15
and mean relevance from 0.31 to 0.36 versus the prior
precision run, while preserving 2/15 exact hits.
Related coverage
8/15
Up from 5/15; now above the 45% promotion threshold.
Mean relevance
0.36
Just over the 0.35 gate; prior precision run was 0.31.
Quality A/B
2 / 0 / 1
Vaner wins / ties / cold wins under Claude Opus judging.
Baseline vs Current
Baseline: v2_precision_local5090_sonnet_opus_15turn. Current: v2_exact_targeting_local5090_sonnet_opus_15turn.
Promotion Gate
| Criterion | Status | Result |
|---|---|---|
| Related rate >= 45% | Pass | 53% |
| Exact rate >= 10% | Pass | 13% |
| Mean score >= 0.35 | Pass | 0.36 |
| Weight drift <= 0.20 | Pass | 0.0432 |
| Source drift <= 0.60 | Pass | 0.2282 |
| Numeric policy drift <= 0.80 | Fail | Above threshold |
Local 35B background preparation per turn, down from 62.2s on the prior precision run.
Higher than the previous 342ms, but still outside answer generation and dominated by local preparation strategy.
No absolute local paths found in the new benchmark artifact or this canvas.
What Improved
1
Coverage crossed the gateRelated prebuilt context rose to 53%, meeting the target that the previous precision run missed.
2
Mean relevance crossed the gateExact targeting lifted average relevance enough to pass the 0.35 threshold under the same Sonnet/Opus setup.
3
Warm answer quality improvedClaude Opus preferred the Vaner-warm answer on proxy flow and IntentScorer questions.
Remaining Failures
A
Tangential primary evidenceSix misses had primary evidence, but it was incomplete or nearby rather than exact.
B
No primary evidenceFive misses still lacked source/config/test evidence for the named mechanism.
C
Policy driftThe adaptive scoring policy still changes too much numerically during the run, even though source and weight drift are controlled.