torann · ESS · neighbour-selection ablation
An exact index, corrupted on purpose. 3 seeds per arm, paired by seed, 4 shapes. Recall turns out to have a floor, not a price — and the floor is low.
Verdict
Half the true neighbours can go almost free; all of them cannot go at all. From an empty start, dropping to recall 0.5 while holding locality fixed costs 0.45% Clark-Evans at d=32 and 3.78% at d=8. Dropping to recall 0 costs 17.2% at d=32 and 44.0% at d=8 — collapsing to the uniform null or below it — even when the substituted neighbours are only 5–12% further away.
The filled start — the case ESS is actually for — is uniformly less sensitive. With anchors that cannot be pushed, recall 0.5 costs 0.29% at d=32 and 2.51% at d=8; recall 0 costs 10.4% and 28.3%. Every arm hurts less than it does from an empty start, so the slack above is a lower bound on the slack in production.
So this lands on the branch where a distance ranking is the product. The index must return real nearest neighbours, but it has genuine slack in how many of them are true — which is roughly where the tuner already sits. Redesigning around cost per plausible local neighbour is ruled out: that arm is indistinguishable from random.
Two measurements pointed opposite ways. Recall 0.69 at d=32 costs only 1.28% CE against perfect recall, which suggests recall is cheap to give up. But cutting LSH tables to lower recall costs 4.3% CE, which suggests it is not. Both can be true, because fewer tables degrades recall and locality together — the experiment confounds them.
This ablation separates the two. The index is exact throughout, so recall is not a property of the index here; it is imposed. Between the query and the force kernel the neighbour list is replaced by a controlled corruption of it, and the true toroidal-L1 distance of whatever was substituted is passed through. Only the selection changes. Force magnitudes are never confounded with it.
mean ratio below is the delivered neighbour distance over
the true k-th distance — the locality knob, made comparable across
dimensions. recall is measured against the exact answer the
same index computed.
Ordered by how local the selection is. Only the first two arms contain true neighbours at all.
True k nearest neighbours. The control.
k drawn from the true 2k nearest — half the true neighbours survive, and locality is essentially untouched.
k drawn from ranks k..4k. Not one true neighbour, but the nearest points that are not true neighbours.
k drawn from ranks 8k..16k. Still local relative to n, several times the true neighbour distance.
k drawn from everything within 2x the k-th distance — the arm as originally specified.
k uniformly random points. The null.
1.0 is the uniform null at every dimension. Higher is more regular — the thing ESS exists to produce.
The minimum pairwise distance, toroidal L1. This is where the mechanism shows: when the true nearest neighbours are withheld, nothing repels the pairs that are actually touching, and they fuse.
rank1-4k is the informative arm: its neighbours are only
12% further away than the
true ones at d=8, and 5%
at d=32. Being nearly-nearest buys nothing. The pairs at risk of fusing
are exactly the ones the force law must see, and plausible mid-range
repulsion does not compensate for missing them — separation falls
97% at d=8 and
40% at d=32.
The severity is strongly dimension-dependent, and this is the part worth not over-reading. At d=8 a near-miss neighbour list is worse than random — CE 0.8253 against a null of 1.0042 — because it actively holds points together while pushing on the wrong pairs, where random neighbours at least apply an isotropic mean-field pressure. At d=32 that gap nearly vanishes (0.9993 against 1.0008): once recall is zero, every arm lands on the null and how local the substitutes were stops mattering.
The arm as originally specified confirms why it needed replacing. At
d=32, ratio2x and uniform agree to four
decimal places on every column — the ball of radius 2x the k-th distance
has swallowed the entire point set, exactly the concentration argument
that governs the rest of this index's design. The rank-window arms were
added for that reason: they pin recall at zero while varying locality
independently, which is the separation the original design intended but
could not achieve at these dimensions.
| arm | recall | mean ratio | Clark-Evans | vs exact | separation | epochs |
|---|---|---|---|---|---|---|
| exact | 1.000 | 1.00 | 1.2074 ±0.0015 | — | 6.0752 | 151 |
| top2k | 0.500 | 1.00 | 1.2019 ±0.0021 | -0.45% | 5.9403 | 126 |
| rank1-4k | 0.000 | 1.05 | 0.9993 ±0.0014 | -17.24% | 3.6295 | 110 |
| rank8-16k | 0.000 | 1.16 | 0.9830 ±0.0039 | -18.59% | 3.6741 | 117 |
| ratio2x | 0.001 | 1.45 | 1.0008 ±0.0003 | -17.11% | 3.9632 | 40 |
| uniform | 0.001 | 1.45 | 1.0008 ±0.0003 | -17.11% | 3.9695 | 40 |
| arm | recall | mean ratio | Clark-Evans | vs exact | separation | epochs |
|---|---|---|---|---|---|---|
| exact | 1.000 | 1.00 | 1.1427 ±0.0021 | — | 4.1058 | 118 |
| top2k | 0.500 | 1.00 | 1.1394 ±0.0024 | -0.29% | 4.1058 | 112 |
| rank1-4k | 0.000 | 1.04 | 1.0236 ±0.0008 | -10.42% | 3.4952 | 118 |
| rank8-16k | 0.000 | 1.14 | 1.0075 ±0.0020 | -11.83% | 3.6029 | 114 |
| ratio2x | 0.001 | 1.45 | 1.0227 ±0.0003 | -10.50% | 3.6974 | 27 |
| uniform | 0.001 | 1.45 | 1.0227 ±0.0003 | -10.50% | 3.6974 | 27 |
| arm | recall | mean ratio | Clark-Evans | vs exact | separation | epochs |
|---|---|---|---|---|---|---|
| exact | 1.000 | 0.99 | 1.4741 ±0.0063 | — | 0.9058 | 166 |
| top2k | 0.500 | 0.99 | 1.4183 ±0.0176 | -3.78% | 0.7703 | 131 |
| rank1-4k | 0.000 | 1.12 | 0.8253 ±0.0210 | -44.01% | 0.0251 | 171 |
| rank8-16k | 0.000 | 1.46 | 0.9440 ±0.0049 | -35.96% | 0.1833 | 151 |
| ratio2x | 0.009 | 1.71 | 1.0083 ±0.0029 | -31.60% | 0.2981 | 38 |
| uniform | 0.002 | 2.28 | 1.0042 ±0.0019 | -31.88% | 0.2959 | 27 |
| arm | recall | mean ratio | Clark-Evans | vs exact | separation | epochs |
|---|---|---|---|---|---|---|
| exact | 1.000 | 0.98 | 1.3375 ±0.0056 | — | 0.3291 | 110 |
| top2k | 0.500 | 0.99 | 1.3040 ±0.0041 | -2.51% | 0.3291 | 106 |
| rank1-4k | 0.000 | 1.12 | 0.9591 ±0.0206 | -28.29% | 0.0526 | 128 |
| rank8-16k | 0.000 | 1.43 | 1.0098 ±0.0052 | -24.50% | 0.2033 | 125 |
| ratio2x | 0.008 | 1.72 | 1.0459 ±0.0083 | -21.81% | 0.2371 | 53 |
| uniform | 0.002 | 2.39 | 1.0379 ±0.0054 | -22.40% | 0.2395 | 51 |