turboquant-pro · code-quality audit
A full elegance-and-rigor pass over the package surfaced defects where the code silently returns wrong results, crashes on supported inputs, or reports a signal that contradicts the project's own acceptance-metric rule — and a follow-up sweep caught the same pattern in the docs. Findings are ranked by consequence — soundness first — and each was verified against source before landing in this report.
These produce a wrong answer with no error, or fail on inputs the API accepts. All nine are fixed, tested, and verified on Atlas across the CPU, torch, and GPU paths; the CUDA finding sat on the default GPU compress path and was invisible to CI (needs CuPy + a GPU).
Each thread cached its own output column of Pi_T but read it as if indexed by the contraction variable, so every one of the 32 output columns received the same bogus diagonal sum. The default GPU compress path produced codes unrelated to the input.
Tile the input row into shared memory and contract correctly against Pi_T[:, out_col]. GPU compression now matches the CPU path exactly.
A recall@k target was answered with mean_cosine (marked # approximate) — the exact blind-metric substitution the project's thesis warns against. A caller asking for recall got a cosine verdict.
Measure true recall@k (exact vs. reconstructed rankings); rank the Pareto frontier and unmet-target fallback on the target's own axis; raise on an unmeasured metric instead of substituting cosine.
TestRecallTarget suite (incl. a no-alias guard) green in CI.pack(ce) took seed as a free argument defaulting to 42, unrelated to the embedding it serialized — so it could stamp a seed that rebuilds a different rotation/codebook on decode, yielding a wrong vector with no error.
CompressedEmbedding now carries seed; pack reads ce.seed by default, unpack returns it on the object, and both reject bits ∉ {2, 3, 4}.
It assigned rather than appended, so index.add(a).add(b) kept only b — despite the accumulating name and the fluent return self.
Concatenate codes / norms across calls. Incremental indexing now reproduces the one-shot codes exactly.
NumPy has no bf16 dtype, so tensor.numpy() raised TypeError — breaking the boundary adapter for any bf16 model, the common serving dtype.
Fall back to a lossless float32 upcast on the unsupported-dtype path; happy paths are untouched.
The basis spans a layer's output space, but the code computed W @ Vᵀ, which requires in == out — so it never worked on the FFN matrices (gate / up / down projections) it exists to compress.
Project output rows onto the top-k activation subspace: Vkᵀ(Vk W). Mismatched bases are skipped, and the docstring/log no longer imply a size or latency win the dense writeback does not deliver.
Effective rank is bounded by min(shape), but the ratio divided by max(shape), understating it on every non-square weight and recommending more aggressive compression than the spectrum justifies.
Divide by min(shape), the true rank ceiling.
The affine check — an algebraic equality that gates fused-decode safety — used a flat 5e-3 absolute tolerance, loose enough to pass a wrong grid mapping. The packed check passed vacuously when a plugin silently ignored packed=True.
Affine tolerance is now scale-relative — max(1e-4, 1e-5·peak) — admitting float32 roundoff but not a real error. Packed must self-report as packed or serialize distinctly, or it no longer earns a clean pass.
The docstrings said "no distributional assumptions whatsoever," but the default robust measure_kappa (2.5 / 97.5 trim) is conditional on trimming ~5% and holds for the central 95%, not unconditionally.
Document the strict-vs-robust regimes honestly; defaults unchanged (the robust regime is the right operating point), only the claim corrected.
The numbers were correct, but a headline verdict rested on reconstruction cosine — the one signal the project explicitly rules out as an acceptance metric. Now closed: this was the last acceptance path gating on cosine.
is_healthy, the alert, and the tqp monitor exit code all reduced to mean_cosine ≥ quality_floor. The module computed the (A2) tangential fraction and radial-drift signals — its own docstring warns these catch ranking damage "cosine cannot see" — but never wired them into the verdict. A stream whose tangential fraction collapsed still reported is_healthy: true.
Health now also requires (A2) noncollapse: a self-calibrating guard trips on a significant downward drift of the tangential stream, and an opt-in tangential_floor (constructor + tqp monitor --tangential-floor) adds a hard level gate. Both no-op when the statistic is disabled or too sparse; alerts now carry their triggering reasons.
Both are the same root cause: reconstruction cosine standing in for an acceptance decision. It reads ~0.97 while retrieval ranking collapses (the v1.2.0 KV-keys incident: 0.095 recon error, perplexity ~10⁴). Cosine is a labeled diagnostic, never a gate.
T0·2 was a silent numeric substitution; T1·1 was the same instinct surviving in a health verdict. Both are now fixed — the "coherent acceptance metric across every component" claim holds end-to-end.
A follow-up sweep of every doc and example for the same pattern in prose: cosine framed as an accept/reject signal rather than a diagnostic. Four genuine cases — two of them stale, describing behavior the T0·2 and T1·1 fixes had already changed. The other cosine mentions across the docs are legitimate and were deliberately kept (see below).
The README one-liner, auto_compress's own module + function docstrings, and the tqp plan embeddings default all led with target="cosine > 0.95" — teaching cosine as the accept signal, and predating the real recall@k support.
All lead with target="recall@10 >= 0.90" (a measured signal); the CLI default is now recall. Text states plainly that the Pareto frontier and accept/reject rank on that axis; cosine/ratio remain accepted for reconstruction-only checks.
The plan.json note and CLI docs called the sweep "cosine/ratio-driven (a library limitation)" — no longer true after T0·2 taught auto_compress to measure recall@k and rank the frontier on it.
Reworded: the frontier ranks on the target metric (measured recall when it is a recall target); the plan's overall acceptance is the distribution-free rank-certificate preview, and cosine is a labeled diagnostic.
The docs said the tqp monitor exit code is "0 when mean cosine ≥ floor" — describing the cosine-only gate that T1·1 had just replaced.
Now: the gate is is_healthy = cosine floor and (A2) noncollapse, with --tangential-floor added to the documented signature.
"The 0.978 mean cosine similarity means search ranking is almost perfectly preserved" — the exact false equivalence the project's thesis refutes (cosine can read high while retrieval order shifts).
Corrected to call the number a reconstruction diagnostic, not a ranking guarantee, pointing to measured recall@k / a rank certificate for acceptance.
Most cosine mentions are legitimate and were left untouched: benchmark-table columns and per-bit fidelity descriptors ("5.1× at 0.978 cosine"), the min_cosine / mean_cosine metric fields and Prometheus gauges, certify --metric cosine (the ranking distance, gated on the τ floor) and probe --consumer cosine (a consumer choice), and — importantly — the correct warnings that cosine is not a recall proxy (README, the research notes, KV_KEYS_FINDING.md: "reconstruction cosine stays deceptively high"). CHANGELOG.md is historical record, left intact.
Ten parallel readers swept the package by cluster, each returning ranked findings with file:line, a concrete failure scenario, and a proposed fix. Every finding in this report was then re-read against current source by hand — nothing is reported on an agent's word alone.
CPU suite on the CI-mirroring env (59 pass), torch suite on Atlas testenv / torch 2.11 (137 pass), GPU compress on a GV100 via a CuPy venv (25 pass), then CI green on Python 3.10 / 3.11 / 3.12 (274 pass, 11 skipped). The GPU finding was reproducible only with CuPy + a device, which is why it outran CI.
A separate pass grepped every doc, example, and the CLI help/JSON the docs mirror for cosine used as an accept/reject signal, reading context around each hit to separate genuine acceptance framing from legitimate diagnostics. Fixes verified with test_cli (63 pass / 3 skipped on Atlas) and CI; the recall-default change was exercised end-to-end.
It consolidates the findings that survived verification at severity Tier 0–1 plus the docs-coherence sweep. Lower-severity polish (style, micro-robustness) was not elevated here, and the tree keeps moving (P3/P4 plugin work landed) — a fresh comprehensive sweep can enumerate residual items on request rather than trusting a stale list.