Metadata-Version: 2.4
Name: tfidf-stability
Version: 0.2.0
Summary: Numerical stability and perturbation behaviour in TF-IDF-based similarity systems
Keywords: tf-idf,information-retrieval,cosine-similarity,numerical-stability,perturbation-analysis,ranking-stability,reproducibility
Author-Email: Matthew Maksymilian Miezaniec <matthewmiezaniec1@gmail.com>
License-Expression: Apache-2.0
License-File: LICENSE.md
License-File: LICENSES/Apache-2.0.txt
License-File: LICENSES/BSD-3-Clause.txt
License-File: LICENSES/CC-BY-4.0.txt
License-File: THIRD_PARTY_NOTICES.md
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: C++
Classifier: Topic :: Scientific/Engineering :: Mathematics
Classifier: Topic :: Text Processing :: Indexing
Project-URL: Repository, https://github.com/Canon-Regularis/tfidf-stability-and-sensitivity
Requires-Python: >=3.11
Requires-Dist: numpy>=1.24
Requires-Dist: pyyaml>=6.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0; extra == "dev"
Requires-Dist: pytest-cov>=5.0; extra == "dev"
Requires-Dist: hypothesis>=6.100; extra == "dev"
Requires-Dist: ruff==0.16.2; extra == "dev"
Requires-Dist: mypy==2.3.0; extra == "dev"
Requires-Dist: scikit-learn>=1.4; extra == "dev"
Requires-Dist: snowballstemmer>=3.0; extra == "dev"
Requires-Dist: types-PyYAML; extra == "dev"
Requires-Dist: pre-commit>=4.0; extra == "dev"
Requires-Dist: reuse[charset-normalizer]>=6.0; extra == "dev"
Provides-Extra: viz
Requires-Dist: matplotlib>=3.8; extra == "viz"
Requires-Dist: pandas>=2.0; extra == "viz"
Provides-Extra: docs
Requires-Dist: mkdocs-material>=9.5; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.25; extra == "docs"
Provides-Extra: nb
Requires-Dist: jupyter>=1.0; extra == "nb"
Requires-Dist: nbmake>=1.5; extra == "nb"
Requires-Dist: nbformat>=5.10; extra == "nb"
Provides-Extra: all
Requires-Dist: tfidf-stability[dev,docs,nb,viz]; extra == "all"
Description-Content-Type: text/markdown

# Numerical Stability and Perturbation Behaviour in TF-IDF-Based Similarity Systems

## Abstract

This repository holds a self-contained implementation and analysis of a
TF-IDF-based document similarity and ranking system, written to make the
algebraic structure and the perturbation behaviour of the pipeline explicit.

Documents are embedded by explicit tokenisation, n-gram construction, smoothed
inverse document frequency (IDF) and sparse TF-IDF vectors. Similarity is
cosine similarity; rankings are induced by a deterministic sorting procedure.
Intermediate quantities stay accessible: document frequencies, IDF weights,
vector norms, similarity scores. Corpus structure and similarity geometry can
therefore be related directly to ranking behaviour under perturbation.

Alongside the numerical sensitivity of TF-IDF embeddings and cosine similarity,
the repository treats ranking stability as a first-class object. Stability is
characterised empirically through score-separation margins: boundary margins
govern top-k membership, adjacent margins govern ordering within the top-k.
These margins yield explicit sufficient conditions under which a ranking is
invariant under bounded perturbations of the similarity scores.

Deterministic tie-breaking rules are isolated as a separate source of
decision-level discontinuity. Ranking outcomes may change while cosine
similarity scores remain equal to within numerical tolerance. Separating the
numerical stability of the scores from the stability of the induced ranking
exposes structural fragility in similarity-based retrieval and content-based
recommendation systems.

---

## 1. Introduction

This project examines the numerical stability of TF-IDF-based similarity and
ranking systems under small perturbations of the data and of the preprocessing.

TF-IDF (term frequency-inverse document frequency) is a foundational technique in
information retrieval and text-based modelling. It represents documents as
vectors in a high-dimensional feature space, where similarity is usually measured
by cosine similarity. Despite its ubiquity, the behaviour of TF-IDF pipelines
under perturbation (changes in corpus composition, token distributions or
preprocessing rules) is rarely analysed in a systematic and explicit manner.

The setting here is controlled: documents arise from text of interest, and
similarity scores induce content-based k-nearest-neighbour (k-NN) rankings. The
system is implemented twice. A normative pure-Python reference defines
correctness and requires no compiler; an optimised C++20 core is required to be
bit-identical to it, with agreement enforced by comparing raw bit patterns rather
than tolerances. Both give explicit control over preprocessing, vocabulary
construction, IDF computation, vector formation and ranking, so corpus structure
and similarity geometry can be related directly to similarity scores,
neighbourhood structure and ranking behaviour under perturbation.

The central question is one of stability:  
**how sensitive are TF-IDF weights, cosine similarities, and induced rankings to small
changes in the underlying data?**
Two aspects of it are addressed:
(i) how score-separation margins govern the stability of similarity-based rankings
under bounded perturbations, and
(ii) how deterministic tie-breaking rules introduce discontinuities in ranking
outcomes that are independent of numerical error in the similarity computation.

---

## 1.1 Purpose and Research Intent

The purpose of this repository is investigative rather than applicative. TF-IDF
is a *classical method*; the aim is to examine the mathematical structures the
pipeline induces, and how they behave under controlled variation of assumptions
and parameters, in the context of content-based similarity and k-nearest-neighbour
ranking.

The implementation exposes preprocessing operations, n-gram structure,
document-frequency thresholds, IDF scaling and sparse vector geometry rather than
hiding the pipeline behind a black box. That access supports detailed analysis of
sensitivity and stability in TF-IDF embeddings, cosine similarity and induced
k-NN neighbourhoods, and in particular the study of cases where small
perturbations of documents, corpus composition or user-derived profiles produce
disproportionate changes in similarity scores, neighbourhood structure or ranking
outcomes.

Ranking stability is a primary object of analysis in its own right. It is
operationalised through empirical score-separation margin distributions, which
quantify the tolerance of a ranking to bounded perturbations. Instability arising
solely from deterministic tie-breaking is isolated separately: ranking outcomes
may change even when similarity scores are equal to within numerical tolerance.

The empirical work concentrates on score-level and decision-level perturbations;
corpus- and embedding-level effects are analysed analytically, to characterise
their influence on downstream stability.

---

## 1.2 Investigative Scope

Retaining the intermediate quantities that higher-level libraries abstract away
permits systematic investigation of:

- the effect of corpus perturbations on **document frequency** and **smoothed IDF values**,
- the response of **TF-IDF embeddings** to token-level edits or preprocessing changes,
- the sensitivity of **cosine similarity** to perturbations in *sparse, non-negative vectors*,
- conditions under which **similarity-based rankings** and **k-NN neighbourhoods** remain
  invariant under bounded perturbations,
- the influence of **user-profile construction** on personalised similarity and
  neighbourhood structure,
- **margin distributions** governing the stability of top-k membership and ordering
  within top-k results,
- **tie-group analysis**, quantifying ranking changes induced by deterministic
  tie-breaking rules when similarity scores fall within numerical tolerance.

Such access joins formal derivation to empirical observation, and makes the
framework suitable for studying conditioning behaviour, perturbation effects and
ranking stability in TF-IDF-based similarity systems.

---

## 1.3 Position Within a Broader Mathematical Context

TF-IDF-based similarity sits at the intersection of information retrieval,
numerical linear algebra, finite-dimensional functional analysis and
probabilistic models of text. The construction gives a concrete setting in which
abstract numerical phenomena (sparsity, scaling behaviour, angular distortion,
perturbation amplification) can be observed and analysed directly.

From a decision-theoretic perspective, similarity-based ranking is a
piecewise-constant functional of the similarity scores, with discontinuities
induced by secondary ordering and tie-breaking rules. This perspective separates
the numerical stability of the underlying similarity computation from the
stability of the ranking it induces.

The same issues recur in high-dimensional feature representations used in machine
learning, in content-based similarity and recommendation systems, and in
retrieval pipelines, where numerical sensitivity and decision discontinuities
carry practical consequences. The work is accordingly an expository and
exploratory study of established mathematical constructions in a TF-IDF setting,
attending to their behaviour under perturbation and to their implications for
ranking stability and interpretability.

---

## 1.4 Intended Use

This repository is not a production-grade recommendation library. It serves as a
reference implementation, an exploratory mathematical environment and a
foundation for *small-scale research investigations* into TF-IDF representations,
cosine similarity, content-based k-NN ranking and ranking stability.

The emphasis throughout is on mathematical behaviour: conditioning, sensitivity
and perturbation effects in sparse vector spaces, together with the relationship
between numerical similarity computation and the ranking decisions downstream of
it. These bear directly on the reliability, reproducibility and interpretability
of similarity-based and content-based recommendation models.

---

## 2. Mathematical Formulation

Let  
𝒟 = { d₁, …, dₙ }  
be a finite corpus of preprocessed documents, and define N := |𝒟|. Each document dᵢ is a
finite sequence of tokens obtained from raw text via a fixed preprocessing map
(normalisation, tokenisation, stopword removal, lemmatisation, and n-gram generation). Throughout,
**n-grams are treated as atomic tokens**.

The normative lemmatiser is the Snowball English (Porter2) **stemmer**. It is called
lemmatisation throughout for continuity with the surrounding literature, but it performs
suffix stripping rather than dictionary lemmatisation, and the two differ on a substantial
fraction of tokens. See `docs/spec_addenda.md`.

All preprocessing operations are deterministic and fixed across all perturbation
experiments.

---

### 2.1 Vocabulary, Document Frequency, and IDF

From the corpus 𝒟, a vocabulary V is constructed by collecting the tokens
(n-grams included) that satisfy a minimum document-frequency threshold and,
optionally, a maximum-feature constraint. For each token t ∈ V, the document
frequency is

df(t) = |{ i : t appears at least once in dᵢ }|.

Let N = |𝒟|. The smoothed inverse document frequency used throughout the implementation is

idf(t) = log((1 + N) / (1 + df(t))) + 1.

Thus idf(t) decays monotonically as df(t) increases, and the additive constant
keeps it strictly positive in the limiting case df(t) = N.

(Here and throughout, log denotes the natural logarithm.)

---

### 2.2 TF-IDF Embedding

For each document dᵢ, let countᵢ(t) denote the number of occurrences of token t ∈ V in
dᵢ. The term frequency is

tfᵢ(t) = countᵢ(t) / ∑ₛ∈V countᵢ(s).

Term frequencies are therefore normalised with respect to **in-vocabulary tokens**,
out-of-vocabulary tokens being ignored once the vocabulary is fixed. A document
whose in-vocabulary token count is zero maps to the zero vector.

The TF-IDF weight of token t in document dᵢ is

wᵢ(t) = tfᵢ(t) · idf(t),

and the document is represented as a sparse vector

wᵢ ∈ ℝ≥0^|V|,

with coordinates indexed by the vocabulary V.

---

### 2.3 Cosine Similarity, Ranking, and Stability Quantities

Given two non-zero TF-IDF vectors u, v ∈ ℝ≥0^|V|, the cosine similarity is

cos(u, v) = (u · v) / (‖u‖₂ ‖v‖₂),

with the convention that the similarity is set to zero if either vector is the zero
vector. All coordinates are non-negative, so

cos(u, v) ∈ [0, 1]

**in exact arithmetic**. In binary64 the computed value may exceed 1 by a few units in the last
place, because the dot product, the two norms and the division round independently; this was
observed in 27% of 40,000 random trials, the largest excess in that sample being 3 ulp. That
figure is a sample maximum and not a bound — an independent resample reached 4 ulp — so a
consumer must not treat 3 ulp as a limit. No clamping is applied, in keeping
with the numerical commitments of section 6, so a consumer converting a similarity to an angle
must clamp at its own call site. See `docs/spec_addenda.md`, G24.

Given a query vector q ∈ ℝ≥0^|V|, embedded using the same vocabulary V and IDF mapping as
the corpus documents, and a collection of document vectors { wᵢ }, the similarity scores

sᵢ = cos(q, wᵢ)

are computed for each document.

---

#### 2.3.1 Ranking Operator

Similarity scores alone do not define a total order when ties occur. The final
ranking is therefore given by a deterministic sorting operator

π = Sort((sᵢ, aᵢ)ᵢ),

where Sort orders documents by decreasing similarity score sᵢ and resolves ties
lexicographically using the fixed attribute tuple aᵢ (e.g. popularity, rating,
engagement), followed by the document **identifier**. The identifier is not an entry of aᵢ
and is never permutable: it is appended implicitly to every key, and it is what makes the
order total. This yields a total ordering (r₁, r₂, …, rₙ), where rⱼ denotes the document at
rank j.

The guarantee is conditional on identifiers being **unique**, which is validated at
construction rather than assumed. With duplicates the order ceases to be uniquely determined
and results become dependent on the sorting algorithm. Given uniqueness the key is injective,
so no two elements compare equal and the sorted output is independent of the input order.

The mapping from similarity scores to rankings is consequently not continuous
globally; it is locally constant away from tie hyperplanes.

---

#### 2.3.2 Score-Separation Margins (A1)

Let score(rⱼ) denote the similarity score of the document at rank j. The **boundary margin**
at rank k is

mₖ = score(rₖ) − score(rₖ₊₁).

This quantity governs the stability of top-k membership under bounded perturbations of
similarity scores.

The **minimum adjacent margin within the top-k** is

m_min^top = min_{1 ≤ j < k} (score(rⱼ) − score(rⱼ₊₁)).

This quantity controls the stability of the ordering within the top-k set.

The corresponding **flip radius** at rank k is

εₖ^flip = mₖ / 2,

the **supremum** of the uniform perturbation magnitudes (in score space) under
which the relative ordering of ranks k and k + 1 is preserved. The supremum is not attained:
the guarantee holds for ε < εₖ^flip, and at ε = εₖ^flip the two scores can be driven
to equality, at which point membership passes to the tie-break. The bound is tight, since a
perturbation of εₖ^flip + δ flips the pair for any δ > 0, but it is worst-case. Under
*random* perturbation no flip was observed until roughly 1.1 × εₖ^flip.

Both margins have edge cases that are reported rather than coerced: mₖ is
undefined for k ≥ N, and m_min^top is undefined at k = 1, where the minimum is over an empty
set. The two constrain **disjoint** sets of gaps, so neither bounds the other; guaranteeing
set and order membership together requires ε < min(mₖ, m_min^top)/2.

---

#### 2.3.3 Tie Groups and Decision Discontinuities (A2)

Fix the ranking  
(r₁, r₂, …, rₙ)  
induced by the deterministic ranking operator π.

To formalise near-ties, fix a numerical tolerance τ ≥ 0. At rank position j, the
associated **tie group** is

G_τ(j) = { i : |sᵢ − score(rⱼ)| ≤ τ }.

The scores sᵢ here are the same similarity scores used to produce the ranking (rⱼ).

Documents within a tie group are indistinguishable *from the reference document rⱼ* at the
level of similarity scores up to numerical tolerance. They are **not** necessarily mutually
indistinguishable: the relation |sᵢ − sⱼ| ≤ τ is reflexive and symmetric but **not transitive**,
so G_τ(j) is a ball around score(rⱼ) rather than an equivalence class, and the family of balls
does not partition the corpus. Practice therefore requires three distinct objects: the
ball above, its transitive closure (single linkage), and the maximal mutually-indistinguishable
sets (complete linkage), together with the ratio between the last two, which measures how far
transitive chaining has inflated the reported group. See `docs/spec_addenda.md`, G1.

In all three cases the final ordering within the group is determined entirely by the
deterministic tie-breaking rules embedded in the ranking operator π.

This construction separates **numerical stability of similarity scores** from
**stability of the induced ranking**, and provides a formal basis for analysing
decision-level discontinuities arising from secondary ordering criteria.

---

## 3. Solution Procedure and Implementation Structure

The repository implements the TF-IDF pipeline and the associated
perturbation-theoretic investigations explicitly and reproducibly. Each stage
exposes intermediate quantities and algebraic structure rather than optimising
performance or hiding implementation detail.

- **Preprocessing and Corpus Construction**  
  A fixed, deterministic preprocessing map is applied to raw **text of interest**:
  normalisation, tokenisation, stopword removal, lemmatisation and n-gram
  generation. The result is a reproducible corpus whose intermediate quantities
  can be inspected directly, which is what makes **controlled perturbation
  analysis** possible.

- **TF-IDF Vectorisation**  
  A pure-Python TF-IDF vectoriser constructs the vocabulary, computes document
  frequencies and smoothed inverse document frequency (IDF) values, and embeds documents
  as vectors in ℝ≥0^|V|. In typical use these embeddings are sparse, the vocabulary
  being large relative to document length.

- **Similarity, Ranking, and k-NN Structure**  
  Cosine similarity is computed between query vectors and corpus vectors, followed by
  deterministic ranking. Under **content-based k-nearest-neighbour
  recommendation**, the top-k elements of this ranking are the neighbourhood
  associated with a query. Secondary attributes, such as auxiliary metadata or
  identifiers, enter through lexicographic tie-breaking and yield a **total order** on
  candidate items.  
  The implementation also computes the **score-separation margins** governing
  top-k membership and within-top-k ordering, and supports stability profiling of
  rankings under bounded perturbations. Observed margins serve as empirical
  certificates of stability, so explicit noise injection is not required. The
  ranking procedure is further instrumented for **tie-breaking ablation
  experiments**, in which ordering is recomputed under alternate tie-break priorities
  or under score-only sorting with a fixed, attribute-independent identifier as the
  final deterministic tie-break, which separates decision-level effects
  attributable to secondary attributes from the numerical similarity computation.

- **User-Profile Documents**  
  User-specific documents are constructed from interactions such as liked, viewed or
  favourited items. They are embedded using the **same vocabulary and IDF mapping
  as the corpus**, so every similarity computation takes place in one vector space.

- **Perturbation Analysis**  
  The explicit structure supports analytical study and targeted empirical
  inspection of how small perturbations in documents, corpus composition or user
  interactions propagate through document frequencies, IDF values, TF-IDF
  embeddings, cosine similarities, induced k-NN neighbourhoods and ranking
  outcomes. Mechanisms that amplify small upstream perturbations become visible
  where sparse vector geometry, IDF scaling, angular similarity and deterministic
  decision rules interact.

The code preserves **algebraic clarity** and exposes intermediate quantities at
every stage, which suits it to further mathematical analysis of **stability**,
**sensitivity** and **decision-level fragility** in TF-IDF-based similarity systems.

Given a fixed corpus, configuration and software environment (library versions
included), all stages of the pipeline are deterministic, so similarity scores and
rankings reproduce across runs.

---

## 4. Error Analysis and Perturbation Quantities

Let wᵢ denote the TF-IDF vector associated with document dᵢ, and let

sᵢ = cos(q, wᵢ)

denote the similarity score between a query vector q and the i-th document. This section
gives quantitative measures for how perturbations affect intermediate quantities
in the TF-IDF pipeline, and how those effects reach the similarity scores and the
induced rankings.

Throughout, perturbations are treated as **bounded changes** in intermediate numerical
quantities. No probabilistic or adversarial noise model is assumed unless stated
explicitly. In the implementation they are analysed mainly at the level of
similarity scores and induced rankings, with upstream effects treated analytically.

---

### 4.1 Perturbations in Document Frequency and IDF

Consider a perturbation of the corpus induced by adding or removing a document, or by
modifying the token content of an existing document. Let df(t) and df′(t) denote the
document frequencies of token t before and after perturbation, and let N and N′ denote
the corresponding corpus sizes.

Under the smoothed IDF definition employed throughout the implementation,

idf(t) = log((1 + N) / (1 + df(t))) + 1,

the corresponding change in IDF is

Δidf(t) = idf′(t) − idf(t)  
    = log((1 + N′)/(1 + df′(t))) − log((1 + N)/(1 + df(t))).

This expression exhibits the competing effects of changes in corpus size and of
changes in the document-frequency distribution. Tokens with low document
frequency remain sensitive to corpus perturbations even under smoothing.

---

### 4.2 Perturbations in TF-IDF Vectors

Let wᵢ and wᵢ′ denote the TF-IDF vectors of a document before and after perturbation.
Writing

wᵢ  = tfᵢ ⊙ idf  
wᵢ′ = tfᵢ′ ⊙ idf′

where ⊙ denotes pointwise (Hadamard) multiplication, we obtain

wᵢ′ − wᵢ  
= (Δtfᵢ) ⊙ idf + tfᵢ ⊙ (Δidf) + (Δtfᵢ) ⊙ (Δidf),

with Δtfᵢ := tfᵢ′ − tfᵢ and Δidf := idf′ − idf.

Applying the inequality ‖a ⊙ b‖₂ ≤ ‖a‖₂ ‖b‖∞ termwise yields the bound

‖wᵢ′ − wᵢ‖₂ ≤ ‖Δtfᵢ‖₂ · ‖idf‖∞  
       + ‖tfᵢ‖₂ · ‖Δidf‖∞  
       + ‖Δtfᵢ‖₂ · ‖Δidf‖∞.

The decomposition separates perturbations arising from **local document edits**
(Δtfᵢ), **global corpus changes** (Δidf), and their interaction. In sparse
high-dimensional embeddings, the interaction of a local change with globally
scaled IDF weights is a natural mechanism for perturbation amplification.

---

### 4.3 Perturbations in Cosine Similarity

Let u, v be TF-IDF vectors and let u′, v′ denote their perturbed counterparts. Under
mild assumptions on the norms of these vectors, a standard inequality yields

|cos(u′, v′) − cos(u, v)| ≤ C (‖u′ − u‖₂ + ‖v′ − v‖₂),

for a constant C depending on lower and upper bounds on ‖u‖₂, ‖v‖₂, ‖u′‖₂, and ‖v′‖₂.

This is a Lipschitz-type bound on **score stability** under bounded perturbations
of TF-IDF vectors. It controls the magnitude of numerical changes in similarity
scores, but it does not by itself determine the stability of induced rankings once
deterministic tie-breaking rules are present (see §4.5), least of all in regimes
where score-separation margins are small or ties occur.

---

### 4.4 Ranking Stability via Score-Separation Margins (A1)

Let (r₁, r₂, …, rₙ) denote the ranking induced by sorting similarity scores in decreasing
order, and let score(rⱼ) denote the similarity score of the document at rank j.

The **boundary margin** at rank k is

mₖ = score(rₖ) − score(rₖ₊₁).

If similarity scores are subject to a uniform perturbation bounded by ε, i.e.

|Δsᵢ| ≤ ε  for all i,

then the top-k set is invariant under perturbation whenever

ε < mₖ / 2.

Similarly, the **minimum adjacent margin within the top-k** is

m_min^top = min_{1 ≤ j < k} (score(rⱼ) − score(rⱼ₊₁)).

Under the same uniform bound, the ordering within the top-k set is preserved whenever

ε < m_min^top / 2.

These are explicit, sufficient criteria for ranking stability in terms of
score-separation margins. They depend only on similarity scores and take no
account of the secondary ordering rules applied in the presence of ties.

---

### 4.5 Tie-Breaking Discontinuities and Decision Sensitivity (A2)

Let aᵢ denote a vector of secondary attributes associated with document i (e.g.
popularity, rating, engagement, identifier). The final ranking operator is

π = Sort(sᵢ, aᵢ),

where similarity scores form the primary key and secondary attributes are applied
lexicographically to resolve ties.

To isolate the effect of tie-breaking, define a **score-only ranking**

π_score = Sort(sᵢ, idᵢ),

where idᵢ is a fixed, stable identifier whose sole role is to impose a
deterministic but attribute-independent order among equal scores.

An **alternate tie-break ranking** reorders the priority of the secondary
attributes:

π_alt = Sort(sᵢ, aᵢ with reordered priority).

Fix a numerical tolerance τ > 0 and consider the regime in which the boundary
margin satisfies

mₖ ≤ τ.

Documents near the top-k boundary may then form a tie group in which
|sᵢ − sⱼ| ≤ τ. Even when

Δsᵢ ≈ 0,

the top-k set or its ordering may differ between π, π_score, and π_alt, through the
choice of deterministic tie-breaking rule alone.

This motivates a notion of **tie-break sensitivity**, measurable for example by:

- an indicator of whether the top-k set differs between π and π_score, and
- a distance between orderings restricted to tie groups (e.g. inversion count or
  Kendall τ distance).

These quantities capture **decision-level instability** that is independent of
numerical error in similarity computation and arises purely from secondary ordering
criteria.

---

## 5. Interpretation and Scope

Several structural features of **TF-IDF-based similarity systems** become clear once the pipeline is expressed in
**operator-level form**:

- **IDF sensitivity is governed by explicit logarithmic dependence** on corpus size and document-frequency counts, as seen in
  Δidf(t) = log((1 + N′)/(1 + df′(t))) − log((1 + N)/(1 + df(t))).
  IDF stability is thereby traceable to perturbations in corpus composition.

- **TF-IDF perturbations admit an explicit decomposition** into *local* (TF), *global* (IDF), and *second-order interaction* terms, which shows transparently how
  small edits propagate through the embedding.

- **Cosine similarity admits a geometric interpretation** as the cosine of the angle between *sparse, non-negative vectors*. This framing clarifies how sparsity
  patterns and IDF scaling influence angular distortion under perturbation.

- **Ranking robustness can be characterised in terms of score-separation margins**, with explicit sufficient conditions ensuring invariance under bounded perturbations. Under
  **content-based k-nearest-neighbour recommendation**, such ranking stability is the stability of the induced neighbourhoods.

- **Ranking stability is governed primarily by margin distributions, rather than by aggregate or average score changes alone.**
  Small score-separation margins near decision boundaries dominate stability behaviour even when aggregate similarity scores are
  numerically well-conditioned.

- **A long tail of near-zero margins can imply rare but extreme fragility.**
  Most rankings may be stable under small perturbations, yet documents near top-k boundaries with vanishing margins can induce abrupt changes in neighbourhood
  structure under otherwise negligible score variation.

- **Deterministic tie-breaking introduces non-perturbative discontinuities.**  
  With no meaningful numerical perturbation at all (Δs ≈ 0), ranking outcomes may change through secondary ordering rules alone: stability of computed similarities
  does not guarantee stability of downstream decisions.

The emphasis throughout is on **derivational transparency** rather than algorithmic optimisation. No dimensionality reduction, latent-semantic modelling, or neural
embeddings are introduced. The aim is to expose the **algebraic and geometric structure** of the TF-IDF pipeline in a form suitable for
**perturbation analysis**, **stability reasoning**, and **controlled experimentation**.

---

## 6. Limitations

Several limitations of the present framework should be noted:

- **High-dimensional sparsity complicates geometric intuition.**  
  TF-IDF vectors inhabit a large, sparse subset of ℝⁿ, where small changes in support can
  produce disproportionately large angular effects, hence instability in
  similarity scores and, by extension, in induced k-NN neighbourhoods.

- **Cosine similarity becomes unstable for low-norm vectors.**  
  When documents are short or contain few in-vocabulary tokens, the norms ‖u‖₂ and ‖v‖₂
  may become small, which raises sensitivity to perturbations in similarity values. In
  such regimes score-separation margins tend to shrink, so ranking outcomes fall
  increasingly to deterministic tie-breaking rules rather than to similarity
  geometry.

- **Smoothed IDF reduces but does not eliminate volatility associated with rare tokens.**  
  Tokens with very low document frequency remain highly sensitive to corpus perturbations,
  even under smoothing, and can dominate similarity computations in sparse settings.

- **Deterministic tie-breaking can dominate outcomes under near-ties.**  
  When score-separation margins fall below numerical tolerance, ranking outcomes may be
  determined primarily by secondary ordering rules. Stability results resting solely
  on similarity scores or margin conditions then fail to capture the
  decision-level instability in the induced rankings.

- **No numerical optimisation techniques are employed.**  
  Stabilising transformations, such as sublinear TF scaling, vector normalisation
  variants or dimensionality reduction, are omitted to preserve **analytical
  clarity** and the direct interpretability of perturbation effects.

The framework is accordingly intended for **analytical**, **educational**, and
**controlled experimental use** rather than production deployment or large-scale
retrieval tasks.

---

## 7. Experimental Protocol and Results

This section gives the evaluation protocol and the empirical results for the two
stability questions analysed throughout:

- (A1) How score-separation margins govern the stability of similarity-based rankings
  under bounded perturbations.
- (A2) How deterministic tie-breaking rules induce decision-level discontinuities in
  the presence of near-ties.

The experiments characterise stability and fragility properties of the implemented
TF-IDF plus cosine ranking pipeline under a fixed, deterministic preprocessing,
embedding and ranking configuration. They do not benchmark retrieval quality
against external baselines.

---

### 7.1 Query Set and Evaluation Setup

**Corpus and representation.**  
All documents are embedded using the fixed preprocessing map, vocabulary construction,
smoothed IDF definition, and TF normalisation specified in §2.1 and §2.2. Similarity scores
are computed exclusively using cosine similarity as defined in §2.3.

**Query construction.**  
Two query construction strategies are used, as implemented in the repository:

- **User-profile queries:** a query document is constructed by aggregating text from a
  user’s interacted items (e.g. liked, viewed, or favourited), and embedded into the
  same TF-IDF space as the corpus.
- **Leave-one-out evaluation:** for users with multiple interactions, one interacted
  item is removed and treated as a held-out target; the remaining interactions form
  the query profile.

Other supported query modes (e.g. item-as-query) are part of the implementation but
are not evaluated in the present experiments.

**k values.**  
All stability metrics are evaluated for k ∈ {5, 10, 20, 50}.

**Near-tie tolerance.**  
A fixed numerical tolerance τ > 0 defines the near-tie regime. Two similarity scores
sᵢ and sⱼ are treated as indistinguishable whenever |sᵢ − sⱼ| ≤ τ, and tie groups are
defined as in §2.3.3. The value of τ is chosen to exceed floating-point noise while
remaining small relative to typical score separations. All tie-break sensitivity
results are explicitly conditional on this choice of τ.

All reported distributions are computed over the full set of evaluated queries;
the exact number of queries and users depends on the dataset configuration provided
in the repository.

Unless stated otherwise, all reported results use a fixed τ across queries.

---

### 7.2 Margin Distributions and Ranking Stability (A1)

For each query, let (r₁, r₂, …) denote the ranking induced by sorting similarity scores
in decreasing order. Boundary and within-top-k margins are defined as in §§2.3.2 and 4.4:

- mₖ = score(rₖ) − score(rₖ₊₁)
- m_min^top = min_{1 ≤ j < k} (score(rⱼ) − score(rⱼ₊₁))

For each k ∈ {5, 10, 20, 50}, empirical distributions of mₖ and m_min^top are computed
across all evaluated queries.

**Reported statistics.**
- Percentile summaries of mₖ and m_min^top (including lower-tail, median, and upper-tail
  behaviour).
- Corresponding flip radii εₖ^flip = mₖ / 2, which provide sufficient bounds for top-k
  stability under uniform score perturbations.

**Visualisation.**
- Empirical cumulative distribution functions (ECDFs) or histograms of mₖ.
- ECDFs or histograms of m_min^top.

These distributions show whether ranking stability is typical (margins well
separated from zero) or fragile (substantial mass concentrated near zero), particularly
at decision boundaries.

---

### 7.3 Tie-Break Ablations and Decision Sensitivity (A2)

To isolate decision-level effects arising from deterministic secondary ordering,
rankings are recomputed under the following sorting operators (§4.5):

- **Full ranking:** π = Sort(sᵢ, aᵢ)
- **Score-only ranking:** π_score = Sort(sᵢ, idᵢ)
- **Alternate tie-break ranking:** π_alt = Sort(sᵢ, aᵢ with reordered priority)

For each query and each k, the following quantities are measured:

- **Top-k disagreement rate:** the fraction of queries for which the top-k set differs
  between π and π_score, and between π and π_alt.
- **Within-top-k reordering:** an ordering distance restricted to tie-affected
  subsets (e.g. inversion count or Kendall τ distance).

Results are stratified by the boundary margin mₖ relative to τ, since tie-break effects
are expected to concentrate in the near-tie regime mₖ ≤ τ.

**Visualisation.**
- Probability of top-k disagreement as a function of mₖ, highlighting the transition
  region around mₖ ≈ τ.

This analysis separates numerical stability of similarity scores from instability
introduced purely by deterministic decision rules.

---

### 7.4 Constructed Near-Tie Case Study

To exhibit decision-level discontinuities explicitly, a near-tie case is reported
for a fixed query:

- Two documents A and B are identified such that |s_A − s_B| ≤ τ near the top-k
  boundary (or as adjacent elements within the top-k).
- The tuple (s_A, s_B, mₖ, τ) and the associated tie-break attributes (a_A, a_B) are
  reported.
- Ranking outcomes under π, π_score, and π_alt are compared.

The case study is **illustrative rather than representative**. It makes concrete
that ranking outcomes can change with Δs ≈ 0, driven solely by deterministic
tie-breaking rules, and it instantiates the decision-level discontinuities
analysed abstractly in §§2.3.3 and 4.5.

---

## 8. References

The following sources provide the theoretical, numerical, and conceptual background that informs the present work.

### Classical Information Retrieval

- Manning, C. D., Raghavan, P., & Schütze, H. (2008). *Introduction to Information Retrieval*. Cambridge University Press.  
  A foundational treatment of TF-IDF, vector-space models, and classical retrieval pipelines.

- Zobel, J., & Moffat, A. (2006). *Inverted files for text search engines*. ACM Computing Surveys.

- Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. *Information Processing & Management*.  
  The classical reference for TF-IDF weighting schemes and early vector-space retrieval.

- Cover, T. M., & Hart, P. E. (1967). *Nearest neighbor pattern classification*. IEEE Transactions on Information Theory.

### Numerical Linear Algebra and Stability

- Higham, N. J. (2002). *Accuracy and Stability of Numerical Algorithms* (2nd ed.). SIAM.  
  A standard reference on perturbation analysis, conditioning, and stability in numerical computation.

- Trefethen, L. N., & Bau, D. (1997). *Numerical Linear Algebra*. SIAM.  
  Provides geometric intuition for high-dimensional vector spaces and operator behaviour.

### Sparse Vector Geometry and Similarity

- Aggarwal, C. C. (2015). *Data Mining: The Textbook*. Springer.  
  Discusses sparsity, high-dimensional geometry, and similarity measures in data-analytic contexts.

- Leskovec, J., Rajaraman, A., & Ullman, J. D. (2020). *Mining of Massive Datasets*. Cambridge University Press.  
  Covers vector-space models, similarity search, and large-scale retrieval behaviour.

### Statistical Learning Context

- Hastie, T., Tibshirani, R., & Friedman, J. (2009). *The Elements of Statistical Learning* (2nd ed.). Springer.  
  Provides broader context for feature representations and similarity-based methods.

- Shalev-Shwartz, S., & Ben-David, S. (2014). *Understanding Machine Learning: From Theory to Algorithms*. Cambridge University Press.  
  Offers a theoretical perspective on learning systems that rely on vector-space representations.

### Online Resources

- *Cosine similarity*. Wikipedia.  
  https://en.wikipedia.org/wiki/Cosine_similarity

- *TF-IDF*. Wikipedia.  
  https://en.wikipedia.org/wiki/Tf%E2%80%93idf

- *Vector space model*. Wikipedia.  
  https://en.wikipedia.org/wiki/Vector_space_model

These online resources provide accessible summaries of standard definitions and terminology used throughout the repository.

---

## 9. Authorship

**Implementation and exposition**  
Matthew Maksymilian Miezaniec  
Email: matthewmiezaniec1@gmail.com  

The implementation covers the full TF-IDF similarity pipeline, the explicit perturbation analysis tooling, **stability
profiling via score-separation margins**, and a **tie-break ablation framework** for isolating decision-level
discontinuities in ranking outcomes.

**Mathematical and theoretical foundations**  
This work draws on **classical information retrieval methodology**, including TF-IDF weighting and vector-space models
(Salton; Manning et al.), and on established treatments of **numerical stability**, **conditioning**, and
**perturbation behaviour** in high-dimensional vector spaces (Higham; Trefethen & Bau).

Reading similarity scores as **content-based k-nearest-neighbour ranking and neighbourhood structure**
follows classical nearest-neighbour and similarity-search perspectives (Cover & Hart), with no learning-based methods introduced.

Broader contextual connections to feature representations and similarity-based reasoning draw on standard statistical learning
references (Hastie, Tibshirani & Friedman; Shalev-Shwartz & Ben-David). Supplementary intuition and terminology come from
widely used online references on **TF-IDF**, **cosine similarity**, and **vector-space models**.

---

## 10. Acknowledgements

The author thanks colleagues and peers for informal discussions that helped clarify aspects of numerical stability, sparse
vector geometry, and similarity-based reasoning. The work also benefited from exposure to standard academic treatments of
information retrieval and numerical linear algebra through coursework, independent study, and open-source documentation.

Any remaining errors or omissions are the responsibility of the author alone. This acknowledgement does not imply endorsement
or direct contribution by any individual or institution.

---

## 11. License

This repository is provided for analytical, educational, and research-oriented use.  
See the accompanying license file for full terms and conditions.
