Metadata-Version: 2.4
Name: mantissa-embed
Version: 0.1.0
Summary: CNN metric learning on the mantissa C engine: image embeddings for similarity, verification and retrieval
Author: Tekin Ertekin
License: MIT
Project-URL: Homepage, https://github.com/tekinertekin/mantissa-embed
Project-URL: Base, https://github.com/tekinertekin/mantissa-cnn
Project-URL: Engine, https://github.com/tekinertekin/mantissa
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: numpy>=1.20
Requires-Dist: mantissa-cnn>=0.2.2
Provides-Extra: viz
Requires-Dist: matplotlib>=3.5; extra == "viz"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Dynamic: license-file

# mantissa-embed

![License](https://img.shields.io/badge/license-MIT-blue.svg)
![Python](https://img.shields.io/badge/python-3.9%2B-3776AB.svg)
[![Base](https://img.shields.io/badge/base-mantissa--cnn-4B8BBE.svg)](https://github.com/tekinertekin/mantissa-cnn)
[![Engine](https://img.shields.io/badge/engine-mantissa-00599C.svg)](https://github.com/tekinertekin/mantissa)

**Learning what "similar" means, with a C engine.** A classifier answers *which
class*; an embedder answers *how alike* — it maps each image to a point in a
vector space where same-thing images land close together and different-thing
images land far apart. That single geometry powers similarity search,
verification ("are these two the same?") and retrieval ("show me the nearest
matches"), for classes the model was never trained to name.

`mantissa-embed` trains that embedding with **metric learning** — the
contrastive and triplet losses — on top of a
[mantissa-cnn](https://github.com/tekinertekin/mantissa-cnn) trunk: its
Conv2D / MaxPool2D / Flatten / Dense layers, its
[mantissa](https://github.com/tekinertekin/mantissa) C-engine and pure-numpy
backends, and its dataset loaders are reused, not reimplemented. This package
adds only what metric learning needs on top of a classifier: the two losses,
an `Embedder` that trains a trunk with them, and the four things embeddings buy
you — `fit` / `embed` / `retrieve` / `verify`.

## The mantissa family

Part of the **mantissa** family: a low-precision engine written in C, with
small Python packages built on top. Each package sits under the one it depends
on — ⭐ marks where you are, and every other name links to its repo.

- [mantissa](https://github.com/tekinertekin/mantissa) — low-precision neural-network engine in C (the core)
  - [mantissa-perceptron](https://github.com/tekinertekin/mantissa-perceptron) — perceptron & ADALINE, the linear classics
  - [mantissa-nn](https://github.com/tekinertekin/mantissa-nn) — shared neural-net primitives (layers, engine binding)
    - [mantissa-cnn](https://github.com/tekinertekin/mantissa-cnn) — convolutional networks for images
      - [mantissa-auto-encoder](https://github.com/tekinertekin/mantissa-auto-encoder) — autoencoders for denoising & super-resolution
      - [mantissa-interpret](https://github.com/tekinertekin/mantissa-interpret) — CNN interpretability (occlusion, saliency, Grad-CAM)
      - ⭐ **mantissa-embed** — CNN metric learning (image embeddings for similarity & retrieval) *(you are here)*
    - [mantissa-mlp](https://github.com/tekinertekin/mantissa-mlp) — multilayer perceptrons, fully-connected nets


## New to metric learning?

A classifier learns a fixed set of labels and a decision boundary between them.
Metric learning learns something more basic and more reusable: a **distance**.
It trains the network so that the Euclidean distance between two images'
embeddings *is* a measure of how similar they are — and it does that using only
the relation between examples ("these two are the same kind", "these two are
not"), never a class *name*. Two consequences follow:

- **It generalizes to classes it never trained on.** Because the loss only ever
  says "closer" or "farther", a model trained on some identities produces useful
  distances for identities it has never seen — the basis of face verification,
  where you cannot retrain for every new person.
- **One embedding serves many tasks.** Compute it once per image, then:
  *verify* by thresholding a distance, *retrieve* by nearest-neighbour search,
  *cluster* by feeding the vectors to any clustering method.

Two classic losses do the training, and this package implements both:

- **Contrastive loss** works on **pairs**. A same-class pair pays its squared
  distance (pulled together); a different-class pair pays a hinge that is zero
  once the pair is at least a `margin` apart (pushed apart, but only until far
  enough) — Hadsell, Chopra & LeCun (2006), "Dimensionality Reduction by
  Learning an Invariant Mapping", *CVPR*.
- **Triplet loss** works on **triples** of (anchor, positive, negative). It asks
  only that the negative be farther from the anchor than the positive, by at
  least a `margin` — a *relative* constraint, which is often easier to satisfy
  and to scale than pinning absolute distances — Schroff, Kalenichenko &
  Philbin (2015), "FaceNet: A Unified Embedding for Face Recognition and
  Clustering", *CVPR*.

**A Siamese/triplet network is just one network run more than once.** The two
legs of a pair (or three of a triplet) share the *same* weights, so there is no
second network to manage: stack the legs into one batch, run a single forward
pass, compute the loss and its per-row gradient on the resulting embeddings,
and backpropagate that gradient through the one trunk. That is exactly what
`Embedder.fit` does — the same custom forward/loss/backward loop the rest of the
family uses, with a metric loss where the classifier's softmax would be.

## Install

```sh
pip install mantissa-embed
```

Pulls in `mantissa-cnn >= 0.2.2` (and transitively `mantissa-nn` + the
`mantissa-core` engine). For the demo's plots, `pip install mantissa-embed[viz]`.

From checkouts (works today, no PyPI needed): clone this repo, `cnn`,
`mantissa-nn` and [mantissa](https://github.com/tekinertekin/mantissa) side by
side, build the engine (`make dist` there), then here:

```sh
pip install -e ../mantissa -e ../mantissa-nn -e ../cnn && pip install -e ".[viz]"
```

mantissa-cnn finds the sibling engine checkout automatically, and its dataset
loaders find a `data/` directory via `MANTISSA_CNN_DATA` (the demo points this
at the sibling `cnn/data/` for you).

## Quickstart

```sh
# datasets are mantissa-cnn's; nothing downloads implicitly — fetch once:
python -m mantissa_cnn.datasets download mnist
```

```python
from mantissa_cnn import datasets
from mantissa_embed import Embedder, models

Xtr, ytr, Xte, yte = datasets.subset("mnist", 6000, 2000, seed=0)

emb = Embedder(models.small_cnn_embedder(embed_dim=16), loss="triplet")  # or "contrastive"
emb.fit(Xtr, ytr, epochs=8, batch_size=64, lr=0.05, verbose=True)

Z = emb.embed(Xte)                       # (2000, 16) embedding vectors
idx, dist = emb.retrieve(Xte[0], Z, k=5) # 5 nearest test images to query 0
same = emb.verify(Xte[0], Xte[1], threshold=1.0)   # same identity? (bool)
```

Or compose your own trunk from mantissa-cnn's layers — any Conv/Pool/Flatten
stack ending in `Dense(embed_dim, act="identity")`:

```python
from mantissa_cnn import Conv2D, MaxPool2D, Flatten, Dense
from mantissa_embed import Embedder

emb = Embedder(
    [Conv2D(16, 3, pad=1), MaxPool2D(2), Flatten(), Dense(32)],  # 32-D embedding
    loss="contrastive", margin=1.0, seed=0)
```

## The API

`Embedder(layers, loss="triplet"|"contrastive", margin=None, seed=0, backend="mantissa")`,
then:

| method | does | returns |
|---|---|---|
| `fit(X, y, epochs, batch_size, lr, verbose)` | trains the trunk by metric learning; `y` only decides same/different | `self` (`history_["loss"]` per epoch) |
| `embed(X)` | one forward pass, chunked | `(n, embed_dim)` embeddings |
| `retrieve(query, gallery, k)` | k nearest gallery items in embedding space (numpy `argpartition`, no sklearn) | `(indices, distances)`, nearest-first |
| `verify(a, b, threshold=None)` | embedding distance between `a` and `b`; with `threshold`, a boolean same/different | distance, or bool |

`retrieve` and `verify` accept raw images **or** a pre-computed `(., embed_dim)`
matrix, so a fixed gallery is embedded once and reused across queries.

Deliberately minimal, like the rest of the family: NCHW float32 images, plain
SGD, the two classic losses, plain Euclidean distances. No autograd graph, no
optimizer zoo, no ANN index. The trunk's convolutions run in C on zero-copy
float32 buffers; the losses are memory-bound reductions over embedding vectors
and honestly stay in numpy (`mantissa_embed.losses`, importable and gradient-
checked on their own). Layers allocate scratch once per batch shape and reuse
it.

## Results

A `small_cnn_embedder` (52,528 params, 16-D embedding) trained on a 6k-image
MNIST subset with the triplet loss and the mantissa C engine (8 epochs, triplet
loss **0.090 → 0.016**), then embedding the held-out 2k test set. Reproduce with
`python examples/embeddings_demo.py`.

The embedding separates the digits with no label ever entering the loss — colour
is the true digit, shown only for the plot:

![MNIST test embeddings, 2-D PCA, coloured by true digit — visible per-digit clusters](https://raw.githubusercontent.com/tekinertekin/mantissa-embed/main/assets/embedding_manifold.png)

And nearest-neighbour retrieval returns same-digit images (green border = a
correct match, i.e. the neighbour's true digit equals the query's):

![retrieval gallery: five query digits, each with its five nearest neighbours in embedding space](https://raw.githubusercontent.com/tekinertekin/mantissa-embed/main/assets/retrieval_gallery.png)

## License

MIT — © Tekin Ertekin. Base package:
[mantissa-cnn](https://github.com/tekinertekin/mantissa-cnn); engine:
[mantissa](https://github.com/tekinertekin/mantissa) — same author, MIT.
