Metadata-Version: 2.4
Name: faster-clip
Version: 0.4.0
Classifier: Programming Language :: Rust
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: POSIX :: Linux
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Intended Audience :: Developers
License-File: LICENSE
Summary: Fast CPU inference (Rust) for CLIP embeddings, inference-free SPLADE, and BERT fill-mask — drop-in replacement for the transformers / sentence-transformers models in a RAG stack. No PyTorch, no ONNX.
Keywords: clip,splade,sparse-embeddings,fill-mask,embeddings,rag,rust,no-torch
Author: cnmoro
License: MIT
Requires-Python: >=3.8
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: Homepage, https://github.com/cnmoro/faster-clip
Project-URL: Repository, https://github.com/cnmoro/faster-clip

# faster-clip

Fast, dependency-light **CPU inference** for the models a multilingual RAG stack
needs — **no PyTorch, no transformers, no sentence-transformers, no ONNX**. Since
0.2 the compute is a hand-written **Rust** engine (a single native extension);
tokenization uses the `tokenizers` crate.

```bash
pip install faster-clip
```

Three drop-in replacements, each **numerically matching** the original HuggingFace
pipeline:

| Class | Replaces | Output |
|---|---|---|
| `ClipModel` | `SentenceTransformer('clip-ViT-B-32')` + `…-multilingual-v1` | 512-d CLIP image / text embeddings (cosine 1.0) |
| `SparseEncoder` | `SparseEncoder('…inference-free-splade…')` | `{token_id: weight}` sparse dicts (**bit-identical**) |
| `FillMask` | `pipeline('fill-mask', 'bert-base-uncased' / 'bert-base-portuguese-cased')` | top-k `{token_str, sequence, score}` (**identical strings**) |

## Performance

CPU (AMD Ryzen 7 3700X, 8 cores), single item, vs the original torch pipeline:

| | torch / sentence-transformers | **faster-clip (Rust)** |
|---|---|---|
| CLIP text embed | ~10 ms | **~11 ms** (≈ parity) |
| CLIP image embed | ~62 ms | **~81 ms** |
| fill-mask (BERT) | ~60 ms | **~20 ms** (3× faster) |
| SPLADE sparse | — | **< 0.05 ms** (pure lookup) |

No torch/transformers install (multi-GB) — one native wheel. Since 0.4 the 2-D
weights are stored as **int8** (per-output-channel symmetric quantization,
upconverted on the fly in the AVX2 kernels); small layernorm/bias params stay f16.
Loading copies everything into owned buffers and drops the source mmap, so nothing
of the original f32 file stays resident. Measured steady-state RSS on CPU:

| loaded models | f32 / torch | **faster-clip 0.4 (int8)** |
|---|---|---|
| CLIP image + text | ~1.8 GB | **~280 MiB** |
| + SPLADE + fill-mask (en+pt) | — | **~495 MiB** |

Accuracy is essentially unchanged: **CLIP cosine ≥ 0.9997** vs torch, SPLADE
bit-identical, fill-mask top-5 exact (en + pt).

## CLIP embeddings

```python
from faster_clip import ClipModel

clip = ClipModel()  # lazily downloads image / text weights on first use
text_emb = clip.encode_text(["um carro vermelho", "a red car"])   # (2, 512) list
img_emb  = clip.encode_image([open("photo.jpg", "rb").read()])    # (1, 512) list
```

* `encode_text(list_of_str)` → list of 512-float vectors.
* `encode_image(list_of_image_bytes)` → list of 512-float vectors. Pass the **raw
  image bytes** (PNG/JPEG/…); decoding + CLIP preprocessing happen in Rust.

Image and multilingual text share the same 512-d space. Normalize yourself for
cosine search.

## SPLADE sparse encoding (no sparse tensor materialised)

```python
from faster_clip import SparseEncoder
sparse = SparseEncoder()
sparse.encode("banco de dados vetorial")     # -> {'15060': 1.09, '2951': 1.11, ...}
```

Inference-free: a token→weight lookup, so only the active dims are returned — no
dense/sparse vector is ever built. `encode`, `encode_query`, `encode_document`
are equivalent; each takes a str or list of str. The weights ship in the wheel.

## Fill-mask autocomplete

```python
from faster_clip import FillMask
fm = FillMask()
fm.predict("the weather is very [MASK]", lang="en")
fm.predict("o brasil é um [MASK]", lang="pt")
# -> [{'token_str': 'lixo', 'sequence': 'o brasil é um lixo', 'score': 0.18}, ...]
```

`lang` is `"en"` (`bert-base-uncased`) or `"pt"` (`neuralmind/bert-base-portuguese-cased`).

## Model cache

CLIP and BERT weights are downloaded on first use and cached under
`~/.cache/faster-clip` (override with `$FASTER_CLIP_CACHE`). Inference releases
the GIL, so one instance is safe to share across threads.

## License

MIT. Model weights belong to their respective authors (OpenAI / sentence-transformers
/ neuralmind / Microsoft) under their original licenses.

