Metadata-Version: 2.4
Name: tiny-turboquant
Version: 0.1.0
Summary: Low-bit vector and KV-cache compression research toolkit for PyTorch
Author: Pradeep Boopathy
License-Expression: MIT
Project-URL: Homepage, https://github.com/pradeepboopathy/tiny-turboquant
Project-URL: Repository, https://github.com/pradeepboopathy/tiny-turboquant
Project-URL: Issues, https://github.com/pradeepboopathy/tiny-turboquant/issues
Keywords: quantization,kv-cache,llm,compression,vector-search,pytorch,rag,transformers
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.1
Requires-Dist: numpy>=1.24
Provides-Extra: dev
Requires-Dist: pytest>=8; extra == "dev"
Requires-Dist: build>=1.2; extra == "dev"
Requires-Dist: twine>=5.0; extra == "dev"
Requires-Dist: ruff>=0.5; extra == "dev"
Provides-Extra: demos
Requires-Dist: matplotlib>=3.7; extra == "demos"
Requires-Dist: faiss-cpu>=1.7.4; extra == "demos"
Requires-Dist: sentence-transformers>=2.6; extra == "demos"
Provides-Extra: llm
Requires-Dist: transformers>=4.40; extra == "llm"
Requires-Dist: accelerate>=0.30; extra == "llm"
Provides-Extra: all
Requires-Dist: matplotlib>=3.7; extra == "all"
Requires-Dist: faiss-cpu>=1.7.4; extra == "all"
Requires-Dist: sentence-transformers>=2.6; extra == "all"
Requires-Dist: transformers>=4.40; extra == "all"
Requires-Dist: accelerate>=0.30; extra == "all"
Dynamic: license-file

# Tiny TurboQuant

Tiny TurboQuant is a lightweight PyTorch research toolkit for low-bit vector compression and KV-cache compression experiments.

It includes:

- real `uint8` bit-packing for low-bit indices
- MSE-style scalar quantization
- product / inner-product-oriented quantization experiments
- outlier-split quantization
- a Hugging Face-compatible KV-cache prototype
- demos for ANN search, embedding compression, and KV-cache memory measurement

## Important limitation

This package demonstrates packed memory compression. It is **not** a production compressed-attention engine. The KV-cache wrapper still dequantizes tensors before attention. Real latency gains require fused CUDA/Triton kernels or integration with an inference engine such as vLLM or TensorRT-LLM.

## Install from local wheel

```bash
pip install tiny_turboquant-0.1.0-py3-none-any.whl
```

## Basic usage

```python
import torch
from tiny_turboquant import TurboQuantMSE, TurboQuantKVCache

x = torch.randn(128, 64)
x = x / x.norm(dim=-1, keepdim=True)

q = TurboQuantMSE.build(d=64, bits=4)
idx = q.quant(x)
x_hat = q.dequant(idx)

cache = TurboQuantKVCache(bits=4)
```

## Run tests

```bash
python -m pytest
```

## Run demos from source checkout

```bash
python -m demos.demo1_distortion_vs_theory
python -m demos.demo2_ann_vs_pq
python -m demos.demo3_real_embeddings
python -m demos.demo4_kv_cache
```

## Scope

Use this package for:

- compressed vector-search experiments
- RAG embedding compression experiments
- KV-cache memory/quality tradeoff experiments
- educational or research benchmarking

Do not claim it provides production-speed LLM inference. It reduces packed storage; speed requires optimized compressed-attention kernels.
