Metadata-Version: 2.4
Name: bitsandbytes-kernels
Version: 0.50.0.dev0
Summary: VQ-L2 quantization kernels (import bitsandbytes). Pulled by the bitsandbytes2 metapackage.
Author-email: Tim Dettmers <tim.dettmers@gmail.com>
License-Expression: MIT
Project-URL: homepage, https://github.com/TimDettmers/bitsandbytes2
Project-URL: changelog, https://github.com/TimDettmers/bitsandbytes2/blob/main/CHANGELOG.md
Project-URL: issues, https://github.com/TimDettmers/bitsandbytes2/issues
Keywords: gpu,quantization,vq-l2,inference,cuda,metal
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: GPU :: NVIDIA CUDA :: 11.8
Classifier: Environment :: GPU :: NVIDIA CUDA :: 12
Classifier: Environment :: GPU :: NVIDIA CUDA :: 13
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Operating System :: POSIX :: Linux
Classifier: Operating System :: MacOS
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: C++
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch<3,>=2.3
Requires-Dist: numpy>=1.17
Requires-Dist: packaging>=20.9
Provides-Extra: benchmark
Requires-Dist: pandas; extra == "benchmark"
Requires-Dist: matplotlib; extra == "benchmark"
Provides-Extra: dev
Requires-Dist: bitsandbytes-kernels[test]; extra == "dev"
Requires-Dist: build<2,>=1.0.0; extra == "dev"
Requires-Dist: ruff~=0.14.3; extra == "dev"
Requires-Dist: pre-commit<4,>=3.5.0; extra == "dev"
Requires-Dist: wheel<1,>=0.42; extra == "dev"
Provides-Extra: test
Requires-Dist: einops~=0.8.0; extra == "test"
Requires-Dist: lion-pytorch==0.2.3; extra == "test"
Requires-Dist: pytest~=8.3; extra == "test"
Requires-Dist: scipy<2,>=1.11.4; extra == "test"
Requires-Dist: transformers<5,>=4.30.1; extra == "test"
Dynamic: license-file

# bitsandbytes2

Quantization kernels, an inference runtime, and a coding agent in one
repository. This is the working branch. A history-purged private orphan
comes later from a cleaned tip of this tree.

| Path | Product | Install name |
|---|---|---|
| `.` (`bitsandbytes/`, `csrc/`) | VQ-L2 / NF4 / int8 kernels | `bitsandbytes-kernels` (import `bitsandbytes`) |
| `bitsandbytes.cpp/` | Hugging Face inference runtime | `bitsandbytes-cpp` |
| `bitsandbytes-code/` | coding agent | `bitsandbytes-code` |
| metapackage | pulls the three | `bitsandbytes2` |

See [ECOSYSTEM.md](ECOSYSTEM.md) for layout, sources, and what is still dirty.

## Install (target)

No compiler on the user machine. One command pulls the three packages:

```text
pip install bitsandbytes2
```

Individual pieces stay installable (`bitsandbytes-kernels`, `bitsandbytes-cpp`,
`bitsandbytes-code`). First public index is TestPyPI. From this checkout:

```text
pip install -e .
```

installs the Python package only. CUDA kernels still need a cmake build in
this development tree; release wheels will ship the `.so` / `.dylib`.

## What this package is

- **VQ-L2** vector quantization (2–5 bits) for weights and KV cache
- **Fused kernels** (`quantize_vq_l2`, `fused_vq_l2_matmul_small_m`, NVFP4)
- **C ABI** (`csrc/bnb_backend_abi.h`) consumed by `bitsandbytes.cpp`
- **Dynamic bit allocation** (`dynamic_bitalloc.py`)

CUDA and CPU binaries exist. Metal sources exist (`csrc/backend_api_metal.mm`);
a release dylib is not in this tree yet. AMD is later.

The coding agent currently needs PostgreSQL at runtime.

## License

MIT. See `LICENSE` and `NOTICE.md`.
