Metadata-Version: 2.4
Name: dynquant
Version: 0.1.1
Summary: Training-dynamics-driven mixed-precision LLM quantization, with prebuilt CUDA kernels.
Project-URL: Homepage, https://github.com/kambojvikram/dynquant
Project-URL: Issues, https://github.com/kambojvikram/dynquant/issues
Project-URL: Changelog, https://github.com/kambojvikram/dynquant/blob/main/CHANGELOG.md
Author: DynQuant authors
License-Expression: Apache-2.0
License-File: LICENSE
Keywords: compression,cuda,llm,mixed-precision,moe,quantization,transformers
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: GPU :: NVIDIA CUDA
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Science/Research
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Requires-Dist: dynquant-core==0.1.1
Requires-Dist: dynquant-kernels<0.2,>=0.1.1; platform_system == 'Linux' and platform_machine == 'x86_64'
Provides-Extra: dev
Requires-Dist: dynquant-core[dev]==0.1.1; extra == 'dev'
Provides-Extra: eval
Requires-Dist: dynquant-core[eval]==0.1.1; extra == 'eval'
Provides-Extra: hf
Requires-Dist: dynquant-core[hf]==0.1.1; extra == 'hf'
Provides-Extra: kernels
Requires-Dist: dynquant-kernels<0.2,>=0.1.1; extra == 'kernels'
Provides-Extra: train
Requires-Dist: dynquant-core[train]==0.1.1; extra == 'train'
Provides-Extra: triton
Requires-Dist: dynquant-core[triton]==0.1.1; extra == 'triton'
Description-Content-Type: text/markdown

# dynquant

**Mixed-precision LLM quantization that decides bit-widths from your fine-tune's own training dynamics.**

```bash
pip install dynquant
dynquant doctor
```

This distribution contains no code. It is the one name to install, and it pulls in:

* [`dynquant-core`](https://pypi.org/project/dynquant-core/) — the Python half:
  signal collection hook, role classification, scoring, allocation, packing, CLI.
  Installs anywhere, no compiler required.
* [`dynquant-kernels`](https://pypi.org/project/dynquant-kernels/) — prebuilt CUDA
  kernels, where a wheel exists for your platform. Without them everything still
  works on the reference backend; you lose inference speed and the VRAM saving,
  not correctness.

## Usage

Collect signals during the fine-tune you were going to run anyway:

```python
from transformers import Trainer
from dynquant import DynQuantCallback

trainer = Trainer(model=model, ..., callbacks=[DynQuantCallback("stats/")])
trainer.train()
```

Allocate and pack:

```bash
dynquant quantize ./merged --stats stats/dynquant_stats.json --target 3.0 -o ./q3
```

Load through plain transformers:

```python
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("./q3")
```

Full documentation: <https://github.com/kambojvikram/dynquant>

## Extras

```bash
pip install 'dynquant[train]'     # transformers, peft, trl, datasets
pip install 'dynquant[eval]'      # lm-eval-harness
pip install 'dynquant[triton]'    # portability fallback for ROCm / newer GPUs
pip install 'dynquant[kernels]'   # force the compiled kernels (builds from sdist)
```

## License

Apache-2.0.
