Metadata-Version: 2.4
Name: fastllm-turbo
Version: 1.0.0
Summary: Run AI models 2x faster on CPU with zero quality loss
Home-page: https://fast-llm.dev
Author: Kasish
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: torch>=2.0.0
Requires-Dist: transformers>=4.40.0
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# Fast LLM

Run any AI model **2x faster** on CPU — zero quality loss, zero training.

```
pip install fast-llm
fast-llm run Qwen/Qwen2.5-0.5B --quantize --benchmark
```

## How it works

INT8 dynamic quantization converts model weights from 32-bit to 8-bit precision.
This cuts memory bandwidth in half — the main bottleneck on CPU inference.
The output is coherent English, indistinguishable from the original.

| Mode | Speed | Output |
|------|-------|--------|
| FP32 (before) | 1.18 tok/s | "The future of AI is in the hands of the people." |
| INT8 (after)  | **2.32 tok/s** | "The future of AI is the future of the world." |

## Install

**Linux / macOS**
```bash
curl -sSf https://fast-llm.dev/install.sh | sh
```

**Windows (PowerShell)**
```powershell
powershell -c "iex ((New-Object Net.WebClient).DownloadString('https://fast-llm.dev/install.ps1'))"
```

**pip (all platforms)**
```bash
pip install fast-llm
```

## Usage

```bash
# Quantize + benchmark
fast-llm run Qwen/Qwen2.5-0.5B --quantize --benchmark

# Quantize only (faster, no benchmark)
fast-llm run microsoft/Phi-3-mini-4k-instruct --quantize

# Exact mode (same output, thread optimization only)
fast-llm run meta-llama/Llama-3.2-1B --exact --benchmark
```

Works with any HuggingFace causal language model:
Qwen, Llama 3, Phi-3, Mistral, Gemma, DeepSeek, and more.

## Requirements

- Python 3.8+
- PyTorch 2.0+ (CPU)
- 1–4 GB RAM per billion parameters

## License

MIT
