Metadata-Version: 2.4
Name: Convlit
Version: 0.1.1
Summary: Chat with local GGUF models (no llama.cpp).
Author: Kavin
License: MIT
Project-URL: Homepage, https://pypi.org/project/convlit/
Keywords: gguf,llama,local,inference,pytorch
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: torch>=2.0
Requires-Dist: regex>=2023.0.0
Dynamic: license-file

# Convlit (GGUF Chat Runner — No llama.cpp)

Chats with a local `.gguf` model **without** using `llama.cpp` / `llama-cpp-python`.

This runner:

- Parses GGUF directly (mmap)
- Keeps the GGUF file mapped while running (so tensors can safely share the mmap)
- Dequantizes common GGML K-quants (`Q4_K`, `Q5_K`, `Q6_K`) and `IQ4_XS`
- Computes RoPE frequencies from GGUF metadata (`llama.rope.freq_base` / `llama.rope.dimension_count`)
- Runs a minimal Llama-3.x style model in PyTorch (CPU)

## Setup

### Install (PyPI)

```powershell
python -m pip install convlit
```

### Install (from source)

```powershell
python -m pip install -r requirements.txt
```

## Run

```powershell
convlit --model .\ai.gguf
```

In-chat commands:

- `/help`, `/reset`, `/system <text>`, `/save <path>`, `/exit`
