Metadata-Version: 2.4
Name: lllm3090
Version: 0.1.0
Summary: Local LLM serving for a single RTX 3090, with a browser control panel
License-Expression: Apache-2.0
Project-URL: GitHub, https://github.com/gilesknap/lllm3090
Project-URL: Documentation, https://gilesknap.github.io/lllm3090
Classifier: Development Status :: 4 - Beta
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.11
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: fastapi
Requires-Dist: uvicorn[standard]
Requires-Dist: huggingface-hub>=0.34
Requires-Dist: pyyaml
Requires-Dist: typer
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Requires-Dist: httpx; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: sphinx-autobuild; extra == "dev"
Requires-Dist: sphinx-copybutton; extra == "dev"
Requires-Dist: sphinx-design; extra == "dev"
Requires-Dist: myst-parser; extra == "dev"
Requires-Dist: pydata-sphinx-theme; extra == "dev"
Requires-Dist: types-PyYAML; extra == "dev"
Dynamic: license-file

# lllm3090

Local LLM serving for a single RTX 3090, with a browser control panel.

A llama.cpp engine, a web UI on loopback that starts and stops it and downloads
models, and a curated model list where every entry has been checked to fit
24 GB with a usable context left over.

## Install

Debian 13 or a derivative (Ubuntu 24.04 / 26.04), an RTX 3090, and the NVIDIA
driver already working:

```bash
git clone https://github.com/gilesknap/lllm3090
cd lllm3090
./install.sh
```

The installer touches nothing outside `$HOME` except a handful of apt packages,
and downloads **no model weights** — you pick those from the panel.

Then open <http://127.0.0.1:8080>, download `Qwen3-8B` (5 GB) to prove the
install works, and `Qwen3.8-27B` (15 GB) for real use.

Just the Python package, without the engine or service:

```bash
pip install lllm3090
```

## Use

```bash
lllm3090 models          # what exists, what fits, what is downloaded
lllm3090 start Qwen3.8-27B
lllm3090 status
lllm3090 claude          # launch Claude Code against the local model
lllm3090 stop            # free the VRAM
```

The engine exposes both the OpenAI API (`/v1/chat/completions`) and Anthropic's
(`/v1/messages`) on `127.0.0.1:1919`, so Claude Code and OpenAI-compatible
clients both work against it without a translation proxy.

## Why it is scoped to one GPU

Every figure in the model catalogue — download size, resident VRAM, KV cache
cost per token, achievable context, expected tokens per second — is computed
for 24 GB of GDDR6X at compute capability 8.6. On another card the software
would still run and every number would be wrong, so the installer checks and
warns.

## Documentation

<https://gilesknap.github.io/lllm3090>
