Metadata-Version: 2.4
Name: make-localllm-easier
Version: 0.7.1
Summary: One command runs the best local AI your PC can handle: measured model picks + a tuned llama.cpp (AMD, NVIDIA, Intel, Apple)
License: MIT
Project-URL: Homepage, https://github.com/phonology024/make-localllm-easier
Keywords: llm,local-llm,llama.cpp,vulkan,amd,gguf,thai,offline-ai
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Dynamic: license-file

# make-localllm-easier — run the best local LLM your GPU can handle, in one command

[![PyPI](https://img.shields.io/pypi/v/make-localllm-easier)](https://pypi.org/project/make-localllm-easier/)
[![tests](https://github.com/phonology024/make-localllm-easier/actions/workflows/tests.yml/badge.svg)](https://github.com/phonology024/make-localllm-easier/actions/workflows/tests.yml)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)

Read this in: English | [தமிழ்](docs/README.ta.md)

**`localllm` checks your GPU, picks the most accurate AI model that fits it *in your language*, downloads it and starts
a private chat. Offline and free.**

<!-- demo GIF goes here (#81): tools/record_demo.py records it from a real run -->

## Install

```
pip install make-localllm-easier
localllm
```

That's it: the first run shows what your PC can run and asks before it downloads the model. No Python yet? Single-file
downloads for Windows, macOS and Linux are coming with the next release ([packaging/README.md](packaging/README.md)).

## The commands you'll use

```
localllm            # pick, download and start the best model, open the chat page
localllm chat       # chat in the terminal, any language
localllm doctor     # what this GPU can run: model sizes, speed, RAM
localllm serve      # just the API: OpenAI, Anthropic, Ollama and Gemini formats at http://127.0.0.1:8080
```

More: `localllm list` (every measured model), `localllm connect hermes` (point an agent or Obsidian at it),
`localllm eval` (score any server in your language). All commands: `localllm --help`.

## Why it's different

- **It picks for your GPU.** It reads your card and RAM and only offers models that fit and run fast on them.
- **Measured in your language.** Models are ranked by real exam scores in Thai, Hindi, Arabic, Japanese and 20+ more
  languages, not by English benchmarks alone.
- **Uses less RAM.** llama.cpp tuned op by op: about 4x less system RAM than its defaults, same speed (below).
- **Private.** After the first download nothing leaves your PC; downloads are checked against pinned SHA-256s.

## Less RAM, same speed

Same 30-turn chat, RX 9070 XT, 32 GB RAM:

| model | llama.cpp defaults | `localllm` | speed |
|---|---|---|---|
| Qwen3.8-27B Q3 | 9.30 GB RAM | **2.37 GB** | 35.2 tok/s both |
| gemma-4-26B-A4B QAT | 9.24 GB RAM | **2.21 GB** | ~85 tok/s both |

## Learn more

[Website and leaderboard](https://phonology024.github.io/make-localllm-easier/) ·
[FAQ](docs/faq.md) ·
[How it works](docs/how-it-works.md) (doctor output, routing, RAM, where it's tested) ·
[APIs and agents](docs/apis.md) ·
[Benchmarks](docs/benchmarks.md) ·
[Research](docs/research.md) ·
[Roadmap](ROADMAP.md) ·
[Stability](docs/stability.md) ·
[Contributing](CONTRIBUTING.md)

Have a GPU we haven't measured? `localllm tune`, `localllm eval --langs en,<yours>`, then `localllm report` opens a
prefilled issue (no names, paths or keys) that adds it to the leaderboard.

## License

MIT. Models keep their own licenses; benchmark data keeps its own (Apache-2.0).
