Metadata-Version: 2.5
Name: localfit
Version: 2.0.0
Summary: Run AI models anywhere — local, Kaggle, or cloud — from one reliable CLI.
Project-URL: Homepage, https://github.com/AnassKartit/localfit
Project-URL: Repository, https://github.com/AnassKartit/localfit
Author: Anass Kartit
License-Expression: Apache-2.0
Keywords: benchmark,gguf,gpu,huggingface,llama.cpp,llm,llmrun,localfit,ollama,unsloth,vram
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Programming Language :: Python :: 3
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Topic :: System :: Hardware
Requires-Python: >=3.10
Requires-Dist: prompt-toolkit>=3.0
Requires-Dist: pyyaml>=6.0
Requires-Dist: rich>=13.0
Provides-Extra: all
Requires-Dist: huggingface-hub>=0.20; extra == 'all'
Requires-Dist: rumps>=0.4; extra == 'all'
Requires-Dist: textual>=1.0; extra == 'all'
Provides-Extra: app
Requires-Dist: py2app>=0.28; extra == 'app'
Requires-Dist: rumps>=0.4; extra == 'app'
Provides-Extra: download
Requires-Dist: huggingface-hub>=0.20; extra == 'download'
Provides-Extra: tui
Requires-Dist: textual>=1.0; extra == 'tui'
Description-Content-Type: text/markdown

# localfit

`One command for local, Kaggle, and cloud inference — and an honest answer to "will it fit?"`

[![PyPI](https://img.shields.io/pypi/v/localfit)](https://pypi.org/project/localfit/)
[![Python](https://img.shields.io/pypi/pyversions/localfit)](https://pypi.org/project/localfit/)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](LICENSE)

```bash
pip install localfit
```

Both `localfit` and `llmrun` are installed as commands and do the same thing.

**Run any model, wherever it fits.** Fits locally? Run it. Doesn't fit? Free Kaggle
GPU. Still too big? RunPod. One command downloads the model, starts the server,
configures your tools and launches the UI.

## Will it fit?

The question every local-LLM guide gets wrong. On Apple Silicon your Mac's RAM is
**not** what the GPU gets — Metal hands it roughly **74 %**:

| your Mac | what Metal actually gives the GPU |
|---|---|
| 16 GB | 11.8 GiB |
| 24 GB | **17.8 GiB** |
| 36 GB | 26.6 GiB |
| 48 GB | 35.5 GiB |

Check your own machine: `ioreg -l | grep recommendedMaxWorkingSetSize`, divide by 2³⁰.

A 19.5 GB model on a 24 GB Mac is **110 % of budget**. It does not fail politely —
it kernel-panics the machine. localfit scores every quant in a repo against the
budget that actually exists:

```bash
localfit qwen3.8-27b                 # score against THIS machine
localfit qwen3.8-27b --gpu-gb 12     # score against a 12 GB card
localfit qwen3.8-27b --gpu-gb 17.8   # ...or someone else's 24 GB Mac
```

`--gpu-gb` is how you answer the question for hardware you do not own.

> Vision projectors (`mmproj-*`, `*-vision-*.gguf`) are excluded from quant
> scoring — unfiltered, a 0.9 GB projector parses as an F16 quant, scores
> quality 100, and beats every real quant in the repo.

---


## One Command — Everything Works

```bash
# Chat + image gen in Open WebUI (gemma4 LLM + Flux Klein 4B images)
llmrun launch openwebui --model gemma4:e4b --img klein-4b

# Code with image gen in localcoder
llmrun launch localcoder --model gemma4:e4b --img klein-4b

# Claude Code with image MCP tools
llmrun launch claude --model gemma4:e4b --img klein-4b

# Model doesn't fit locally? Run on free Kaggle GPU
llmrun launch openwebui --model gemma4:26b --remote kaggle

# Need more power? RunPod cloud ($0.22/hr)
llmrun launch openwebui --model gemma4:26b --img klein-9b --remote runpod --budget 30m
```

llmrun auto-detects your hardware, downloads models, installs Ollama if needed, starts the image server, configures Open WebUI with image generation enabled, and opens your browser. Zero manual config.

## The Run Menu

```bash
llmrun run gemma4:e4b
```

Interactive menu with **arrow key navigation** — pick your backend before anything downloads:

```
╭──────────────────────── gemma-4-E4B ────────────────────────╮
│   LOCAL                                                     │
│   › MLX   gemma-4-E4B-4bit                      3.5GB      │
│     GGUF  Q4_K_M                                4.4GB      │
│     GGUF  Q8_0                                  7.5GB      │
│                                                             │
│   REMOTE                                                    │
│     Kaggle  T4 16GB                               free      │
│     RunPod  RTX 3090 24GB                     $0.22/hr      │
╰──────────────────────── Apple Silicon 24GB ─────────────────╯
```

## Image Generation

One server, two APIs — works with Open WebUI, Claude Code, and any OpenAI-compatible client.

```bash
# Starts automatically with --img flag, or manually:
python -m llmrun.image_server 8189 klein-4b 4
```

The server implements OpenAI-compatible `POST /v1/images/generations` and
multipart `POST /v1/images/edits`. llmrun configures both Open WebUI engines
automatically when `--img` is present.

### Supported Image Models

| Model | Pipeline | Params | Local Mac | Kaggle T4 | RunPod 3090 |
|-------|----------|--------|:---------:|:---------:|:-----------:|
| FLUX.2 Klein 4B | Flux2KleinPipeline | 4B | **24s** | 65s | **2s** |
| FLUX.2 Klein 9B | Flux2KleinPipeline | 9B | mflux | cpu_offload | 3090+ |
| FLUX.1 Schnell | FluxPipeline | 12B | 79s | cpu_offload | 3090 |
| FLUX.1 Dev | FluxPipeline | 12B | mflux | cpu_offload | 3090+ |
| Z-Image-Turbo | ZImagePipeline | 6B | 90s | T4 | 3090 |
| Qwen-Image | QwenImagePipeline | 20B | — | — | A6000+ |
| Qwen-Image-Edit | QwenImageEditPlusPipeline | 20B | — | — | A100 |
| SDXL | StableDiffusionXLPipeline | 6.6B | diffusers | T4 | 3090 |
| SD 3.5 Large | StableDiffusion3Pipeline | 8B | diffusers | T4 | 3090 |

All models auto-detected via `DiffusionPipeline.from_pretrained()`.

### MCP Server for Claude Code

```bash
claude mcp add llmrun-image --transport stdio -- python3 -m llmrun.mcp_image
```

6 tools available:

| Tool | Description |
|------|-------------|
| `check_resources` | GPU/VRAM info, loaded model, ETA estimates |
| `generate_image` | Text-to-image with timing + Image object |
| `edit_image` | Image-to-image editing |
| `show_image` | Display image in terminal via timg |
| `list_image_models` | Available models |
| `image_server_status` | Server health |

### Benchmarks (Verified)

| Platform | GPU | Model | Size | Steps | Time |
|----------|-----|-------|------|-------|------|
| **Mac** | M4 Pro | Klein 4B | 1024x1024 | 4 | **24s** |
| **Mac** | M4 Pro | Schnell | 512x512 | 4 | **79s** |
| **Kaggle** | T4 16GB | Klein 4B | 512x512 | 4 | **65s** |
| **RunPod** | RTX 3090 | Klein 4B | 512x512 | 4 | **2s** |

## LLM Backends

### Ollama (Default)

```bash
llmrun run gemma4:e4b               # auto-installs Ollama, pulls model, serves
```

### MLX (Apple Silicon)

```bash
pip install mlx-lm
llmrun run gemma-3-4b-it            # auto-picks mlx-community model
```

### GGUF (llama.cpp)

```bash
llmrun run gemma4:26b               # downloads best GGUF quant for your GPU
```

## Remote Serving

### Kaggle (Free — 30h/week GPU)

```bash
llmrun login kaggle
llmrun run gemma4:e4b --remote kaggle
```

### RunPod (Paid — any GPU)

```bash
llmrun login runpod
llmrun run gemma4:26b --cloud --budget $2
```

### Combined Remote Chat + Diffusers

```bash
llmrun launch openwebui \
  --model HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive \
  --quant Q8_K_P \
  --img Qwen/Qwen-Image-Edit-2511 \
  --lora ScottzillaSystems/qwen-image-edit-plus-nsfw-lora \
  --remote runpod \
  --budget 30m
```

Text and image workloads use separate right-sized RunPod pods and separate state
files. Each pod receives the same absolute provider-side deadline, an in-pod
deadline guard, and the scheduled host reaper. Open WebUI is launched only after
both OpenAI-compatible endpoints pass health checks. `llmrun --stop` discovers
and terminates every managed pod, including sessions whose local state vanished.

This is full-weight Diffusers loading; no image quantization is silently applied.
GPU cost is shown and approved separately for each pod.

### ComfyUI on Kaggle or RunPod

```bash
# Current official ComfyUI + Manager on a free Kaggle T4, auto-stop after 15m
llmrun comfyui --remote kaggle --duration 15

# Large video/audio profile on RunPod; GPU and exact maximum cost are confirmed
llmrun comfyui MiniMaxAI/MiniMax-H3 --cloud --budget 30m
```

ComfyUI is a separate visual workflow runtime, not an OpenAI image endpoint.
RunPod ComfyUI pods use the same provider `terminateAfter`, in-container guard,
five-minute launchd reaper, and provider orphan discovery as text and Diffusers
pods. Kaggle uses current official ComfyUI and its built-in Manager instead of
the obsolete PyTorch/ComfyUI pins in older community notebooks.

MiniMax H3 is video plus native stereo audio, not a chat or still-image model.
It cannot fit Kaggle's API-addressable 16GB GPU and is routed to an 80GB RunPod
profile. Because H3 is a multi-file workflow, llmrun starts ComfyUI and leaves
asset placement to its maintained H3 template/Manager instead of guessing paths.

## Rebrand and compatibility

`llmrun` is the primary package, executable, UI name, and documentation name as
of 2.0. The deprecated `localfit` executable and Python implementation package
remain during the migration. Existing `~/.localfit` state, RunPod environment
metadata, ntfy topics, and tool-provider IDs are intentionally retained so old
pods and configurations remain recoverable. Do not delete that directory while
a legacy session may still exist.

## Launch Any Tool

| Tool | Command |
|------|---------|
| Open WebUI | `llmrun launch openwebui --model MODEL --img IMAGE` |
| Claude Code | `llmrun launch claude --model MODEL --img IMAGE` |
| localcoder | `llmrun launch localcoder --model MODEL --img IMAGE` |
| OpenAI Codex | `llmrun launch codex --model MODEL` |
| OpenCode | `llmrun launch opencode --model MODEL` |
| aider | `llmrun launch aider --model MODEL` |

All tools are auto-configured with the right endpoints. Env vars scoped to subprocess — your normal setup is never touched.

## Make It Fit — Remote Quantization

```bash
llmrun makeitfit Qwen2.5-7B-Instruct
# → Quantizes on Kaggle (free) or RunPod
# → Uploads to your HuggingFace repo
# → Run it: llmrun run yourname/model-Q4_K_M-GGUF-llmrun
```

## All Commands

```bash
# Model management
llmrun run MODEL                    # interactive menu → serve
llmrun run MODEL --remote kaggle    # serve on free Kaggle GPU
llmrun run MODEL --cloud            # serve on RunPod
llmrun run MODEL --img IMAGE_MODEL  # serve LLM + image model
llmrun show MODEL                   # quants + fit analysis + pricing
llmrun list                         # installed models
llmrun stop                         # stop servers

# Image generation
llmrun launch openwebui --img klein-4b    # with Open WebUI
llmrun launch claude --img klein-4b       # with Claude Code MCP

# Tools
llmrun launch TOOL --model MODEL    # serve + launch tool
llmrun doctor                       # check all tool configs

# Hardware
llmrun                              # GPU dashboard + trending models
llmrun health                       # VRAM, temp, processes
llmrun bench                        # benchmark models
llmrun simulate                     # "will this model fit?"

# Cloud
llmrun login kaggle                 # save credentials
llmrun login runpod                 # save API key
llmrun login huggingface            # save HF token
llmrun comfyui --remote kaggle --duration 15
llmrun comfyui MODEL --cloud --budget 30m
llmrun --remote-status              # check Kaggle session
llmrun --remote-stop                # stop remote session

# System
llmrun check                        # check prerequisites
llmrun cleanup                      # free GPU memory
llmrun makeitfit MODEL              # quantize remotely
```

## Supported Platforms

| Platform | GPU | LLM | Image Gen |
|----------|-----|-----|-----------|
| macOS Apple Silicon | Metal | MLX + llama.cpp + Ollama | mflux (MLX native) |
| Linux NVIDIA | CUDA | llama.cpp + Ollama | diffusers (CUDA) |
| Linux AMD | ROCm | llama.cpp + Ollama | diffusers (ROCm) |
| Windows (WSL2) | CUDA | llama.cpp + Ollama | diffusers (CUDA) |
| Kaggle (free) | T4 16GB | Ollama via tunnel | diffusers via tunnel |
| RunPod (paid) | Any GPU | Ollama via tunnel | diffusers via tunnel |

## Requirements

- Python 3.10+
- Optional: [mflux](https://github.com/filipstrand/mflux) for Mac image gen
- Optional: [Ollama](https://ollama.com) (auto-installed by llmrun)

```bash
pip install llmrun                  # core
pip install mflux                     # + image generation (Mac)
```

## License

Apache-2.0
