Metadata-Version: 2.4
Name: ai-launcher-cli
Version: 0.1.0
Summary: Simple CLI to chat with GGUF models locally (no Ollama/LM Studio required)
Author-email: ailaunch <user@example.com>
License: MIT
Keywords: gguf,llm,chat,cli,local-llm
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: llama-cpp-python>=0.3.0
Provides-Extra: cuda
Requires-Dist: llama-cpp-python[cuda]>=0.3.0; extra == "cuda"
Provides-Extra: metal
Requires-Dist: llama-cpp-python[metal]>=0.3.0; extra == "metal"

# GGUF Chat CLI

Simple Python CLI to chat with `.gguf` models locally using `ctransformers`. No Ollama, llama.cpp, LM Studio, or online providers required.

## Install

```bash
pip install -r requirements.txt
```

Or install the package directly:

```bash
pip install -e .
```

Then use the `ailaunch` command:

```bash
ailaunch path/to/model.gguf
```

## Usage

```bash
# Basic usage
python chat.py path/to/model.gguf

# Or installed as CLI
ailaunch path/to/model.gguf

# With custom settings
ailaunch model.gguf -c 8192 -t 0.8 --max-tokens 1024

# Disable streaming (wait for full response)
ailaunch model.gguf --no-stream

# Custom system prompt
ailaunch model.gguf --system "You are a coding assistant."

# Options:
#   -c, --ctx-size      Context window size (default: 4096)
#   -g, --gpu-layers    GPU layers to offload (-1 = all, default: -1)
#   -t, --threads       CPU threads (0 = auto, default: 0)
#   --temperature       Sampling temperature (default: 0.7)
#   --max-tokens        Max tokens to generate (default: 512)
#   --no-stream         Disable streaming output
#   --system            Custom system prompt
```

## GPU Acceleration

Install with GPU extras for acceleration:

```bash
# NVIDIA CUDA
pip install ctransformers[cuda]

# Apple Metal (M1/M2/M3)
pip install ctransformers[metal]
```

Then use `-g -1` to offload all layers to GPU.

## Requirements

- Python 3.8+
- `ctransformers` (installs automatically via pip)

## Exit

Type `exit`, `quit`, `q`, or press `Ctrl+C` to exit.
