Metadata-Version: 2.4
Name: llamastudio
Version: 1.0.8
Summary: FastAPI backend & desktop launcher for running local models via llama.cpp
Author: LLamaStudio contributors
Project-URL: Homepage, https://github.com/gnulnx/LlamaStudio
Project-URL: Repository, https://github.com/gnulnx/LlamaStudio
Project-URL: Issues, https://github.com/gnulnx/LlamaStudio/issues
Keywords: llama.cpp,llm,local-ai,fastapi,gguf
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Console
Classifier: Framework :: FastAPI
Classifier: Intended Audience :: End Users/Desktop
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: fastapi>=0.115.0
Requires-Dist: httpx
Requires-Dist: jinja2
Requires-Dist: pydantic-settings
Requires-Dist: rich
Requires-Dist: rich-click
Requires-Dist: textual<9,>=8.2.8
Requires-Dist: uvicorn
Provides-Extra: dev
Requires-Dist: pre-commit; extra == "dev"
Requires-Dist: pytest; extra == "dev"
Requires-Dist: ruff; extra == "dev"

# 🦙 LLamaStudio

A desktop chat interface and local server manager for `llama.cpp`, crafted with **FastAPI** + **HTMX** for ultra-lightweight, zero-framework execution. 

**LLamaStudio** is a self-contained local workspace that manages model lifecycles, features a smart VRAM estimator, scans local folders, and lets you search and download models directly from the Hugging Face Hub.

---

## 📸 Screenshots & Showcase

### Terminal tour

[![Watch the LlamaStudio terminal tour](imgs/tui/demo.gif)](imgs/tui/demo.mp4)

[Watch the MP4](imgs/tui/demo.mp4): `lls tui`, then Discover, Models, a live chat,
and inference logs—all using an already loaded model.
[Regenerate the recording](#recording-the-demo) after changing the palette.

### 1. Main Chat Dashboard
A Pop!_OS-harmonized dark interface with streaming, collapsible markdown reasoning (thinking) processes, and real-time agentic tool execution logs.
![Main Chat Dashboard](imgs/chat_interface.png)

### 2. GGUF Model Browser & Settings
A dynamic local model explorer that scans your directories and lets you adjust context length, GPU offload layers, CPU threads, flash attention, and KV cache quantizations on the fly.
![GGUF Model Browser](imgs/model_settings.png)

### 3. Hugging Face Discover Hub
Browse the entire Hugging Face GGUF catalog. Features a **Smart VRAM Offload Estimator** calibrated to your hardware, and a floating background download progress card with live speed (MB/s), ETA, and cancel controls.
![Hugging Face Discover Hub](imgs/discover_models.png)

---

## ✨ Key Features

- **⚡ Zero Node Modules**: Built with HTMX, Tailwind CSS (via CDN), and Vanilla JS. It is incredibly fast, responsive, and has a memory footprint of just a few megabytes.
- **🧭 Hugging Face Discover Tab**: Search the public Hugging Face Hub for GGUF models directly inside the app, view readmes, select quantizations, and download files in the background.
- **🚀 Smart VRAM Estimator**: calulated specifically for your hardware (fits fully on **RTX 5090 32GB VRAM**, partial offload warning, or heavy CPU fallback warning).
- **📂 Automatic Model Scanning**: Scans standard directories (like `~/.lmstudio/models`) automatically on startup or via a one-click rescan button.
- **🪐 Process Lifecycle Manager**: The underlying `llama-server` process only spins up when you explicitly load a model, releasing all system resources and GPU VRAM instantly when you click "Eject".
- **🔧 Configurable Workspace Sandboxing**: Supports sandboxed agentic tool use (file read/write, commands, etc.) with real-time logs in the UI. Workspace and permission defaults are stored in the first-class app config.
- **👁️ Multimodal Media Chat**: Drag raster images or WAV/MP3/FLAC audio into chat, or attach workspace media with `lls oneshot --image` / `--audio`; FLAC is normalized to llama.cpp-compatible WAV with `ffmpeg`.
- **🎙️ Local Push-to-Talk**: Click the microphone once to record and again to stop. A managed `whisper.cpp` service transcribes locally and places editable text in the chat box; audio is never sent to the chat model.
- **🖥️ XDG-Compliant Persistence**: App config, conversations, and first-class model profiles are stored outside the codebase directory in standard `~/.config/llamastudio/` with automated backward-compatible migrations.
- **📦 Full Linux & macOS Portability**: Server binaries and model directories are resolved dynamically on startup.

---

## 🛠️ Installation & Setup

LLamaStudio is compatible with **Linux** and **macOS** out-of-the-box. Choose your OS and python virtual environment preference below.

### 🐧 1. Linux Installation

#### Prerequisites
1. **Python 3.10+** (Recommended: Python 3.13)
2. **llama.cpp** built from source (or pre-compiled binary):
   - By default, the app dynamically looks for the `llama-server` binary globally on your system PATH or locally inside your home directory at `~/llama.cpp/build/bin/llama-server`.

#### Environment Setup

##### Option A: Install from PyPI
```bash
pip install llamastudio
```

##### Option B: Conda / Miniconda from source
```bash
# 1. Clone the repository
git clone https://github.com/gnulnx/LlamaStudio.git
cd LlamaStudio

# 2. Create and activate a conda environment
conda create -n llamastudio python=3.13 -y
conda activate llamastudio

# 3. Install LlamaStudio and its dependencies
pip install -e .
```

##### Option C: Python Virtualenv (`venv`) from source
```bash
# 1. Clone the repository
git clone https://github.com/gnulnx/LlamaStudio.git
cd LlamaStudio

# 2. Create and activate a python venv environment
python3 -m venv .venv
source .venv/bin/activate

# 3. Install LlamaStudio and its dependencies
pip install -e .
```

#### 🖥️ Linux Desktop Launcher Integration (Optional)
To integrate LLamaStudio directly into your Linux Application launcher menu (e.g., GNOME / Pop!_OS):
```bash
# 1. Copy the desktop file to your local applications directory
cp llamastudio.desktop ~/.local/share/applications/

# 2. Copy the custom SVG icon to your local icons directory
mkdir -p ~/.local/share/icons/hicolor/128x128/apps/
cp llamastudio.svg ~/.local/share/icons/hicolor/128x128/apps/

# 3. Update your desktop database and icon cache
update-desktop-database ~/.local/share/applications/
gtk-update-icon-cache -f -t ~/.local/share/icons
```
*Note: If you are using a virtualenv, edit the executable path inside `~/.local/share/applications/llamastudio.desktop` to point to your specific `.venv/bin/python` interpreter.*

---

### 🍏 2. macOS Installation

#### Prerequisites
1. **Python 3.10+**
2. **llama.cpp** installed globally via Homebrew (highly recommended for macOS):
   ```bash
   brew install llama.cpp
   ```
   *(This automatically places the `llama-server` binary globally on your system PATH, which LLamaStudio will auto-detect immediately!)*

#### Environment Setup

##### Option A: Install from PyPI
```bash
pip install llamastudio
```

##### Option B: Conda / Miniconda from source
```bash
# 1. Clone the repository
git clone https://github.com/gnulnx/LlamaStudio.git
cd LlamaStudio

# 2. Create and activate environment
conda create -n llamastudio python=3.13 -y
conda activate llamastudio

# 3. Install LlamaStudio and its dependencies
pip install -e .
```

##### Option C: Python Virtualenv (`venv`) from source
```bash
# 1. Clone the repository
git clone https://github.com/gnulnx/LlamaStudio.git
cd LlamaStudio

# 2. Create and activate venv
python3 -m venv .venv
source .venv/bin/activate

# 3. Install LlamaStudio and its dependencies
pip install -e .
```

---

### 🪟 3. Windows Installation
*Note: Native Windows execution is currently **untested**.* 
However, you can run LLamaStudio on Windows seamlessly via **WSL2** (Windows Subsystem for Linux) by following the standard **Linux Installation** guide above.

Pull requests extending native Windows support (e.g., resolving `.exe` binaries) are highly welcome!

---

## 🚀 Running the Application

### Option A: Via Unified CLI (`lls` - Recommended)
You can link and install LlamaStudio's CLI utility locally to control the desktop app and server seamlessly:
```bash
# Start the desktop application server and open browser UI
lls start
```

### Option B: Via App Launcher Command
After installing from PyPI or source, run:
```bash
llamastudio
```

### Via Application Menu (Linux)
Search for **LLamaStudio** in your desktop search bar (press Super, type "Llama") and click to launch!

---

## 🛠️ Unified Command-Line Interface (`lls`)

LlamaStudio features a CLI built using `rich-click` for visual dashboards and operational efficiency. 

### CLI Subcommands Reference

| Command | Usage | Description |
| :--- | :--- | :--- |
| `start` | `lls start` | Starts the desktop app and opens the browser to the right first-run/chat/models/discover view. |
| `tui` | `lls tui [--view discover\|models\|chat\|logs]` | Interactive terminal interface with mouse, keyboard, and adaptive layouts. Starts the backend without opening a browser when needed. |
| `reload` | `lls reload` | Gracefully restarts the desktop FastAPI application backend. |
| `status` | `lls status` | Visual dashboard of FastAPI backend status, loaded model parameters, and GPU memory (VRAM). |
| `ls` | `lls ls` | Prints an elegant table of all GGUF models scanned across local directories. |
| `load` | `lls load [MODEL]` | Boots the server with a GGUF model. If `MODEL` is omitted, prompts you with an interactive menu. |
| `eject` | `lls eject` | Gracefully unloads the active model to free GPU and CPU RAM. |
| `oneshot`| `lls oneshot [--image PATH] [--audio PATH] [--no-thinking] [--max-tokens N] "prompt"` | Streams text, optional reasoning, tool calls, and multimodal workspace images/audio directly in your terminal. Use `--no-thinking` for low-latency direct answers. |
| `speech status` | `lls speech status` | Shows the local Whisper installation, selected model, compute mode, and server state. |
| `speech install` | `lls speech install [--model small.en]` | Installs pinned, checksum-verified `whisper.cpp` Linux binaries and a local Whisper model. |
| `speech load/eject` | `lls speech load [MODEL] [--gpu\|--cpu]` | Starts or stops the persistent speech-to-text server independently of the chat model. |
| `speech transcribe` | `lls speech transcribe AUDIO` | Transcribes a workspace audio file locally, with optional language and English translation controls. |
| `speech record` | `lls speech record [--device default]` | Starts terminal microphone capture immediately; press Enter to stop and print the transcript. |

Set up push-to-talk once, then use the microphone beside the chat input:

```bash
lls speech install --model small.en
lls speech status
lls speech record
```

The browser control is a toggle, not a hold action. The first click starts recording, the red stop button ends it, and the transcript is inserted without auto-sending so it can be corrected first. Browser microphone access requires the loopback URL (`http://127.0.0.1:8765`) or HTTPS.

For low-latency vision classification, disable reasoning and keep the answer budget small:

```bash
lls oneshot --no-thinking --temperature 0 --max-tokens 32 \
  --image camera-frame.png "Answer in 10 words or fewer: what is ahead?"
```

For audio transcription or translation with an audio-capable model and projector:

```bash
lls oneshot --no-thinking --audio recording.flac \
  "Transcribe this audio, translate it to English, and respond briefly."
```

For example, to boot a model interactively:
```bash
$ lls load
Available Scanned Models:
  1. Qwen3.6-35B-A3B-UD-Q5_K_M (25.2 GB)
  2. gemma-4-26B-A4B-it-Q8_0 (25.0 GB)
  3. DeepSeek-R1-Distill-Qwen-32B-Q5_K_M (21.7 GB)

Select a model number to load: 3
Loading model 'DeepSeek-R1-Distill-Qwen-32B-Q5_K_M'...
```

---

### Terminal interface

![LlamaStudio TUI Discover with live Hugging Face results](imgs/tui/discover.png)

```bash
lls tui
lls tui --view chat
lls tui --no-start     # Connect only; fail if the backend is offline
```

The TUI shares the web app's backend, model profiles, downloads, and saved
conversations. Closing it leaves the backend and loaded model running. Discover
supports Hugging Face search, sorting, README/details/files, GGUF selection,
download progress, and cancellation. Models provides filtering, rescan, saved
load settings, load/eject, and confirmed deletion. Chat supports multiline text,
saved conversations, streamed Markdown, collapsible reasoning, and tool activity.
Logs tails the application or inference server, with filtering and follow control.

The header adds a two-line wordmark, the primary GPU, live device-wide VRAM
usage, inference state (including CPU/GPU mode), and the active model. NVIDIA
memory usage refreshes every three seconds; unsupported telemetry is shown as
unavailable, not zero. Apple unified memory is labeled separately. Narrow or
short terminals use a two-line status summary instead of the full header.

Use F2–F5 to switch sections, Tab/Shift+Tab to move between controls, arrows and
Enter to select, and F1 for help. In chat, Enter inserts a newline and Ctrl+S sends.
Ctrl+R refreshes; Ctrl+Q quits. Every workflow also has mouse controls. In terminal
multiplexers, forward shortcuts to the application (for example, use zellij's
locked mode if its bindings intercept Ctrl+S). On macOS you may need Fn with
function keys, or use the navigation buttons.

Layouts adapt from a navigation rail and side-by-side panels to a navigation bar
and separate list/detail views below 110 columns. An 80×24 terminal is supported;
larger windows show more columns and model information. No Nerd Font, terminal
image extension, or graphics protocol is needed. Standard terminal colors are
used when true color is unavailable, and `NO_COLOR` is respected. For SSH, run
`lls tui` on the host running LlamaStudio, using `ssh -t` when launching directly.
The backend's filesystem and GPU are the ones shown in the TUI.

All TUI colors live in [app/tui/palette.json](app/tui/palette.json), shared by the
stylesheet, header, capability badges, model status, and logs. Edit that file and
press **Ctrl+P** in the TUI to reload it without restarting, refetching model lists,
or losing a chat draft. Purple marks actions/selections, green and teal mark
status/compatibility, gold marks capabilities/tool activity/warnings, and lilac
grays mark secondary text. Layout rules stay in `app/tui/studio.tcss`.

For a separate experiment (or an installed wheel), create a JSON file inside the
configured workspace containing just the roles you want to override, for example:

```json
{
  "primary": "#8b5cf6",
  "focus": "#9d6bff",
  "border": "#6d3df0"
}
```

Run `lls tui --palette colors.json`; Ctrl+P reloads that file. Colors must be
six-digit `#RRGGBB` values. Invalid edits keep the last working palette and show
an error. `--palette` also works with `--screenshot`. Terminal color capability
and `NO_COLOR` still apply; a palette cannot add true color to a terminal that
does not support it.

This first version handles text chat; media input remains in the web app. Split
GGUF shards are identified but not offered as individual model downloads: fetch
the complete set from the linked Hub repository. The Discover memory bar compares
GGUF weight bytes with reported GPU/unified memory, **not** guaranteed load capacity; context
cache and runtime allocations need additional memory. The backend still shares
one active conversation, so avoid sending simultaneously from the web app and TUI.

Capture the real interface for visual review without a terminal:

```bash
lls tui --screenshot .runtime/tui/discover.svg --size 190x52
lls tui --view models --screenshot .runtime/tui/models.svg --size 80x24
```

Screenshots must be new `.svg` files within the configured workspace. They contain
the live backend's data; use an appropriate conversation before sharing chat captures.

#### Recording the demo

From the checkout, with the backend running and a model already loaded:

```bash
lls status
lls demo-tui
# Optional palette experiment and separate output:
lls demo-tui --palette colors.json --output imgs/tui/experiment.mp4
```

The reproducible script is [app/tui/demo.tape](app/tui/demo.tape); edit its pauses,
typing, and navigation to adjust the tour. The command uses the current TUI and
palette—not canned screens—and produces `imgs/tui/demo.mp4`, a still
`imgs/tui/demo-poster.png`, and `imgs/tui/demo.gif`. The lightweight README
animation links to the full-quality MP4.

Install [VHS](https://github.com/charmbracelet/vhs#installation) (tested with
**0.11.0**), `ttyd`, FFmpeg (including `ffprobe`), Bash, and Chrome/Chromium.
The tape uses DejaVu Sans Mono. No recording dependencies are added to the app's
runtime requirements. Allow a few minutes for capture, encoding, and validation.

The tour opens an edge-to-edge terminal matching the TUI palette's background,
with three window dots and no outer frame. It types `lls tui`, browses all four sections,
and requests a short real reply. It waits for Hub results and chat completion;
network speed and the loaded model affect its duration and content. It never
loads, ejects, downloads, or changes a model profile. A temporary demo conversation
is created and deleted afterward; the previously active conversation is restored
unless you switched it elsewhere. Keep other clients idle during recording.

**Review before publishing:** real conversation titles, model names, filesystem
paths, and logs can appear in the video. This is a live capture, not an anonymizer.
The recorder enables true color only in its child environment, preserves shell
history, and exports 1920×1080 H.264/yuv420p with fast-start playback. The GIF is a
960×540, 3 fps, 96-color looping preview, kept below 10,000,000 bytes for
[GitHub's image limit](https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/attaching-files).
All outputs are staged; the MP4 passes metadata and full-decode checks and
the GIF passes a full-decode check before replacing the previous recording.
Changing the palette and rerunning the same command refreshes the demo without
hand-editing the video.

## ⚙️ Configuration & Customization

The application runs fully out-of-the-box with no manual configuration. On first launch, LlamaStudio creates its runtime config under:

```text
~/.config/llamastudio/
  config.json
  model_profiles.json
  conversations.json
  logs/
```

`config.json` stores app defaults, model search directories, workspace permissions, and launch state. `model_profiles.json` stores first-class per-model load and inference profiles. Older `model_settings.json` files are migrated automatically.

---

## 🛡️ Workspace Sandboxing & Embodiment

By default, LlamaStudio restricts agent tools (like reading, writing, and listing files) to the configured workspace directory to prevent accidental path traversals. For CLI launches, the first-run workspace defaults to the directory where `lls start` was run.

Workspace configuration is saved in `~/.config/llamastudio/config.json`. Environment variables are still supported for advanced/bootstrap overrides, but normal users should not need a `.env` file.

Developer details for the config/profile architecture live in [DEV.md](DEV.md).

---

## 🧪 Testing Suite

LlamaStudio features both standard unit tests and comprehensive GGUF integration tests.

### 1. Standard Unit Tests
Verify local installation and confirm backend routing, regex parsing, and sandboxing safety behaviors by running our mock-based test suite:
```bash
python -m unittest discover tests
```

### 2. GGUF Model Integration Tests
For local environments containing active GPUs and downloaded models, you can run the full multi-model GGUF tool-calling integration suite to verify real-time execution robustness across various chat templates:
```bash
# Run GGUF model integration tests locally
./tests/test_all.sh
```
*(These tests are automatically skipped in standard CI/CD environments and default `pytest` runs using `@pytest.mark.skipif` to keep pipeline checks fast.)*

### 3. Terminal UI tests

The default suite includes HTTP/SSE contract tests, CLI lifecycle tests, and
Textual Pilot tests exercising actual widgets with isolated backend responses:

```bash
python -m pytest tests/test_tui.py tests/test_tui_client.py tests/test_tui_cli.py -q
```

For a real model response, reasoning, tool activity, persistence, and screenshots:

```bash
lls status  # A model must already be loaded
RUN_TUI_LIVE=1 python -m pytest tests/test_tui_live.py -q -s
```

This opt-in test creates and removes only its own conversation, restores the
previous selection, and writes review SVGs under `.runtime/tui/`.

---

## 🏗️ Project Structure

```
LlamaStudio/
├── pyproject.toml         # Package metadata, CLI entrypoint, and dependencies
├── DEV.md                 # Development notes for runtime config and profiles
├── llamastudio.desktop    # GNOME/Linux desktop launcher metadata
├── llamastudio.svg        # Custom application vector icon
├── app/
│   ├── config.py          # Settings & dynamic path configurations
│   ├── config_store.py    # First-class runtime config and model profiles
│   ├── main.py            # FastAPI backend endpoints & routing
│   ├── chat.py            # Conversations registry, templates & chat streaming
│   ├── downloader.py      # Async background download manager (chunked writes)
│   ├── model_manager.py   # Scans local paths and Hugging Face Hub
│   ├── server_manager.py  # llama-server subprocess process lifecycle controller
│   ├── logger.py          # Centralized logger
│   ├── tools.py           # Sandboxed local workspace tools for LLM agent use
│   └── templates/
│       └── index.html     # Interactive HTMX frontend interface
├── tests/
│   └── *.py               # Unit and integration-adjacent test coverage
└── imgs/
    ├── chat_interface.png # Screenshot: Main Chat interface
    ├── model_settings.png # Screenshot: Model explorer & settings
    └── discover_models.png# Screenshot: HF Discover & Downloader panel
```

---

## 📄 License

LLamaStudio is open-source software licensed under the [MIT License](LICENSE).
