arc-llama

Local inference
Checking server

LOCAL INFERENCE

Models

Find a model, choose it from your library, and open chat.

Checking your setup

Waiting for server status.

System status

Waiting for server status

Find models on Hugging Face

Search Hugging Face to download a model. It will appear in Your models when added.

Memory fit is an estimate. Review the downloaded model before chatting.

Your models

Models on this computer. Choose one to review and chat.

Disk usage

Plugins

Installed add-ons discovered through arc_llama.plugins.

Measurements

Real request measurements from this session: cold starts, time to first token, generation speed, and queue waits. No numbers are shown until they were actually measured.

DASHBOARD

Customize plugin buttons

Choose where installed plugin actions appear. Use the arrows to reorder them.

INTEGRATION

Connect a frontend

Point a chat app at the arc-llama API running on this machine. Choose a frontend: every option is copy-only, nothing is changed for you.

  1. In Open WebUI, open Settings → Admin Panel → Connections and add an OpenAI connection.
  2. Paste this Base URL:
Detecting…

API Key: any non-empty string; arc-llama does not validate client keys.

Models then appear automatically in the Open WebUI model picker. arc-llama never signs in to or modifies your Open WebUI account; you add the connection yourself.

Prefer to type it yourself? The API listens on OpenAI paths under ….