arc-llama

Local inference
Checking server

LOCAL INFERENCE

Models

Select a local GGUF to open chat. Nothing starts until you send a message.

Readiness

Waiting for server status

Available models

Core details are visible here; launch settings stay out of the way until you need them.

Plugins

Installed add-ons discovered through arc_llama.plugins.

Measurements

Real request measurements from this session: cold starts, time to first token, generation speed, and queue waits. No numbers are shown until they were actually measured.

Selected model

Select a model to see its launch settings.

We will show its configured GPU, context, and memory capacity before you start.

Open chat

Select a model to continue.

Start chatting

DASHBOARD

Customize plugin buttons

Choose where installed plugin actions appear. Use the arrows to reorder them.

INTEGRATION

Connect a frontend

Point a chat app at the arc-llama API running on this machine. Choose a frontend: every option is copy-only, nothing is changed for you.

  1. In Open WebUI, open Settings → Admin Panel → Connections and add an OpenAI connection.
  2. Paste this Base URL:
Detecting…

API Key: any non-empty string; arc-llama does not validate client keys.

Models then appear automatically in the Open WebUI model picker. arc-llama never signs in to or modifies your Open WebUI account; you add the connection yourself.

Prefer to type it yourself? The API listens on OpenAI paths under ….