Get started with hal0 — the local AI inference appliance

OrientationThe shape of the platform

Four ideas explain almost everything in hal0: slots run things, models describe things, profiles tune things, and one control plane supervises it all.

Who owns which flag

At launch, hal0 layers three tiers into the final runner command. Knowing which tier owns a setting tells you where to edit it — and which migrator moves it (see migrations).

What's around the core

• Seeded capability slots — a fresh install seeds agent (the capable default/fallback anchor), utility (cheap helper for background jobs), brain, coder, embed, rerank, tts, img, and the NPU-backed flm trio. Capabilities are on-demand — enable/disable from the dashboard. → guides/manage-slots
• hal0-brain — the built-in platform steward behind the dashboard's Agent Chat button (POST /api/brain/chat). It reaches the whole 180-tool admin MCP catalog: create slots, pull models, run benchmarks, read health — with an operator-approval gate on destructive actions. → concepts/agents
• Hermes agents — installable autonomous agents (hal0 agent install hermes) with personas, driven through the same MCP surface. → guides/run-agents
• Hindsight memory — long-term memory service for agents and chat, exposed over the /mcp/memory mount (26 tools). → concepts/memory, guides/enable-memory
• OpenWebUI — prewired chat UI on :3001, pointed at the local API out of the box. → getting-started/first-chat

PrerequisitesRequirements

hal0 runs on any systemd Linux on x86_64. The installer self-detects your package manager, container runtime, and GPU/NPU backend — there is no minimum-spec gate; backend selection degrades to CPU rather than refusing to run.

Inference backends

Installing under Proxmox or WSL? Dedicated walkthroughs: Proxmox LXC (GPU/NPU passthrough) · bare metal Ubuntu · WSL (experimental).

New usersInstall hal0

The one-liner

# The bootstrap is the trust boundary: it fetches the signed release
# manifest, cosign-verifies the tarball, unpacks it, and hands off to
# the real installer (installer/install.sh).
curl -fsSL https://hal0.dev/install.sh | sudo bash

# Pick a release channel (default: stable)
HAL0_CHANNEL=preview curl -fsSL https://hal0.dev/install.sh | sudo bash

Common variations

# Point model pulls at a larger disk
curl -fsSL https://hal0.dev/install.sh | sudo bash -s -- --models-dir=/mnt/ai-models

# Set everything up but don't start the units yet
curl -fsSL https://hal0.dev/install.sh | sudo bash -s -- --no-start

# From a clone: local-only dev install under $PWD/.hal0ai (editable, no systemd)
bash installer/install.sh --dev

Environment overrides

What the installer does

Sixteen idempotent steps — safe to re-run after a partial failure, and existing config files are never clobbered. Condensed (full list: getting-started/install):

1. Pre-flight — architecture, systemd, Python, container runtime, disk, free ports. Accurate diagnostics on failure (see troubleshooting).
2. Layout + venv — code at /usr/lib/hal0/hal0-<version> behind a current symlink; the hal0 service user; config in /etc/hal0/; state in /var/lib/hal0/. Links /usr/local/bin/hal0. --dev lands everything under $PWD/.hal0ai/.
3. Dashboard UI — release tarballs ship a prebuilt ui/dist; source checkouts build it (Node auto-provisioned).
4. Config + units — writes hal0.toml, api.env, upstreams.toml, openwebui.env; installs + enables hal0-api and hal0-openwebui (unless --no-start).
5. Slot seeds — copies the curated slot TOMLs (agent, brain, coder, utility, embed, rerank, tts, img, flm, …) into /etc/hal0/slots/, never overwriting operator edits.
6. Hardware-anchored provisioning — probes CPU/RAM/GPU/NPU into /etc/hal0/hardware.json, seeds a hardware-recommended steward slot + profile, installs the FastFlowLM NPU runtime on AMDXDNA/apt hosts (fail-soft elsewhere), sets up the ComfyUI model share, writes the first-run bundle catalog, installs bundled agent skills, and starts the services.

New usersFirst run

There's no separate setup wizard — the install itself probes your hardware and provisions a coherent set of slots, models, and extensions in one pass. Your first run is a verification lap.

1. Open the dashboard — browse to http://<host>:8080. The Overview page shows the unified-memory map, throughput, and the seeded slot roster. A first-run bundle picker offers curated, RAM-sized model collections if you'd rather start from a preset.
2. Verify health — hal0 doctor all should report green across slots, memory, Hermes, services, and (new in 1.0) the MCP mounts; GET :8080/api/health returns 200.
3. Look around — the Slots page (sidebar → Slots) is mission control: per-slot cards with model select, live tok/s / TTFT / ctx, device + profile pills, and lifecycle buttons.

# Sanity from the CLI
hal0 doctor all --json
hal0 slot list
hal0 status
curl -fsS http://127.0.0.1:8080/api/health

First model & first chat

Models live in a local registry — pull one from Hugging Face, bind it to a slot, talk to it. (Deep dive: first-model · first-chat · choose-models.)

# See what's already registered, pull a starter model
hal0 model list
hal0 model pull qwen3-4b

# Bind it to the agent slot and confirm
hal0 slot edit agent --model qwen3-4b
hal0 status

# Chat — the slot NAME is the model id on the OpenAI surface
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"agent","messages":[{"role":"user","content":"hello!"}]}'

Prefer a UI? OpenWebUI is prewired on http://<host>:3001, already pointed at the local API. Or hit the dashboard's Agent Chat button and let the brain steward do the driving:

Existing usersUpgrade from beta — land cleanly on 1.0

Upgrading a 0.9.x beta box is an idempotent, in-place operation — not a reinstall. The swap is a cosign-verified, atomic symlink flip of /usr/lib/hal0/current; your config (/etc/hal0) and state (/var/lib/hal0) are untouched by the swap itself. Full mechanics: guides/update-and-rollback.

Which path is yours?

# Check what your channel offers, then apply
hal0 update --check
hal0 update            # stages, shows release notes + breaking/migration callouts, confirms, applies

# Switch channels first if needed (persists in hal0.toml, also runs a check)
hal0 update --channel stable

# Pin a specific version / escape the rc.1 visibility bug
hal0 update --target 1.0.0

Migrations — what's automatic, what's operator-run

v1.0 re-drew the ownership lines between slots, models, and profiles (see the diagram). One sweep runs automatically; the rest are deliberate, operator-run, dry-run-by-default one-shots for a maintenance window. Your box keeps working on the old shapes until you run them (see back-compat).

Automatic at first v1.0 boot

Operator-run, at a window

All four share the same shape: bare command = dry-run preview, --apply writes after a timestamped backup under /var/lib/hal0/backups/, --yes skips the prompt, and --stop-services stops hal0-api + active hal0-slot@* units first (without it, --apply refuses while they're active). Full CLI reference: reference/cli.

# The pattern, using migrate-flags as the example — same for -caps and -hw
sudo hal0 slot migrate-flags                 # 1. dry-run: read the per-slot plan
sudo hal0 slot migrate-flags --apply --yes --stop-services
# → "backup written to /var/lib/hal0/backups/<timestamp>.tar.gz"
sudo systemctl daemon-reload && sudo systemctl start hal0-api
hal0 doctor                                  # 3. verify: expect all-green

# Optional: flip to id-keyed slot artefacts
sudo hal0 slot migrate-id-keying             # dry-run plan (name → id)
sudo hal0 slot migrate-id-keying --apply --yes --stop-services
hal0 slot list && systemctl list-units 'hal0-slot@*'

Coming from much older boxes

Why your box keeps working before you migrate

Rollback

• hal0 update --rollback swaps current back to the previous tree (recorded in /var/lib/hal0/hal0.previous). It never touches config — the profiles.toml reseed and applied slot migrations are not undone by a code rollback.
• Release tarballs are immutable and cosign-signed — any prior tag can be re-installed from its GitHub release.
• Rolling back past the v1.0 boundary needs the config backup taken before upgrading: pre-v1.0 code reads a missing enabled as true, so slots the sweep cleared come back model-less rather than re-enabled.
• The slot migrators write timestamped backups under /var/lib/hal0/backups/ before changing anything — restoring one is the undo.

ReleaseWhat's new in 1.0

The 1.0 line is the sum of the R5 rework (rc.1), the feature-complete GA content (previewed by rc.2), and a 38-fix hardening pass (rc.3). The short version, by theme — the full story is in the CHANGELOG.

Ownership model finalized R5

• A slot is (id, name, model, port, state) plus a typed hardware grid; models carry the materialized tune in defaults; profiles are per-slot templates that stamp, not bind. The migrators in Migrations move existing boxes across.
• New slot lifecycle controls: autoload (binding a model no longer implies boot start) and eviction priority (0–100, default 50). The inert lru = true opt-in is retired — every non-pinned resident slot is now an eviction candidate under memory pressure; pinned is the only exemption.
• Stopping a slot is POST /api/slots/{name}/unload (409 slot.pinned on pinned slots) — config writes no longer have lifecycle side effects.
• On-demand capability slots: disabling a capability clears the slot's model (the pick survives in capabilities.toml); the retired vision lane and curated type-tag chips are gone in favour of typed model fields.

Agents & memory

• Admin MCP catalog: 92 → 180 tools — services, ComfyUI, updater/doctor/health, hardware + request telemetry, slots/models long-tail, bench, activity, approvals, runner images, NPU load/unload. The brain steward reaches all of it through POST /api/brain/chat.
• Memory MCP surface at feature parity with Hindsight 0.8.4: 26 tools (was 5) — reflect, curate/history, mental models, directives, async operations, bank introspection. The /mcp/memory mount dropped to CLIENT-tier: memory-only agents no longer need the platform-admin key.
• Hermes bootstrap MCP wiring fixed end-to-end (auth token injection, --repair refresh after key rotation); hal0 doctor all gained live MCP-mount and Hermes-auth preflights.

Voice, image, dashboard

• Moonshine reinstated as the CPU STT engine in its own toolbox image — voice.stt is a device-keyed switch: cpu → Moonshine, npu → whisper-v3:turbo (FLM trio), GPU → no STT engine (the old silent fall-through to a llama chat profile is fixed).
• Image Gen pane: state-typed engine pill (stopped/starting/running/generating·%/error) and a real Stop that drives the GPU arbiter back to inference mode.
• Slot drawer: runner image is a catalog dropdown (with a custom-ref escape hatch) that repopulates the binary dropdown; Auto-Load and Pinned pair in the header; profile rides the model choice.
• GET /api/slots latency cut sharply on wide boxes (single-flight snapshot, TTL-cached image inspects, no inspects for stopped slots).

Hardening (rc.3)

• The update path works end-to-end on real hosting: manifest fetch follows redirects, the privileged stage honours HAL0_RELEASES_URL, wrappers refresh on every activate.
• Id-keyed boxes are first-class in every lane (capability toggles, NPU trio, config writes).
• Memory surfaces tell the truth: real operations envelope, accurate bank-delete blast radius, per-agent stats scoping, ACL-aware pagination, extraction-slot changes actually reach the daemon.
• Removed: the experimental standalone browser MCP server (never mounted; Hermes brings its own browser tooling).
• [brain_chat] tool_model is wired for real: rc.1 deleted the key because nothing read it, and it came back with an implementation behind it. Tool rounds run on tool_model (default hal0/agent) while chat stays on the brain slot; set it to off, none or disabled to route tool turns nowhere.

OperateTroubleshooting

Known issues on the beta → 1.0 path

• 0.9.8's CLI reports a spurious error at the end of a successful update — it polls the API it's restarting and treats the mid-restart connection refusal as fatal. Verify the real outcome: hal0 --version + curl http://127.0.0.1:8080/api/health.
• The profile-catalog reset doesn't fire during the 0.9.8 → 1.0 update itself (#1585) — the commit phase runs inside the old daemon. Nothing is lost; the reset lands on the next update applied by v1.0 code.
• rc.1 boxes see "nothing to update" (#1663) — use hal0 update --target <version> (see upgrade paths).
• First-boot installs can lose the dpkg lock race to unattended-upgrades (#1584) — hermes provisioning degrades gracefully; re-run hal0 agent install hermes afterwards.

unprivileged LXC Install refuses at the container-runtime preflight

Symptom: podman run fails with crun: create keyring: Disk quota exceeded. Cause: the host kernel keyring byte-quota (kernel.keys.maxbytes) is exhausted — usually leaked keyrings from repeated failed healthchecks. hal0 correctly refuses (a real slot couldn't start either).

# Clear it, then re-run install
# reboot the container  — or —
keyctl clear @s
# or raise the quota on the Proxmox / host kernel
sysctl -w kernel.keys.maxbytes=2000000

GPU Slot runs but on CPU / GPU access denied

If /dev/dri/renderD128 is owned by a group that isn't your host's render group, older builds group-added the wrong GID. v1.0 derives the GID from the device node itself. Confirm your hal0 user is in the device's owner group:

stat -c '%G %g' /dev/dri/renderD128
id -nG hal0

Driver/firmware setup (incl. Proxmox passthrough): getting-started/drivers · getting-started/proxmox.

Hardened host: permissions look wrong after install

A CIS/STIG umask (0027/0077) leaking into the install can leave the tree 0700. The installer normalizes to 022 for its own body; if you see PermissionError reading /usr/lib/hal0 or /etc/hal0/slots, run hal0 doctor perms --fix.

ReferencePaths & commands

Environment variables

The install-time variables are in the Install section. These are the runtime ones you'll actually reach for — the full table (60+ vars, grouped by subsystem) is at reference/env-vars.