Set up hal0 — the local AI inference appliance

PrerequisitesRequirements

hal0 runs on any systemd Linux on x86_64. It self-detects your package manager and GPU/NPU backend.

Inference backends

New usersInstall hal0

The one-liner

# Fetches the signed release tarball, cosign-verifies it against the
# workflow OIDC identity, then hands off to install.sh.
curl -fsSL https://hal0.dev/install.sh | bash

From a clone (development / air-gapped)

# Standard install at /opt/hal0
sudo bash installer/install.sh

# Point model pulls at a larger disk
sudo bash installer/install.sh --models-dir=/mnt/ai-models

# Local-only dev install under $PWD/.hal0ai (editable, no systemd)
bash installer/install.sh --dev

# Set everything up but don't start the units yet
sudo bash installer/install.sh --no-start

Environment overrides

What the installer does

1. Pre-flight — Python 3.11–3.14, systemd, x86_64, disk, free ports, container-runtime smoke test. Accurate diagnostics on failure (see troubleshooting).
2. Layout — code at /usr/lib/hal0/hal0-<version> behind a current symlink + shared venv; config in /etc/hal0/; state in /var/lib/hal0/{models,registry,slots,cache}. --dev lands everything under $PWD/.hal0ai/.
3. Install — creates the venv, installs the release tree, links /usr/local/bin/hal0 + hal0-agent.
4. Dashboard UI — builds ui/dist if missing and npm is present (self-hosted fonts — renders identically offline).
5. Config defaults — writes hal0.toml, api.env, upstreams.toml. capabilities.toml ships empty by design → the first-run dashboard renders the bundle picker. Existing files are never clobbered.
6. systemd — installs + enables hal0-api + hal0-openwebui and starts them (unless --no-start). Per-slot hal0-slot@<name>.service units are managed by hal0 when slots load.
7. Hardware probe — sanity-checks GPU/Vulkan/ROCm/NPU device access.

New usersFirst run

1. Open the dashboard — browse to http://<host>:8080. On first load, capabilities.toml is empty, so the dashboard renders the bundle picker / setup.
2. Pick a tier — choose a hardware-anchored bundle (a curated model collection sized to your RAM) or start empty and add models yourself.
3. Verify health — hal0 doctor should report all-green; GET :8080/api/health returns 200. Services show active in the dashboard.

# Sanity from the CLI
hal0 doctor --json
hal0 slot list
curl -fsS http://127.0.0.1:8080/api/health

First model & first chat

# Pull a model into the catalog, create a slot, chat
hal0 model pull <model>
hal0 slot create my-slot --model <model> --provider llama-server
hal0 chat --slot my-slot

Existing usersUpgrade in place — no reinstall

Upgrading from an earlier hal0 is an idempotent, non-destructive re-run — not a fresh install. It converges units/services/config through the new code + one daemon-reload, heals prior failed slot units, and never clobbers your config.

# In place — re-run the installer (or `hal0 update`)
curl -fsSL https://hal0.dev/install.sh | bash
# hal0 update swaps the /current symlink atomically
sudo hal0 update

Migrations (run deliberately, at a window)

The current CLI ships several one-shot migrations. None run automatically on boot or update. Preview each plan, apply it deliberately, then verify the result.

The migrate CLI at a glance

hal0 migrate model-layout — end-to-end

Use this for a v0.1.x model store under /mnt/ai-models/. It builds the canonical symlink tree under /var/lib/hal0/models/.

Model files are never moved or copied. The command is dry-run by default and idempotent after a successful apply.

Standard usage — plan, apply, verify:

# 1. Plan the move. No files written. Read the action table.
sudo hal0 migrate model-layout
# → table of: action / link / target / source / reason
# → "planned by leaf" rollup
# → "--dry-run — would create N symlink(s); pass --apply to write."

# 2. Apply.
sudo hal0 migrate model-layout --apply
# → "applied N symlink(s) under /var/lib/hal0/models."
# Idempotent — re-running --apply after success is a no-op.

# 3. Verify. Exit 0 means current; exit 1 means links are still pending.
sudo hal0 doctor migrations
sudo hal0 doctor models
find /var/lib/hal0/models -type l

hal0 slot migrate-hw — end-to-end

This migration restores physical settings to slots: NGL, runner binary, and deliberate image pins. It also clears the folded model hardware columns.

The bare command is a dry-run. --apply rewrites slot TOMLs and the registry DB after creating a timestamped backup.

# 1. Preview the fold (no files written, no columns nulled).
sudo hal0 slot migrate-hw
# → per-slot list of: set_ngl · set_binary · set_image_pin · drops

# 2. Apply. Let the CLI stop active hal0 units.
sudo hal0 slot migrate-hw --apply --yes --stop-services
# → "backup written to /var/lib/hal0/backups/<timestamped>.tar.gz"
# → "Applied hardware-ownership fold: ..."
# → "Restart hal0-api (and daemon-reload) to pick up the slot HW grid."

sudo systemctl daemon-reload
sudo systemctl start hal0-api
hal0 doctor                          # expect all-green for slots

hal0 slot migrate-id-keying — end-to-end

This optional migration changes slot artefacts from mutable-name keys to stable IDs. It renames the TOML, state directory, systemd unit, and podman container.

Run it only after upgrading to the bilingual id-keying runtime. See the full runbook before applying.

# 1. Stop hal0. Even the preview refuses while these units are active.
sudo systemctl stop 'hal0-slot@*' hal0-api

# 2. Preview — prints the computed name→id plan, no file moves.
sudo hal0 slot migrate-id-keying --dry-run

# 3. Apply. The command backs up slots/ + hal0.db, then flips artefacts.
sudo hal0 slot migrate-id-keying --yes

sudo systemctl daemon-reload
sudo systemctl start hal0-api
hal0 slot list                       # confirm every slot appears once
systemctl list-units 'hal0-slot@*'

Honcho → Hindsight

Only boxes that previously ran Honcho need the memory migration. Run it before upgrading; Hindsight-only boxes are already current.

hal0 memory migrate --from honcho --to hindsight

Why your box keeps working before you migrate

ReleaseWhat's new — R5 / v1.0.0-alpha.0

R5 hardens install and runtime behavior. The current alpha adds workload-oriented profiles, slot-owned hardware, stable slot IDs, a first-class brain route, and clearer slot/model drawers.

Current Workload profiles & slot-owned hardware

• Seeded profiles are workload-oriented templates rather than hardware identity. Applying one stamps defaults; later model edits do not mutate the profile.
• Physical launch facts now live on each slot: device, NGL, threads, resolved runner binary, profile, and optional image_pin.
• Models remain logical and device-agnostic. The slot-purity update moves chat_template to the model and sunsets the old slot tier.
• hal0 slot migrate-hw folds prior model/profile hardware values back onto slots. It is dry-run-first, backup-first, and maintenance-window gated.
• Slot artefacts can be converted from names to stable IDs with hal0 slot migrate-id-keying; name-keyed boxes remain supported.

Dashboard drawers

• The slot drawer presents a typed hardware grid: Device, Image, Profile, Threads, and NGL.
• It shows the resolved runner binary, profile-default image placeholder, and actual running image reference with slot ID.
• Parallel and Extra Args now sit in the slot's Model section. Reasoning and MTP remain model-owned.
• The model drawer adds the typed Thinking capability (auto, on, or off) before MTP.

Memory, brain & Hermes

• Memory tools use the Hindsight surface: hindsight_recall, hindsight_retain, and hindsight_reflect. Honcho is removed.
• The first-class hal0-brain module now serves the primary POST /api/brain/chat steward route.
• Hermes persona and brain-profile registration run in the API boot lifespan, so fresh installs, updates, and dev starts share one path.
• hal0 agent install hermes --reset-personas restores canonical personas; --repair remains non-destructive.
• Hermes uninstall now stops its service before removal, and provisioning accepts both string and Path database paths.

Install & runtime hardening

• Container-runtime preflight now surfaces the real cause on failure (e.g. kernel keyring-quota exhaustion) instead of a misleading nesting/keyctl message.
• GPU preflight + renderer now derive the render group from the device node's owner GID (not the group name) — fixes silent GPU-access loss on hosts where renderD128's group is misnamed.
• Slot units emit StartLimit* in [Unit] so restart rate-limiting actually applies.
• Hermes gateway install runs as the hal0 user. Uninstall now stops the Hermes service before removing it.
• hal0 agent status --json emits real JSON; reconcile_listeners wired into /api/ports.
• Shared APU models standardized on Vulkan (benchmark-backed).

Housekeeping (SWEEP)

• Deprecated surfaces remain machine-stamped HAL0-SUNSET: v1.0.0 until their scheduled removal.
• Python and UI package manifests now report 1.0.0-alpha.0 consistently.
• Boot is split into 14 named, observable phases with a typed BootState.
• The bilingual slot runtime reads both name-keyed and id-keyed layouts; the operator chooses when to flip.

OperateTroubleshooting

unprivileged LXC Install refuses at the container-runtime preflight

Symptom: podman run fails with crun: create keyring: Disk quota exceeded. Cause: the host kernel keyring byte-quota (kernel.keys.maxbytes) is exhausted — usually leaked keyrings from repeated failed healthchecks. hal0 correctly refuses (a real slot couldn't start either).

# Clear it, then re-run install
# reboot the container  — or —
keyctl clear @s
# or raise the quota on the Proxmox / host kernel
sysctl -w kernel.keys.maxbytes=2000000

GPU Slot runs but on CPU / GPU access denied

If /dev/dri/renderD128 is owned by a group that isn't your host's render group, older builds group-added the wrong GID. v1.0.0-alpha.0 derives the GID from the device node itself. Confirm your hal0 user is in the device's owner group:

stat -c '%G %g' /dev/dri/renderD128
id -nG hal0

Hardened host: permissions look wrong after install

A CIS/STIG umask (0027/0077) leaking into the install can leave the tree 0700. The installer normalizes to 022 for its own body; if you see PermissionError reading /usr/lib/hal0 or /etc/hal0/slots, run hal0 doctor perms --fix.

ReferencePaths & commands