#!/bin/bash
# hal0-systemctl — privileged seam for hal0-api's slot/systemd ops (P3-perms).
#
# Background: hal0-api now runs as the unprivileged `hal0` system user
# (installer/install.sh's hal0-api.service ships User=hal0 — the OwnershipStore
# flip in src/hal0/install/perms.py). It still needs a handful of genuinely-root
# operations for slot lifecycle: writing the per-slot systemd unit under
# /etc/systemd/system, `daemon-reload`, and start/stop/restart/enable/disable/
# reset-failed of a slot unit — plus restarting itself on self-update. This
# script is the ENTIRE privileged surface for those ops, modeled exactly on
# hal0-agentenv / hal0-benchctl: every argument is validated (a slot id must
# match a strict identifier regex), no shell is ever evaluated, and no
# wildcards are accepted.
#
# AGENT UNITS (#453 follow-up): the seam was originally slot-only, so
# HermesDriver._stop_services() had nothing to call and shelled out to a BARE
# `systemctl stop hal0-agent@hermes.service`. Unprivileged systemctl on a
# system unit escalates through polkit — i.e. an interactive password dialog
# in the middle of an uninstall (and, when cancelled, a unit that was never
# actually stopped). stop-agent/disable-agent close that gap: same posture as
# the slot verbs — a dedicated validator (validate_agent_id), the unit name is
# built HERE from a validated id (the caller never supplies a unit string),
# the systemctl verb is a literal, no shell, no wildcards.
#
# DROP-IN CONTENT (#1716): the two drop-in verbs take their whole payload on
# stdin, and stdin is attacker-controlled at this privilege boundary — the
# sudoers grant lets the unprivileged `hal0` account run this script as root.
# Copying that payload verbatim into a root-owned systemd fragment is a direct
# path to root: a `[Service]` body with `User=root` + a replaced `ExecStart=`,
# followed by this same wrapper's `daemon-reload` and `svc-restart` verbs, runs
# an arbitrary command as root. Pinning the destination path (already true for
# both verbs) does not constrain the directives written there. So the body is
# ALLOW-LISTED here, on the root side of the boundary — see
# validate_dropin_body below. The unprivileged caller's own templating is a
# convenience, never a control.
#
# QUADLET CONTENT (#1740): the same hole was still open one case arm below.
# `write-quadlet` installed its whole stdin verbatim as a root-owned
# /etc/containers/systemd/hal0-slot@<id>.container. A .container file is NOT
# just a container spec: podman's generator copies its `[Unit]`, `[Service]`
# and `[Install]` sections through VERBATIM into the generated system unit, so
# a `[Service]` with `ExecStartPre=/bin/sh -c '…'` plus this wrapper's own
# `daemon-reload` + `start` verbs was an unconditional root exec — no container
# involved. `write-unit` (a raw /etc/systemd/system/*.service write) was the
# same hole with no validation possible at all; it had no producer left after
# P3-quadlet and is DELETED rather than guessed at. What remains is
# validate_quadlet_body below: an allow-list of exactly the sections, keys and
# value shapes _render_quadlet_from_plan (src/hal0/providers/container.py)
# emits, and — as with the drop-ins — the validated RECONSTRUCTION is what is
# written, never raw stdin.
#
# HONEST BOUNDARY (read this before calling the seam a sandbox): the allow-list
# removes the DIRECT host-side exec primitive — no host `[Service]`/`[Unit]`
# directive beyond the five/three fixed ones, no `Exec*=` on the host side, no
# `User=`/`Group=`, no second section, no `[Install] WantedBy=` other than
# hal0.target. `PodmanArgs=` IS host-side too — quadlet copies it into the
# generated unit's `podman run` argv — so it is pinned to exactly the flags
# hal0's providers emit (`--group-add`/`--security-opt`/`--ipc`/`--ulimit`); a
# `--runtime`/`--hooks-dir` root exec is refused (#1759, validate_podman_args).
# It does NOT make an
# hal0-account compromise non-root-equivalent: `[Container]` still legitimately
# carries `Image=`, `Volume=`, `AddDevice=` and in-container `Exec=`, whose
# values are operator/config-derived and cannot be pinned here (the model-store
# and data roots are runtime config, and a path allow-list would be bypassable
# via a symlink under a root the hal0 user already owns). Slots run under ROOTFUL
# podman, so anyone who can author a slot's container spec can mount host paths
# into a container they control. That containment is "run slots rootless / pin
# mount roots", tracked separately. Claim only what is true: this seam is a
# *syntactic* boundary on the unit file, and a hard boundary against direct host
# exec — including the PodmanArgs host-exec flags, now that they are pinned.
#
# NOTE: the iptables FORWARD-chain repair the original P3-perms design doc
# sketched for this seam is NOT needed — that repair already runs as its own
# independent root oneshot unit (packaging/systemd/hal0-podman-forward.service,
# no User= — always root), entirely unaffected by hal0-api's User=hal0 flip.
#
# STALE-DNAT REPAIR (#1814): `prune-dnat` is the one verb here that touches the
# firewall. It exists because a container that dies without netavark's teardown
# running leaves its DNAT rule behind in inet netavark / nv_<netid>_dnat, and
# nftables is first-match — so that dead rule permanently black-holes the port
# for every later container. Deleting the handle is root-only, so it cannot be
# a raw `nft` subprocess from the unprivileged API/CLI. It is NOT a general
# "run nft" primitive: see validate_prunable_dnat below. The caller supplies a
# port and a handle (both numeric), and the ROOT side re-derives everything
# that matters — which chain the handle lives in, that the rule at that handle
# really is a DNAT rule for that port, and that its target IP belongs to NO
# running container. A handle naming a live container's rule, a rule in any
# chain outside nv_*_dnat, or anything that is not a dport/dnat rule is
# refused. The reachable effect is therefore bounded to "delete a netavark
# DNAT rule that already routes nowhere".

set -euo pipefail

UNIT_PREFIX="hal0-slot@"
UNIT_DIR="/etc/systemd/system"
SELF_UNIT="hal0-api.service"

# Bundled-agent template unit (hal0-agent@<id>.service). Separate prefix +
# separate validator from the slot family on purpose: the two id namespaces are
# unrelated, and a single shared "id" concept would let a slot verb reach an
# agent unit (or vice versa) if either regex is ever loosened.
AGENT_UNIT_PREFIX="hal0-agent@"

# P3-quadlet: per-slot Podman Quadlet source files live here, root-owned by
# design; podman's systemd generator turns hal0-slot@<id>.container into
# hal0-slot@<id>.service on daemon-reload.
QUADLET_DIR="/etc/containers/systemd"

# Fixed, literal drop-in path for the hermes gateway secrets fragment. Written
# by the hermes provisioner when it runs unprivileged (as the hal0 user) and so
# cannot write /etc/systemd/system itself. LITERAL — no variable, no traversal:
# the caller supplies only the body on stdin, never the path.
GATEWAY_DROPIN_DIR="/etc/systemd/system/hermes-gateway.service.d"
GATEWAY_DROPIN_FILE="${GATEWAY_DROPIN_DIR}/10-hal0-secrets.conf"

# Fixed, literal drop-in path for the hindsight-api extraction override (#1641 /
# ADR-0023): HINDSIGHT_API_LLM_MODEL + HINDSIGHT_API_LLM_TIMEOUT, repointed by
# `hal0 memory graph enable --slot <name>` and the Memory dashboard. Same posture
# and same reason as the gateway drop-in above: hal0-api runs as the
# unprivileged hal0 user and cannot write /etc/systemd/system itself. LITERAL —
# the caller supplies only the body on stdin, never the path.
HINDSIGHT_DROPIN_DIR="/etc/systemd/system/hindsight-api.service.d"
HINDSIGHT_DROPIN_FILE="${HINDSIGHT_DROPIN_DIR}/extraction-model.conf"

die() { echo "hal0-systemctl: $1" >&2; exit 64; }

validate_slot_id() {   # arg: slot id (the "<id>" in hal0-slot@<id>.service)
  local id="$1"
  [[ -n "$id" ]] || die "missing slot id"
  [[ "$id" =~ ^[A-Za-z0-9_-]{1,64}$ ]] || die "bad slot id: $id"
}

validate_agent_id() {  # arg: agent id (the "<id>" in hal0-agent@<id>.service)
  # Deliberately as strict as validate_slot_id: a bounded identifier charset
  # with NO '.', '/', '@' or whitespace, so a validated id can never carry a
  # second unit name, a path traversal, an extra systemd instance spec, or an
  # option-looking token ('-x' is a legal id but is only ever concatenated into
  # ${AGENT_UNIT_PREFIX}${id}.service, never passed as a bare argv word).
  local id="$1"
  [[ -n "$id" ]] || die "missing agent id"
  [[ "$id" =~ ^[A-Za-z0-9_-]{1,64}$ ]] || die "bad agent id: $id"
}

# ── drop-in content allow-list (#1716) ─────────────────────────────────────
#
# Root-side validation for the `write-*-dropin` verbs. Deliberately in the same
# style as the id validators above: strict regexes, a closed set of accepted
# tokens, no shell evaluation of anything read from stdin, and a loud `die` on
# the first thing that is not explicitly permitted.
#
# The grammar accepted is a small subset of systemd unit syntax:
#   * `#` comment lines and blank lines,
#   * exactly one `[Service]` section header, before any directive,
#   * `Key=Value` directives whose Key is in the per-verb allow-list below and
#     whose Value matches that key's regex.
# Everything else — any other section, any other directive (`ExecStart`,
# `ExecStartPre`, `User`, `Group`, …), leading whitespace, tabs, `;` comments,
# CR, a trailing backslash (systemd line continuation, which would let a second
# logical directive hide on a "value" line), any control byte — is rejected.
#
# On success DROPIN_BODY holds the RECONSTRUCTION of the validated lines, and
# that — not the raw stdin — is what the write arms persist. What was checked
# is therefore byte-for-byte what lands in /etc.

DROPIN_BODY=""

_DROPIN_MAX_LINES=200
_DROPIN_MAX_LINE=512

validate_dropin_directive() {  # args: kind, "Key=Value" line
  local kind="$1" line="$2" key value ename evalue
  [[ "$line" == *=* ]] || die "drop-in rejected: not a Key=Value directive: $line"
  key="${line%%=*}"
  value="${line#*=}"
  # No spaces, no tabs, no quoting around the key — a systemd-legal `Key = v`
  # or ` Key=v` is rejected rather than normalised, so there is exactly one
  # accepted spelling per directive.
  [[ "$key" =~ ^[A-Za-z]+$ ]] || die "drop-in rejected: bad directive key: $key"

  case "$kind" in
    hindsight)
      # ADR-0023: the extraction drop-in sets two Environment= assignments and
      # nothing else. `Environment=` takes `NAME=VALUE`, so the NAME is checked
      # against its own closed list; the value charsets exclude whitespace, so
      # a second `NAME=VALUE` pair cannot ride along on one line.
      [[ "$key" == "Environment" ]] || die "drop-in rejected: directive not allowed here: $key"
      [[ "$value" == *=* ]] || die "drop-in rejected: Environment= needs NAME=VALUE: $value"
      ename="${value%%=*}"
      evalue="${value#*=}"
      case "$ename" in
        HINDSIGHT_API_LLM_MODEL)
          # Always the `hal0/<slot>` virtual (never a raw model id), and a slot
          # id is the same bounded identifier the slot verbs accept.
          [[ "$evalue" =~ ^hal0/[A-Za-z0-9_.-]{1,64}$ ]] \
            || die "drop-in rejected: bad HINDSIGHT_API_LLM_MODEL value: $evalue"
          ;;
        HINDSIGHT_API_LLM_TIMEOUT)
          [[ "$evalue" =~ ^[0-9]{1,9}$ ]] \
            || die "drop-in rejected: bad HINDSIGHT_API_LLM_TIMEOUT value: $evalue"
          ;;
        *) die "drop-in rejected: environment name not allowed here: $ename" ;;
      esac
      ;;
    gateway)
      # #437: the gateway drop-in only points systemd at the hal0 secrets
      # vault. The value is confined to that directory (an EnvironmentFile
      # anywhere else would let a compromised caller inject env — LD_PRELOAD —
      # into a root unit); a leading `-` (optional-file marker) is allowed.
      [[ "$key" == "EnvironmentFile" ]] || die "drop-in rejected: directive not allowed here: $key"
      [[ "$value" =~ ^-?/var/lib/hal0/secrets/[A-Za-z0-9_./-]{1,128}$ ]] \
        || die "drop-in rejected: EnvironmentFile outside the hal0 secrets vault: $value"
      [[ "$value" != *".."* ]] || die "drop-in rejected: EnvironmentFile traversal: $value"
      ;;
    *) die "unknown drop-in kind: $kind" ;;
  esac
}

validate_dropin_body() {  # arg: kind (gateway|hindsight); body on stdin
  local kind="$1"
  local LC_ALL=C
  local line out="" seen_service=0 directives=0
  local -a lines=()

  case "$kind" in
    gateway|hindsight) ;;
    *) die "unknown drop-in kind: $kind" ;;
  esac

  mapfile -t lines
  (( ${#lines[@]} <= _DROPIN_MAX_LINES )) \
    || die "drop-in rejected: too many lines (${#lines[@]} > ${_DROPIN_MAX_LINES})"

  for line in "${lines[@]}"; do
    (( ${#line} <= _DROPIN_MAX_LINE )) || die "drop-in rejected: line too long (${#line} bytes)"
    # No control bytes: rejects CR, tab, NUL and the rest of C0/DEL in one
    # check, so nothing can smuggle a second logical line or an invisible
    # directive. High bytes are allowed — the managed headers are UTF-8 prose
    # ("—") and comments are inert; every *directive* is separately pinned by
    # its own regex below.
    [[ ! "$line" =~ [[:cntrl:]] ]] || die "drop-in rejected: control byte in line: $line"
    # A trailing backslash continues the directive onto the next line in
    # systemd's parser, which would defeat the per-line checks below.
    [[ "$line" != *\\ ]] || die "drop-in rejected: line continuation: $line"

    case "$line" in
      "")   ;;                                            # blank line
      "#"*) ;;                                            # comment (control-byte-free, checked above)
      "["*)
        [[ "$line" == "[Service]" ]] || die "drop-in rejected: section not allowed: $line"
        (( seen_service == 0 )) || die "drop-in rejected: duplicate [Service] section"
        seen_service=1
        ;;
      *)
        (( seen_service == 1 )) || die "drop-in rejected: directive before [Service]: $line"
        validate_dropin_directive "$kind" "$line"
        directives=$(( directives + 1 ))
        ;;
    esac
    out+="${line}"$'\n'
  done

  (( directives > 0 )) || die "drop-in rejected: no allowed directives in body"
  DROPIN_BODY="$out"
}

# ── quadlet content allow-list (#1740) ─────────────────────────────────────
#
# Root-side validation for `write-quadlet`. Same design as the drop-in
# allow-list above — closed key set, pinned value charsets, validated
# reconstruction written instead of raw stdin — but over the richer grammar
# `_render_quadlet_from_plan` actually emits:
#
#   # comment lines (the two generated-by banners)
#   [Unit]      Description= StartLimitIntervalSec= StartLimitBurst=
#   [Container] Image= ContainerName= LogDriver= Network= AddDevice=
#               AddCapability= PodmanArgs= Volume= Environment= PublishPort=
#               HealthCmd= HealthStartPeriod= HealthInterval= HealthRetries=
#               HealthTimeout= Exec=
#   [Service]   Restart= RestartSec= RestartSteps= RestartMaxDelaySec=
#               RestartPreventExitStatus= SyslogIdentifier= StandardOutput=
#               StandardError=
#   [Install]   WantedBy=hal0.target        (omitted when autoload = false)
#
# Sections may appear at most once and only in that order; [Container] is
# mandatory. Every other section (`[Unit]`-lookalikes, a second `[Service]`,
# `[Path]`, `[Timer]`, …) and every key outside its section's list is refused,
# which is what closes the ExecStartPre/User=root smuggling path. Leading
# whitespace, `Key = v`, `;` comments, CR/tab/NUL and any other control byte,
# and a trailing backslash (systemd line continuation, which would hide a
# second logical directive on a value line) are all refused too.
#
# See the HONEST BOUNDARY note in the header for what this deliberately does
# NOT claim: [Container] values are config-derived and only charset-pinned.

QUADLET_BODY=""

_QUADLET_MAX_LINES=400
_QUADLET_MAX_LINE=4096

# Value patterns, kept as named variables so the case arms below read as a
# table and the regexes are not re-quoted per use.
_QRE_DESCRIPTION='^hal0 container inference slot \([A-Za-z0-9_.-]{1,64}\)$'
_QRE_UINT='^[0-9]{1,9}$'
_QRE_IMAGE='^[A-Za-z0-9][A-Za-z0-9._/:@-]{0,254}$'
_QRE_CTRNAME='^hal0-slot-[A-Za-z0-9_.-]{1,64}$'
_QRE_NETWORK='^[A-Za-z0-9_.:-]{1,64}$'
_QRE_DEVICE='^[A-Za-z0-9_./:=-]{1,255}$'
_QRE_CAP='^[A-Za-z0-9_]{1,64}$'
_QRE_VOLUME='^[A-Za-z0-9_./:,+@%=-]{1,512}$'
_QRE_ENV='^[A-Za-z_][A-Za-z0-9_]*=.{0,1024}$'
# host:hostport:ctrport. The host is [slots].publish_host, which its schema
# validator (_publish_host_sane) already constrains to a bare IPv4 or hostname
# — no spaces, ':' or '/' — so the host segment here is colon-free; IPv6
# literals are unsupported at that layer. A bare-IPv4-only charset would
# false-reject the documented hostname case (e.g. hal0.local) and brick every
# bridge-mode slot on such a box.
_QRE_PUBLISH='^[A-Za-z0-9._-]{1,253}:[0-9]{1,5}:[0-9]{1,5}$'
_QRE_DURATION='^[0-9]{1,9}(ns|us|ms|s|m|h|d|w)?$'
_QRE_RETRIES='^[0-9]{1,4}$'
_QRE_RESTART='^(always|on-failure|no)$'
_QRE_WANTEDBY='^hal0\.target$'
# The four flags hal0's providers emit into PodmanArgs= (GPU/llama:
# --group-add/--security-opt; comfyui: --ipc=host; flm: --ulimit memlock=-1).
# See validate_podman_args (#1759).
_QRE_GID='^[0-9]{1,7}$'
_QRE_SECOPT='^[A-Za-z0-9_.:=@,+-]{1,128}$'
_QRE_IPC='^(host|none|private|shareable|container:[A-Za-z0-9_.-]{1,64})$'
_QRE_ULIMIT='^[a-z]{1,16}=-?[0-9]{1,20}(:-?[0-9]{1,20})?$'

_quadlet_die() { die "quadlet rejected: $1"; }

# PodmanArgs= is NOT in-container argv — podman's quadlet generator copies it
# VERBATIM into the generated root unit's ExecStart
# `/usr/bin/podman run … <PodmanArgs> … <image> <Exec>`, so a persistent flag
# like `--runtime <path>` or `--hooks-dir <dir>` makes podman EXEC an
# attacker-named binary as root with no container in the way (#1759). It cannot
# be left as a "non-empty" charset pin. Allow-list exactly the flags hal0's
# own providers emit — nothing else, fail-closed:
#   --group-add <numeric-gid>   GPU/llama-server compat (render group ids)
#   --security-opt <token>      GPU/llama-server + comfyui (seccomp/apparmor/label)
#   --ipc <host|none|…>         comfyui (--ipc=host)
#   --ulimit <name=soft[:hard]> flm/NPU (--ulimit memlock=-1)
# The exec-family (--runtime/--hooks-dir/…) and everything else is refused. The
# deprecated free-form `extra_args` escape hatch (which already logs
# container.extra_args_deprecated) is only honoured insofar as its flags fall in
# this list; anything outside it is refused at the seam.
#
# Both `--flag=value` (one token) and `--flag value` (two tokens) are accepted —
# comfyui emits the `=` form, the GPU path emits the space form.
validate_podman_args() {  # arg: the PodmanArgs= value
  local value="$1"
  local -a toks=()
  # The renderer joins shlex.quote'd tokens with single spaces; every value we
  # emit (gids, security-opt/ipc/ulimit tokens) is quote-free, so a plain
  # word-split is exact. A value that DID carry a quote/space fails closed here.
  read -r -a toks <<<"$value"
  local n=${#toks[@]} i=0 t flag arg
  (( n >= 1 )) || _quadlet_die "empty PodmanArgs"
  while (( i < n )); do
    t="${toks[i]}"
    if [[ "$t" == --*=* ]]; then
      flag="${t%%=*}"; arg="${t#*=}"; i=$(( i + 1 ))
    elif [[ "$t" == --* ]]; then
      flag="$t"
      (( i + 1 < n )) || _quadlet_die "PodmanArgs $flag missing value"
      arg="${toks[i+1]}"; i=$(( i + 2 ))
    else
      _quadlet_die "PodmanArgs expected a --flag, got: $t"
    fi
    case "$flag" in
      --group-add)    [[ "$arg" =~ $_QRE_GID ]]    || _quadlet_die "bad --group-add gid: $arg" ;;
      --security-opt) [[ "$arg" =~ $_QRE_SECOPT ]] || _quadlet_die "bad --security-opt: $arg" ;;
      --ipc)          [[ "$arg" =~ $_QRE_IPC ]]    || _quadlet_die "bad --ipc: $arg" ;;
      --ulimit)       [[ "$arg" =~ $_QRE_ULIMIT ]] || _quadlet_die "bad --ulimit: $arg" ;;
      *) _quadlet_die "PodmanArgs flag not allowed: $flag" ;;
    esac
  done
}

validate_quadlet_directive() {  # args: section, "Key=Value" line
  local section="$1" line="$2" key value
  [[ "$line" == *=* ]] || _quadlet_die "not a Key=Value directive: $line"
  key="${line%%=*}"
  value="${line#*=}"
  # No spaces, no tabs, no quoting around the key: `Key = v` / ` Key=v` are
  # refused rather than normalised, so there is one accepted spelling per
  # directive and no second parse to disagree with systemd's.
  [[ "$key" =~ ^[A-Za-z][A-Za-z0-9]*$ ]] || _quadlet_die "bad directive key: $key"

  case "${section}|${key}" in
    # ── [Unit] ──────────────────────────────────────────────────────────
    # Only the three the renderer emits. Notably absent, and refused:
    # OnFailure=, Requires=, ConditionPathExists= and every other unit key.
    'Unit|Description')            [[ "$value" =~ $_QRE_DESCRIPTION ]] || _quadlet_die "bad Description: $value" ;;
    'Unit|StartLimitIntervalSec')  [[ "$value" =~ $_QRE_UINT ]]        || _quadlet_die "bad StartLimitIntervalSec: $value" ;;
    'Unit|StartLimitBurst')        [[ "$value" =~ $_QRE_UINT ]]        || _quadlet_die "bad StartLimitBurst: $value" ;;

    # ── [Container] ─────────────────────────────────────────────────────
    # Charset-pinned, not semantics-pinned — see the HONEST BOUNDARY note.
    'Container|Image')          [[ "$value" =~ $_QRE_IMAGE ]]    || _quadlet_die "bad Image: $value" ;;
    'Container|ContainerName')  [[ "$value" =~ $_QRE_CTRNAME ]]  || _quadlet_die "bad ContainerName: $value" ;;
    # passthrough is what the renderer emits now (unit-journaled, single
    # copy); none is accepted for the skew window where an older venv
    # renders through a newer wrapper.
    'Container|LogDriver')      [[ "$value" == "passthrough" || "$value" == "none" ]] \
                                                                 || _quadlet_die "bad LogDriver: $value" ;;
    'Container|Network')        [[ "$value" =~ $_QRE_NETWORK ]]  || _quadlet_die "bad Network: $value" ;;
    'Container|AddDevice')
      [[ "$value" =~ $_QRE_DEVICE ]] || _quadlet_die "bad AddDevice: $value"
      [[ "$value" != *".."* ]] || _quadlet_die "AddDevice traversal: $value"
      ;;
    'Container|AddCapability')  [[ "$value" =~ $_QRE_CAP ]]      || _quadlet_die "bad AddCapability: $value" ;;
    'Container|Volume')
      [[ "$value" =~ $_QRE_VOLUME ]] || _quadlet_die "bad Volume: $value"
      [[ "$value" != *".."* ]] || _quadlet_die "Volume traversal: $value"
      ;;
    'Container|Environment')    [[ "$value" =~ $_QRE_ENV ]]      || _quadlet_die "bad Environment: $value" ;;
    'Container|PublishPort')    [[ "$value" =~ $_QRE_PUBLISH ]]  || _quadlet_die "bad PublishPort: $value" ;;
    'Container|HealthCmd')      [[ -n "$value" ]]                || _quadlet_die "empty HealthCmd" ;;
    'Container|HealthStartPeriod'|'Container|HealthInterval'|'Container|HealthTimeout')
      [[ "$value" =~ $_QRE_DURATION ]] || _quadlet_die "bad $key: $value" ;;
    'Container|HealthRetries')  [[ "$value" =~ $_QRE_RETRIES ]]  || _quadlet_die "bad HealthRetries: $value" ;;
    # PodmanArgs= lands host-side in the generated unit's `podman run` argv, so
    # it is allow-listed to the flag shapes hal0's providers emit — anything
    # else (notably --runtime/--hooks-dir, a direct root exec) is refused. See
    # validate_podman_args (#1759). Exec= is the argv INSIDE the container
    # (after the image), so it has no fixed shape and stays bounded only.
    'Container|PodmanArgs')     validate_podman_args "$value" ;;
    'Container|Exec')           [[ -n "$value" ]]                || _quadlet_die "empty Exec" ;;

    # ── [Service] ───────────────────────────────────────────────────────
    # Copied VERBATIM into the generated system unit by podman's quadlet
    # generator — i.e. host-side root. Exactly the eight the renderer emits;
    # every Exec*=, User=, Group=, WorkingDirectory=, … lands in the default
    # arm below and dies. RestartPreventExitStatus= (#2037) is pinned to a
    # single uint — the renderer emits exactly `64`; signal names, lists and
    # ranges systemd would also accept are refused here.
    'Service|Restart')           [[ "$value" =~ $_QRE_RESTART ]] || _quadlet_die "bad Restart: $value" ;;
    'Service|RestartSec')        [[ "$value" =~ $_QRE_UINT ]]    || _quadlet_die "bad RestartSec: $value" ;;
    'Service|RestartSteps')      [[ "$value" =~ $_QRE_UINT ]]    || _quadlet_die "bad RestartSteps: $value" ;;
    'Service|RestartMaxDelaySec') [[ "$value" =~ $_QRE_UINT ]]   || _quadlet_die "bad RestartMaxDelaySec: $value" ;;
    'Service|RestartPreventExitStatus') [[ "$value" =~ $_QRE_UINT ]] || _quadlet_die "bad RestartPreventExitStatus: $value" ;;
    'Service|SyslogIdentifier')  [[ "$value" =~ $_QRE_CTRNAME ]] || _quadlet_die "bad SyslogIdentifier: $value" ;;
    'Service|StandardOutput')    [[ "$value" == "journal" ]]     || _quadlet_die "bad StandardOutput: $value" ;;
    'Service|StandardError')     [[ "$value" == "journal" ]]     || _quadlet_die "bad StandardError: $value" ;;

    # ── [Install] ───────────────────────────────────────────────────────
    # hal0.target only: a WantedBy= on any other target would let a slot unit
    # be pulled into an arbitrary boot path.
    'Install|WantedBy')          [[ "$value" =~ $_QRE_WANTEDBY ]] || _quadlet_die "bad WantedBy: $value" ;;

    *) _quadlet_die "directive not allowed in [$section]: $key" ;;
  esac
}

validate_quadlet_body() {  # body on stdin -> QUADLET_BODY
  local LC_ALL=C
  local line out="" section="" rank=0 next_rank directives=0 seen_container=0
  local -a lines=()

  mapfile -t lines
  (( ${#lines[@]} <= _QUADLET_MAX_LINES )) \
    || _quadlet_die "too many lines (${#lines[@]} > ${_QUADLET_MAX_LINES})"

  for line in "${lines[@]}"; do
    (( ${#line} <= _QUADLET_MAX_LINE )) || _quadlet_die "line too long (${#line} bytes)"
    # One check kills CR, tab, NUL and the rest of C0/DEL, so nothing can
    # smuggle a second logical line or an invisible directive. High bytes stay
    # legal (the banner comments and Description= are UTF-8 prose).
    [[ ! "$line" =~ [[:cntrl:]] ]] || _quadlet_die "control byte in line: $line"
    [[ "$line" != *\\ ]] || _quadlet_die "line continuation: $line"

    case "$line" in
      "")   ;;                                          # blank line
      "#"*) ;;                                          # banner comment
      "["*)
        case "$line" in
          "[Unit]")      next_rank=1 ;;
          "[Container]") next_rank=2; seen_container=1 ;;
          "[Service]")   next_rank=3 ;;
          "[Install]")   next_rank=4 ;;
          *) _quadlet_die "section not allowed: $line" ;;
        esac
        # Strictly increasing: each section at most once, and only in the
        # order the renderer emits them. A second [Service] (the classic
        # override-smuggling trick) fails here even though [Service] itself
        # is a legal section.
        (( next_rank > rank )) || _quadlet_die "section out of order or duplicated: $line"
        rank=$next_rank
        section="${line:1:${#line}-2}"
        ;;
      *)
        [[ -n "$section" ]] || _quadlet_die "directive before any section: $line"
        validate_quadlet_directive "$section" "$line"
        directives=$(( directives + 1 ))
        ;;
    esac
    out+="${line}"$'\n'
  done

  (( seen_container == 1 )) || _quadlet_die "no [Container] section"
  (( directives > 0 )) || _quadlet_die "no allowed directives in body"
  QUADLET_BODY="$out"
}

# ── stale-DNAT repair guard (#1814) ────────────────────────────────────────
#
# Sets PRUNE_CHAIN + PRUNE_TARGET_IP when (port, handle) names a rule this verb
# is allowed to delete; dies otherwise. Fail-closed at every step.
#
# The unprivileged caller is trusted for NOTHING beyond two integers. In
# particular it does not name the chain, does not hand over a rule expression,
# and cannot make this reach a chain outside nv_*_dnat: the chain is discovered
# HERE by walking `nft -a list table inet netavark` and tracking which chain
# each line belongs to. The rule text at the requested handle must then match
# the exact netavark DNAT shape for the requested port, and its target must be
# an IP that no running container holds.
#
# Worst case if the hal0 account is compromised: it can delete netavark DNAT
# rules that are already black holes — i.e. it can do nothing an unrepaired box
# is not already doing to itself. It cannot ADD a rule, cannot REDIRECT a port
# at an attacker-chosen address, cannot touch the filter/masquerade chains, and
# cannot break a working published port (the live-target check refuses that
# case explicitly). Claim only that: this is a *destructive-but-bounded* verb,
# not a firewall-editing primitive.

PRUNE_CHAIN=""
PRUNE_TARGET_IP=""

_PRUNE_DNAT_CHAIN_RE='^nv_[A-Za-z0-9_]+_dnat$'

validate_port() {   # arg: a TCP/UDP port number
  local p="$1"
  [[ -n "$p" ]] || die "missing port"
  [[ "$p" =~ ^[0-9]{1,5}$ ]] || die "bad port: $p"
  (( p >= 1 && p <= 65535 )) || die "port out of range: $p"
}

validate_handle() { # arg: an nftables rule handle
  local h="$1"
  [[ -n "$h" ]] || die "missing handle"
  [[ "$h" =~ ^[0-9]{1,10}$ ]] || die "bad handle: $h"
  (( h >= 1 )) || die "handle out of range: $h"
}

# Echo one IPv4 per line for every RUNNING container. Empty output (no
# containers) is legitimate and simply means every rule target is dead.
_running_container_ips() {
  local ids
  ids="$(podman ps -q 2>/dev/null)" || die "prune-dnat: podman ps failed"
  [[ -n "$ids" ]] || return 0
  # shellcheck disable=SC2086 — ids is a newline list of hex ids from podman
  podman inspect --format '{{range .NetworkSettings.Networks}}{{.IPAddress}}
{{end}}' $ids 2>/dev/null || die "prune-dnat: podman inspect failed"
}

validate_prunable_dnat() {  # args: port, handle
  local port="$1" handle="$2"
  local LC_ALL=C
  local line chain="" rule_re ip

  command -v nft >/dev/null 2>&1 || die "prune-dnat: nft not found"
  command -v podman >/dev/null 2>&1 || die "prune-dnat: podman not found"

  # Both interpolations are already pinned to digits by the validators above.
  rule_re="^[[:space:]]*(ip[[:space:]]+daddr[[:space:]]+[0-9.]+[[:space:]]+)?"
  rule_re+="(tcp|udp)[[:space:]]+dport[[:space:]]+${port}[[:space:]]+"
  rule_re+="dnat[[:space:]]+ip[[:space:]]+to[[:space:]]+([0-9]{1,3}(\.[0-9]{1,3}){3}):[0-9]{1,5}"
  rule_re+="[[:space:]]*#[[:space:]]*handle[[:space:]]+${handle}[[:space:]]*$"

  local candidate
  while IFS= read -r line; do
    # A chain header switches context. Anything that is not an nv_*_dnat chain
    # sets chain="" so its rules can never be reached by this verb, whatever
    # handle the caller asked for.
    if [[ "$line" =~ ^[[:space:]]*chain[[:space:]]+([^[:space:]]+)[[:space:]]*\{ ]]; then
      candidate="${BASH_REMATCH[1]}"   # save: the next =~ clobbers BASH_REMATCH
      chain=""
      if [[ "$candidate" =~ $_PRUNE_DNAT_CHAIN_RE ]]; then
        chain="$candidate"
      fi
      continue
    fi
    if [[ "$line" =~ ^[[:space:]]*\}[[:space:]]*$ ]]; then
      chain=""
      continue
    fi
    if [[ -n "$chain" ]] && [[ "$line" =~ $rule_re ]]; then
      PRUNE_CHAIN="$chain"
      PRUNE_TARGET_IP="${BASH_REMATCH[3]}"
      break
    fi
  done < <(nft -a list table inet netavark 2>/dev/null || true)

  [[ -n "$PRUNE_CHAIN" ]] \
    || die "prune-dnat refused: no dport-${port} dnat rule at handle ${handle} in any nv_*_dnat chain"

  # The whole point of the verb is that the target is already unreachable.
  # Refuse to cut a live container's traffic, whatever the caller believes.
  # Word-split, not line-read: podman's range template emits a trailing space
  # per address, and a whitespace-padded token would silently never compare
  # equal — i.e. the guard would pass for a LIVE container. Addresses are
  # digits and dots, so splitting is exact.
  for ip in $(_running_container_ips); do
    if [[ "$ip" == "$PRUNE_TARGET_IP" ]]; then
      die "prune-dnat refused: ${PRUNE_TARGET_IP} belongs to a running container"
    fi
  done
}

cmd="${1:-help}"; shift || true

case "$cmd" in
  # write-unit was REMOVED in #1740. It installed an unvalidated stdin as a
  # root-owned /etc/systemd/system/hal0-slot@<id>.service — an unconditional
  # root exec primitive — and P3-quadlet left it with no producer at all: the
  # sole renderer of slot-unit text (_render_quadlet_from_plan) emits a
  # `.container`, written through write-quadlet. There is no render contract
  # left to derive an allow-list from, so the verb is deleted rather than
  # guessed at. remove-unit stays: it only ever rm -f's a pinned name, and
  # legacy pre-quadlet .service files still need cleaning up.

  remove-unit)                # remove-unit <slot-id>   (rm -f semantics: ok if absent)
    id="${1:-}"; validate_slot_id "$id"
    rm -f "${UNIT_DIR}/${UNIT_PREFIX}${id}.service"
    ;;

  write-quadlet)              # write-quadlet <slot-id>   (.container body on stdin)
    # #1740: the body is allow-listed BEFORE anything is created, and what
    # lands in /etc/containers/systemd is the validated reconstruction
    # ($QUADLET_BODY), never raw stdin. Atomic (tmp + mv) like the drop-in
    # arms — `install /dev/stdin` misbehaves overwriting an existing target
    # when the source is a pipe, and a half-written .container would make the
    # next daemon-reload drop the slot's generated unit entirely.
    id="${1:-}"; validate_slot_id "$id"
    validate_quadlet_body
    quadlet_path="${QUADLET_DIR}/${UNIT_PREFIX}${id}.container"
    install -d -m 0755 -o root -g root "$QUADLET_DIR"
    umask 022
    tmp="$(mktemp "${quadlet_path}.XXXXXX")"
    trap 'rm -f "$tmp"' EXIT
    printf '%s' "$QUADLET_BODY" > "$tmp"
    chown root:root "$tmp"
    chmod 0644 "$tmp"
    mv -f "$tmp" "$quadlet_path"
    trap - EXIT
    ;;

  remove-quadlet)             # remove-quadlet <slot-id>   (rm -f semantics: ok if absent)
    id="${1:-}"; validate_slot_id "$id"
    rm -f "${QUADLET_DIR}/${UNIT_PREFIX}${id}.container"
    ;;

  write-gateway-dropin)       # write the hermes-gateway secrets drop-in (body on stdin)
    # Literal fixed path — the caller never supplies it. 0644 root:root so
    # systemd can read the unit fragment; the referenced secrets live in the
    # 0600 vault, not here. Atomic (tmp + mv), mirroring hal0-agentenv's
    # write-driver-env — `install /dev/stdin` misbehaves overwriting an
    # existing target when the source is a pipe.
    #
    # #1716: the body is allow-listed BEFORE anything is created, and what gets
    # written is the validated reconstruction ($DROPIN_BODY), never raw stdin.
    validate_dropin_body gateway
    install -d -m 0755 -o root -g root "$GATEWAY_DROPIN_DIR"
    umask 022
    tmp="$(mktemp "${GATEWAY_DROPIN_FILE}.XXXXXX")"
    trap 'rm -f "$tmp"' EXIT
    printf '%s' "$DROPIN_BODY" > "$tmp"
    chown root:root "$tmp"
    chmod 0644 "$tmp"
    mv -f "$tmp" "$GATEWAY_DROPIN_FILE"
    trap - EXIT
    ;;

  write-hindsight-dropin)     # write the hindsight-api extraction drop-in (body on stdin)
    # Literal fixed path — the caller never supplies it, so this verb can only
    # ever reach that one file. 0644 root:root so systemd can read the fragment.
    # Atomic (tmp + mv), same shape as write-gateway-dropin.
    #
    # #1716: allow-listed body (only the two ADR-0023 Environment= keys), and
    # the validated reconstruction is what is written — never raw stdin.
    validate_dropin_body hindsight
    install -d -m 0755 -o root -g root "$HINDSIGHT_DROPIN_DIR"
    umask 022
    tmp="$(mktemp "${HINDSIGHT_DROPIN_FILE}.XXXXXX")"
    trap 'rm -f "$tmp"' EXIT
    printf '%s' "$DROPIN_BODY" > "$tmp"
    chown root:root "$tmp"
    chmod 0644 "$tmp"
    mv -f "$tmp" "$HINDSIGHT_DROPIN_FILE"
    trap - EXIT
    ;;

  check-dropin)               # check-dropin <gateway|hindsight>   (body on stdin)
    # Side-effect-free dry run of the #1716 allow-list: validates the body and
    # echoes back exactly what a write would persist, or dies 64 with the
    # reason. Writes nothing, touches no unit, reloads nothing — it exists so
    # the allow-list can be exercised (by the test suite, and by an operator
    # debugging a rejected body) without a privileged write.
    kind="${1:-}"
    validate_dropin_body "$kind"
    printf '%s' "$DROPIN_BODY"
    ;;

  check-quadlet)              # check-quadlet [<slot-id>]   (.container body on stdin)
    # Side-effect-free dry run of the #1740 allow-list: validates the optional
    # slot id and the body, then echoes back exactly what a write would
    # persist, or dies 64 with the reason. Writes nothing, touches no unit,
    # reloads nothing — it exists so the allow-list can be exercised (by the
    # test suite against the SHIPPED bash, and by an operator debugging a
    # rejected unit) without a privileged write.
    if (( $# > 0 )); then validate_slot_id "${1:-}"; fi
    validate_quadlet_body
    printf '%s' "$QUADLET_BODY"
    ;;

  prune-dnat)                 # prune-dnat <port> <handle>   (#1814)
    # Delete ONE stale netavark DNAT rule. Everything that decides whether the
    # delete is legal is re-derived root-side by validate_prunable_dnat: the
    # chain, the rule shape, and that the target IP is dead. The nft argv below
    # is built from that validated result, never from caller strings.
    port="${1:-}"; handle="${2:-}"
    validate_port "$port"
    validate_handle "$handle"
    validate_prunable_dnat "$port" "$handle"
    nft delete rule inet netavark "$PRUNE_CHAIN" handle "$handle" \
      || die "prune-dnat: nft delete failed for handle ${handle} in ${PRUNE_CHAIN}"
    echo "pruned ${PRUNE_CHAIN} handle ${handle} (dport ${port} -> ${PRUNE_TARGET_IP}, dead)"
    ;;

  check-dnat)                 # check-dnat <port> <handle>   (#1814, side-effect-free)
    # Dry run of the prune-dnat guard: says what a prune WOULD delete, or dies
    # 64 with the refusal reason. Deletes nothing. Exists so the guard can be
    # exercised by the test suite and by an operator debugging a refusal
    # without touching the firewall — the check-quadlet / check-dropin posture.
    port="${1:-}"; handle="${2:-}"
    validate_port "$port"
    validate_handle "$handle"
    validate_prunable_dnat "$port" "$handle"
    echo "prunable ${PRUNE_CHAIN} handle ${handle} (dport ${port} -> ${PRUNE_TARGET_IP}, dead)"
    ;;

  daemon-reload)
    exec systemctl daemon-reload
    ;;

  start|restart)
    # START-LIMIT RECOVERY (#1424/#1791): a slot unit that crash-looped past
    # StartLimitBurst is parked in `failed` with result `start-limit-hit`, and
    # systemd then refuses EVERY start/restart for the rest of
    # StartLimitIntervalSec ("Start request repeated too quickly"). The slot is
    # unloadable via the API until an operator runs `reset-failed` by hand.
    # Clear it here — on the root side of the seam, so it holds for every
    # caller of this verb, not just the one Python path that remembers to ask.
    # Deliberately conditional and best-effort: `is-failed` is true ONLY for a
    # unit systemd has given up on (a healthy or merely-stopped unit reports
    # inactive/active and is left untouched), and a reset-failed that itself
    # fails must not block the start it precedes.
    id="${1:-}"; validate_slot_id "$id"
    unit="${UNIT_PREFIX}${id}.service"
    if systemctl is-failed --quiet "$unit"; then
      systemctl reset-failed "$unit" || true
    fi
    exec systemctl "$cmd" "$unit"
    ;;

  stop|enable|disable|reset-failed)
    id="${1:-}"; validate_slot_id "$id"
    exec systemctl "$cmd" "${UNIT_PREFIX}${id}.service"
    ;;

  stop-agent)                 # stop-agent <agent-id>   -> systemctl stop hal0-agent@<id>.service
    # The systemctl verb is a LITERAL here (not "$cmd" as the slot family does)
    # so the agent-unit surface can never grow a verb by accident: adding one
    # means adding a new case arm, which is a reviewable diff.
    id="${1:-}"; validate_agent_id "$id"
    exec systemctl stop "${AGENT_UNIT_PREFIX}${id}.service"
    ;;

  disable-agent)              # disable-agent <agent-id> -> systemctl disable hal0-agent@<id>.service
    id="${1:-}"; validate_agent_id "$id"
    exec systemctl disable "${AGENT_UNIT_PREFIX}${id}.service"
    ;;

  # Dashboard Services page lifecycle (#1590): the /api/services surface
  # advertises start/restart/enable for the bundled-agent card too, and until
  # these arms existed the API fell through to a bare unprivileged systemctl —
  # polkit's "Interactive authentication required" — for every one of them.
  # Same posture as stop-agent: literal verb per arm, validated id, unit name
  # built here.
  start-agent)                # start-agent <agent-id>  -> systemctl start hal0-agent@<id>.service
    id="${1:-}"; validate_agent_id "$id"
    exec systemctl start "${AGENT_UNIT_PREFIX}${id}.service"
    ;;

  restart-agent)              # restart-agent <agent-id> -> systemctl restart hal0-agent@<id>.service
    id="${1:-}"; validate_agent_id "$id"
    exec systemctl restart "${AGENT_UNIT_PREFIX}${id}.service"
    ;;

  # try-restart-agent (#1882): restart the unit ONLY if it is already running.
  # The Hermes terminal-tool posture change refreshes a live agent, and it runs
  # from every provisioning entry point — including ones the operator drove
  # with the agent deliberately stopped. `restart` would resurrect it; this
  # verb is a no-op on an inactive unit. Strictly narrower than restart-agent.
  try-restart-agent)          # try-restart-agent <agent-id> -> systemctl try-restart hal0-agent@<id>.service
    id="${1:-}"; validate_agent_id "$id"
    exec systemctl try-restart "${AGENT_UNIT_PREFIX}${id}.service"
    ;;

  enable-agent)               # enable-agent <agent-id> -> systemctl enable hal0-agent@<id>.service
    id="${1:-}"; validate_agent_id "$id"
    exec systemctl enable "${AGENT_UNIT_PREFIX}${id}.service"
    ;;

  # Companion-service units (#1590). A CLOSED name->unit map — the caller
  # supplies a service key, never a unit string, so this family can only ever
  # reach the units enumerated here. Growing the map is a reviewable diff.
  svc-start|svc-stop|svc-restart|svc-try-restart|svc-enable|svc-disable)
    key="${1:-}"
    case "$key" in
      openwebui)  unit="hal0-openwebui.service" ;;
      hindsight)  unit="hindsight-api.service" ;;
      # #1863: the Hermes terminal-tool posture only reaches the model once the
      # process re-reads its config, and the gateway is a second consumer of it.
      # Reached via svc-try-restart in practice (#1882 — a posture refresh must
      # not start a gateway the operator stopped), but the svc-* family shares
      # one map.
      hermes-gateway) unit="hermes-gateway.service" ;;
      *) die "bad service key: ${key:-<missing>}" ;;
    esac
    exec systemctl "${cmd#svc-}" "$unit"
    ;;

  restart-self)                # literal-only: restart hal0-api.service, never a variable name
    # --no-block: this is invoked BY hal0-api, from inside hal0-api.service's
    # own cgroup. A blocking `systemctl restart` waits for the unit to stop —
    # but stopping the unit SIGTERMs this systemctl too, so the caller's
    # subprocess.run() returns a signal-killed status for a restart that
    # actually succeeded (#1540). Queue the job and return instead.
    exec systemctl restart --no-block "$SELF_UNIT"
    ;;

  help|"")
    cat <<EOF
usage: hal0-systemctl <command> [slot-id|agent-id]
  remove-unit <slot-id>            rm -f ${UNIT_DIR}/${UNIT_PREFIX}<id>.service (legacy pre-quadlet)
  write-quadlet <slot-id>          write ${QUADLET_DIR}/${UNIT_PREFIX}<id>.container (allow-listed body on stdin)
  remove-quadlet <slot-id>         rm -f ${QUADLET_DIR}/${UNIT_PREFIX}<id>.container
  write-gateway-dropin             write ${GATEWAY_DROPIN_FILE} (body on stdin)
  write-hindsight-dropin           write ${HINDSIGHT_DROPIN_FILE} (body on stdin)
  check-dropin <gateway|hindsight> validate a drop-in body (stdin) without writing it
  check-quadlet [<slot-id>]        validate a .container body (stdin) without writing it
  prune-dnat <port> <handle>       delete ONE stale (dead-target) netavark DNAT rule
  check-dnat <port> <handle>       say whether that rule is prunable, without deleting it
  daemon-reload                    systemctl daemon-reload
  start|stop|restart <slot-id>     systemctl <verb> ${UNIT_PREFIX}<id>.service
  enable|disable <slot-id>         systemctl <verb> ${UNIT_PREFIX}<id>.service
  reset-failed <slot-id>           clear a crash-looped unit's failed sub-state
  stop-agent <agent-id>            systemctl stop ${AGENT_UNIT_PREFIX}<id>.service
  disable-agent <agent-id>         systemctl disable ${AGENT_UNIT_PREFIX}<id>.service
  start-agent <agent-id>           systemctl start ${AGENT_UNIT_PREFIX}<id>.service
  restart-agent <agent-id>         systemctl restart ${AGENT_UNIT_PREFIX}<id>.service
  try-restart-agent <agent-id>     systemctl try-restart ${AGENT_UNIT_PREFIX}<id>.service (no-op if stopped)
  enable-agent <agent-id>          systemctl enable ${AGENT_UNIT_PREFIX}<id>.service
  svc-<verb> <openwebui|hindsight|hermes-gateway>
                                   systemctl <verb> on the mapped companion unit
  restart-self                     systemctl restart ${SELF_UNIT}
EOF
    ;;

  *) die "bad cmd: $cmd" ;;
esac
