{% extends "_base.html" %} {% block content %}
Distributed · inference

Cluster

Connect two or more Macs over Thunderbolt and use them as one. Peers are found automatically, so there are no IPs or hostfiles to set up. OptiQ splits a model's layers across the Macs by RAM and keeps them resident, so the combined memory serves models that wouldn't fit on any one machine.

nodes
combined RAM
fits now
thunderbolt
status
Topology

Serve a model on the cluster

Serves the model split across the ring, with an OpenAI endpoint that Chat and Arena use automatically. Thunderbolt only. Pipeline inference needs the low latency, so a slow link is refused rather than run slower than one Mac. Layers are held resident, so the model has to fit the the cluster can spare right now; anything larger is refused before it loads. Free up memory on a busy Mac to make room for a bigger model.

Loading shards across the ring…
A large model can take a few minutes to page into memory on both Macs. Chat opens automatically when it's ready — you don't need to wait here.
This cluster is your active inference backend
Chat and Arena now run across the ring. Nothing else to switch.
Open Chat →
{% endblock %}