You just installed muLLM. Welcome — you're setting it up for the first time. Below are the optional capability layers that extend the core router. You don't need any of these to start chatting. Read each one carefully and only enable what you'll actually use — each module installs dependencies and changes the interface. You can always come back and change this later.
muLLM routes most queries to your local GPU for free. Ollama manages local models.
ollama pull qwen3:8b (general model, ~5GB)ollama pull nomic-embed-textBeginners can skip this. Advanced users can expose experimental and nightly profiles for llama.cpp, TabbyAPI, ExLlamaV3, MTP, and large-context coding comparisons.
Enable additional language packs for chat input/output, translation memory, speech input, and TTS. English remains the routing fallback and cannot be disabled.
mullm1-language-pack.Generated GLB, audio, image, and sidecar metadata files are saved under the asset root. Use a large local disk, mounted NAS/NFS path, or external drive for big generation runs.
[studio.assets] root in mullm.toml. The storage layer already has room for local, s3, azure_blob, and external API backends; Perforce, Git LFS, and NFS can sit behind the same asset root or backend interface later.Cloud models handle complex queries. You only pay for what muLLM can't handle locally (~15% of queries).
Configure alternative inference providers for cost/speed variety. Keys use the selected local secret storage.
Enable or disable individual AI providers. When a provider is disabled, muLLM automatically falls back to the next cheapest available provider. Changes take effect immediately — no restart needed.
Choose how muLLM selects among configured cloud providers after groundtruth, cache, and local routing have already had a chance to answer.
Enable or disable individual routing capabilities. Changes take effect immediately — no restart needed.
http://127.0.0.1:8188. Install ComfyUI separately for local image,
texture, LoRA, and Hunyuan3D workflows, then keep it running before enabling local media routing.
Read the muLLM ComfyUI setup notes.
git clone https://github.com/comfyanonymous/ComfyUI && cd ComfyUI && python -m venv venv && ./venv/bin/pip install -r requirements.txt && ./venv/bin/python main.py
venv\Scripts\python.exe main.py.
Configure these with environment variables or mullm.toml. HTTP classifiers must return JSON with category, complexity, and confidence.
MULLM_CLASSIFIER_BACKEND=auto|rules|deberta|http|onnx
MULLM_CLASSIFIER_MODEL_PATH=/path/to/transformers-model
MULLM_CLASSIFIER_HTTP_URL=http://127.0.0.1:9001/classify
MULLM_CLASSIFIER_ONNX_MODEL_PATH=/path/to/model.onnx
MULLM_CLASSIFIER_RETRAIN_ENABLED=true
MULLM_CLASSIFIER_RETRAIN_SCHEDULE=daily
Web search is off by default. DuckDuckGo is the keyless default when a user explicitly enables web search; keyed providers can be configured for citation-heavy research and enterprise deployments.
use_web=true only from the explicit chat web-search control or a future policy-approved research flow.Configure burst GPU and private GPU endpoints. These are intended as a 2.5 tier between local desktop and expensive frontier cloud calls.
.env on your local machine only, never sent to muLLM's code or any third party. The .env file is automatically added to .gitignore and permissions are set to owner-only (600).OLLAMA_API_KEY blank for local use.
# Step 1: restrict to owner-read-only
chmod 0400 ~/.config/mullm/.env
# Step 2: make immutable (Linux — requires sudo)
sudo chattr +i ~/.config/mullm/.env
WARNING: After chattr +i the file cannot be modified or deleted without removing the immutable flag first. muLLM reads but never writes to a sealed config.
sudo chattr -i ~/.config/mullm/.env # remove immutable flag
chmod 0600 ~/.config/mullm/.env # restore write access
On macOS use sudo chflags schg ~/.config/mullm/.env and sudo chflags noschg to unseal.
.env entirely, muLLM supports loading values from an external secret manager at startup. Set values in .env using the SECRET: URI prefix and muLLM resolves them at startup — the plaintext never touches disk.
ANTHROPIC_API_KEY=SECRET:pass:infra/openai_keys/anthropic
OPENAI_API_KEY=SECRET:pass:infra/openai_keys/openai
File-based (org secret mount):
ANTHROPIC_API_KEY=SECRET:file:/opt/secrets/mullm/anthropic_key
Environment passthrough (CI / Docker): If a key is already in the process environment (e.g. injected by Kubernetes Secrets or Doppler), just leave it out of .env entirely — muLLM picks it up from the environment automatically.op), Doppler CLI — contact support or see router/secrets.py for the resolver interface.
mullm.your-domain.com {
reverse_proxy 127.0.0.1:6856
tls {
protocols tls1.3
}
}
nginx + post-quantum (X25519MLKEM768 key exchange):
ssl_protocols TLSv1.3;
ssl_ecdh_curve X25519MLKEM768:X25519;
ssl_stapling on;
add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload";
All outbound calls to Anthropic/OpenAI/Google already use TLS 1.3 via Python's httpx with system OpenSSL. The post-quantum key exchange applies to inbound connections from browsers and other clients.
GOOGLE_CLIENT_ID=<your-client-id>.apps.googleusercontent.com
GOOGLE_CLIENT_SECRET=<your-client-secret>
MULLM_BASE_URL=https://<your-public-host>:6856
https://<your-public-host>:6856/auth/callback as an
authorized redirect URI in your
Google Cloud OAuth credentials.pip install mullm[auth]GOOGLE_CLIENT_ID is not set, auth is completely disabled
— no redirects, no 401s, everything works exactly as before.
/v1/chat/completions endpoint works with Aider, Continue, Codex, LLM CLI, and anything that accepts a custom base URL
# In ~/.claude/settings.json (or project .claude/settings.json)
{
"mcpServers": {
"mullm": {
"command": "npx",
"args": ["-y", "@0101/mullm-mcp"],
"env": { "MULLM_URL": "http://127.0.0.1:6856" }
}
}
}
Or via CLI: claude mcp add mullm -e MULLM_URL=http://127.0.0.1:6856 -- npx -y @0101/mullm-mcp~/.continue/config.json):
{
"models": [{
"title": "muLLM (local-first)",
"provider": "openai",
"model": "mullm-auto",
"apiBase": "http://127.0.0.1:6856/v1",
"apiKey": "mullm"
}]
}
aider --openai-api-base http://127.0.0.1:6856/v1 --openai-api-key mullm --model mullm-auto~/.aider.conf.yml:
openai-api-base: http://127.0.0.1:6856/v1
openai-api-key: mullm
model: mullm-auto
http://127.0.0.1:6856/v1mullm (or any string — not validated for local use)mullm-auto (let muLLM choose) or any cloud model nameMake muLLM start automatically when you log in. One command installs the right service for your OS:
Creates a systemd user service — no root required. Runs after login, restarts on failure.
mkdir -p ~/.config/systemd/user cat > ~/.config/systemd/user/mullm.service <<'EOF' [Unit] Description=muLLM local LLM router (HTTPS :6856) After=network.target [Service] Type=simple WorkingDirectory=~/newdev/newestmullm ExecStart=~/.venv/bin/python -m router.main Restart=on-failure RestartSec=5 Environment=PYTHONUNBUFFERED=1 Environment=MULLM_REMOTE_ACCESS=true Environment=MULLM_OPEN_BROWSER=0 Environment=MULLM_SSL_CERTFILE=~/newdev/newestmullm/certs/server.crt Environment=MULLM_SSL_KEYFILE=~/newdev/newestmullm/certs/server.key [Install] WantedBy=default.target EOF systemctl --user daemon-reload systemctl --user enable mullm systemctl --user start mullm
The Terrain Builder (/terrain) works out of the box with procedural generation and USGS (free, CONUS). Add these keys to unlock global real-world DEM data.
~/.config/systemd/user/mullm.service):Environment=OPENTOPOGRAPHY_API_KEY=your_key_here
Or set in your shell profile for the session:
export OPENTOPOGRAPHY_API_KEY=your_key_here
Covers: NASADEM (30m), SRTM GL1 (30m), SRTM GL3 (90m), Copernicus DEM (30m). Excellent for medium/large-scale terrain (~5km+).
Environment=MAPBOX_TOKEN=pk.eyJ1...
Resolution varies by zoom level: zoom 10 ≈ 100m/px, zoom 13 ≈ 10m/px. Best for city-scale or high-detail small areas.
systemctl --user restart mullm.service) then reload the Terrain Builder — the data source status dots will turn green.
Co-authored-by trailer to the commit message — giving muLLM
provenance without replacing any human author.mullm --install-git-hook