Combo Routing: Multi-Tier Model Failover¶
Combo Routing allows a single Avo conversation to orchestrate multiple models across prioritized tiers (for example: account quota → cheap API → free local floor).
When rate limits (HTTP 429) or quota errors occur mid-turn, Avo transparently fails over to the next configured tier, persisting full run history and recording a route_failover event in the SQLite ledger.
1. Architecture & Tier Concepts¶
┌─────────────────────────────────────────────────────────┐
│ Avo Runtime │
└───────────────────────────┬─────────────────────────────┘
│
┌─────────────▼─────────────┐
│ ComboRouterProvider │
└─────────────┬─────────────┘
│
┌─────────────────────┼─────────────────────┐
│ (Tier 1: Primary) │ (Tier 2: Cheap) │ (Tier 3: Free Floor)
┌─────▼──────────────┐┌─────▼──────────────┐┌─────▼──────────────┐
│ Claude / ChatGPT ││ OpenRouter / Groq ││ Ollama (Local) │
│ (Account / Quota) ││ (Pay-per-token) ││ (Zero Cost / Free) │
└────────────────────┘└────────────────────┘└────────────────────┘
│ (429/Quota) │ (429/Quota)
└──────► Fallback ────┴──────► Fallback ────► Done
Failover Trigger Matrix¶
| Condition | Detected Error Signatures | Action |
|---|---|---|
| Rate Limited (429) | 429 Too Many Requests, rate_limit_exceeded |
Failover to next tier, mark current tier cooling down |
| Quota / Credits Exhausted | insufficient_quota, credit balance too low, RESOURCE_EXHAUSTED |
Failover to next tier, mark current tier cooling down |
| Client Error (400) | invalid_argument, context_length_exceeded |
Fail-closed: raise error immediately (do not mask bug) |
| Authentication Error (401) | invalid_api_key, unauthorized |
Fail-closed: raise error immediately |
2. Built-in Preset Combos¶
Avo includes three built-in presets out of the box:
| Preset Name | Tier 1 (Account) | Tier 2 (Cheap API) | Tier 3 (Free Local Floor) |
|---|---|---|---|
default |
claude (claude-sonnet-5) |
openrouter (llama-3.3-70b-instruct) |
ollama (llama3.2) |
coder |
claude (claude-sonnet-5) |
openrouter (llama-3.3-70b-instruct) |
ollama (qwen2.5-coder:7b) |
budget |
(none) | openrouter (llama-3.3-70b-instruct) |
ollama (llama3.2) |
3. CLI Management (avo combo)¶
Use the avo combo command-line tool to list, create, inspect, and remove combo profiles:
Listing Combos¶
Output:
NAME TIERS DESCRIPTION
budget cheap (openrouter/meta-llama/...) -> free... OpenRouter Llama -> Local Ollama
coder account (claude/claude-sonnet-5) -> ... Claude account -> OpenRouter Llama -> Local Qwen Coder
default account (claude/claude-sonnet-5) -> ... Claude account -> OpenRouter Llama -> Local Ollama
To emit machine-readable JSON:
Inspecting a Profile¶
Output:
Profile : coder (built-in preset)
Description : Claude Sonnet -> OpenRouter Llama -> Local Qwen Coder
Tiers (priority order):
1. account provider=claude model=claude-sonnet-5 timeout=60.0s cooldown=60.0s
2. cheap provider=openrouter model=meta-llama/llama-... timeout=60.0s cooldown=60.0s
3. free provider=ollama model=qwen2.5-coder:7b timeout=60.0s cooldown=60.0s
Creating a Custom Combo¶
Use avo combo new to define custom tiers. Tier specifications follow [name:]provider:model[:timeout[:cooldown]]:
avo combo new team_pipeline \
--tier primary:anthropic:claude-3-5-sonnet-latest:60:120 \
--tier secondary:openrouter:meta-llama/llama-3.3-70b-instruct:45:60 \
--tier local:ollama:llama3.2:30:30 \
--description "Production pipeline with local fallback"
Removing a Custom Combo¶
[!NOTE] Built-in presets (
default,coder,budget) cannot be deleted, but you can override them with a custom profile of the same name. Deleting the override restores the built-in preset.
4. Interactive Chat REPL Integration¶
Starting the REPL with a Combo¶
Set AVO_PROVIDER=combo and select your profile via AVO_COMBO:
The startup banner displays the active combo configuration:
▄██▄ Avo CLI 0.1.7
▄██████▄ Fqih
███ ███ provider: combo · model: coder [account -> cheap -> free]
███ ▄▄ ███ workspace: ~/Project/Loopward
███ ▀▀ ███ session: a1b2c3d4e5f6
──────────────────────────────────────────────────────
The /combo Slash Command¶
Inspect the live health and circuit breaker status of each tier during a session:
Output:
Active Combo Profile: coder
Description: Claude Sonnet -> OpenRouter Llama -> Local Qwen Coder
Tiers (priority order):
Tier Provider Model Status Cooldown Latency Fails
-------------- ------------ ------------------------ --------- --------- -------- -----
account claude claude-sonnet-5 HEALTHY 0s 240.2ms 0
cheap openrouter meta-llama/llama-3.3... HEALTHY 0s - 0
free ollama qwen2.5-coder:7b HEALTHY 0s - 0
Available combo profiles: budget, coder, default, team_pipeline
Switch with: /combo <NAME>
Hot-Swapping Combos Mid-Session¶
You can switch combo profiles dynamically without restarting the chat:
Live Fallback Notices¶
When a tier fails due to quota or rate limits, Avo displays an inline notice before streaming from the fallback tier:
The conversation proceeds smoothly and the reply is rendered without interruption.
5. Authenticate the Combo's Vendors¶
Login is independent per vendor. This command opens each missing browser account flow once, in sequence, and skips credentials already stored:
Ollama Local needs no account. For OpenRouter or other API tiers, Avo prints the corresponding API-key command instead of asking for passwords.
6. Setting up Ollama as the Free Tier Floor¶
Using Ollama as your lowest tier guarantees that your agent loops will never crash due to vendor outages, rate limits, or billing issues.
-
Install and launch Ollama:
-
Inspect the recommendation, then pull with confirmation:
-
Verify connectivity:
Avo's default Ollama provider connects to http://localhost:11434 unless AVO_OLLAMA_BASE_URL is set.
Ollama Cloud is a separate remote route; use avo login ollama-cloud --key-stdin
and never treat a Cloud model as a local download.