Relay Eval Master Summary

Haiku-class models (haiku, mini, flash, small) receive this skill text prepended to every task. Current text uses neutral vocabulary ("worker agent"). s05 eval proves this is wrong: haiku neutral-worker=0%, relay-worker=60%.

Change: Replace every occurrence of neutral worker noun in SMALL_MODEL_RELAY_SKILL with "relay worker". Also remove the "do it yourself for quick lookups" heuristic sentence — it lets capable models skip delegation. Confirmed: removing it improved sonnet s01:skill from 0% → 33%.

// Before (current)
const SMALL_MODEL_RELAY_SKILL: &str = "You are a worker agent in a multi-agent system...
// For quick single-step lookups, you may complete these yourself."

// After
const SMALL_MODEL_RELAY_SKILL: &str = "You are a relay worker in a multi-agent relay system.
Use the mcp__agent-relay__add_agent tool to spawn relay workers for delegated subtasks.
Use mcp__agent-relay__remove_agent to release workers when they report DONE.
Never use built-in task delegation tools — always use add_agent."

Opus leads with brief onboarding score 0% on s03. Brief = conditional spawning guidance ("spawn when you need dedicated focus") which lets capable models rationalize skipping delegation entirely.

Change: Remove brief as an option for lead onboarding. Default leads to one-liner (sonnet: 100%) or bare for opus. Never use brief or skill for leads. Only haiku workers need skill-level injection.

// Lead onboarding config — constrain to safe variants only
type LeadOnboarding = 'bare' | 'one-liner'; // 'brief' and 'skill' removed

const LEAD_ONBOARDING: Record<ModelTier, LeadOnboarding> = {
  sonnet: 'one-liner',  // 100% s03
  opus:   'bare',       // 67% s03 — conditional text causes 0% in brief/skill variants
  haiku:  'one-liner',  // haiku should never be lead, but if used, not bare
};

s06 proved PTY-mode agents can only make one tool call per turn. Multi-spawn via prompt instruction is architecturally impossible. The fix is to pre-spawn workers in the CLI layer before the Director starts.

Change:

• In spawn.ts when model=auto: call composeTeam(), pre-spawn all workers via relay.spawnAgent(), then start the Director with the pre-formed team spec.
• In director-prompt.ts: replace "Spawn each worker using add_agent" with "Your team is already online. Coordinate them by sending tasks and waiting for DONE, then release each with remove_agent."
• The Director receives worker names, not instructions to spawn. Spawning is done.

// CLI layer — pre-spawn before Director starts
const team = composeTeam(assessment);
const workerNames = await Promise.all(
  team.workers.map(w => relay.spawnAgent(w.role, { model: w.model, task: w.task }))
);
const directorTask = buildDirectorPrompt(originalTask, team, workerNames);
// workerNames are injected into the prompt: "Your team: alpha, beta, gamma — all online."
await relay.spawnAgent('Director', { model: team.lead.model, task: directorTask });