# Multi-Target Parallel Evaluation Implementation Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add `agentsec multi-evaluate --config multi-target.yaml` to evaluate multiple OpenClaw instances in parallel, producing per-target artifacts and a cross-target `summary.md`.

**Architecture:** New `MultiTargetConfig` + `TargetEntry` Pydantic models in `config.py`; new `TargetResult` dataclass + `render_multi_summary` renderer in `reports/multi_summary.py`; new `multi_evaluate` CLI command in `cli.py` that runs `_evaluate_one_target` (new private helper) per target via `ThreadPoolExecutor`. The existing `evaluate` command and all existing helpers are untouched.

**Tech Stack:** Python 3.11+, Pydantic v2, Typer, Rich, `concurrent.futures.ThreadPoolExecutor`

---

## File Map

| File | Action | Responsibility |
|------|--------|----------------|
| `src/agentsec/config.py` | Modify | Add `TargetEntry`, `MultiTargetConfig` |
| `src/agentsec/reports/multi_summary.py` | Create | `TargetResult` dataclass + `render_multi_summary` |
| `src/agentsec/reports/__init__.py` | Modify | Re-export `TargetResult`, `render_multi_summary` |
| `src/agentsec/cli.py` | Modify | Add `_evaluate_one_target` helper + `multi_evaluate` command |
| `tests/test_config_model.py` | Modify | `MultiTargetConfig` / `TargetEntry` unit tests |
| `tests/test_multi_summary.py` | Create | `render_multi_summary` unit tests |
| `tests/integration/test_multi_evaluate.py` | Create | End-to-end CLI integration tests |
| `tests/fixtures/multi-target.example.yaml` | Create | Example config fixture |
| `CHANGELOG.md` | Modify | v0.6.0 entry |
| `ROADMAP.md` | Modify | Tick off multi-target item |

---

## Task 1: Config Models — `TargetEntry` + `MultiTargetConfig`

**Files:**
- Modify: `src/agentsec/config.py`
- Modify: `tests/test_config_model.py`

- [ ] **Step 1.1: Write the failing tests**

Append to `tests/test_config_model.py`:

```python
import pytest
from pydantic import ValidationError

from agentsec.config import (
    MultiTargetConfig,
    OutputConfig,
    RemoteConfig,
    TargetEntry,
)


# ── MultiTargetConfig ────────────────────────────────────────────────────────

def test_multi_target_config_parse_two_targets():
    cfg = MultiTargetConfig.model_validate({
        "output_base": "./multi-report/",
        "targets": [
            {"name": "prod", "remote": {"adapter": "http", "target": "http://prod:8080", "test_suites": []}},
            {"name": "staging", "remote": {"adapter": "http", "target": "http://staging:8080", "test_suites": []}},
        ],
    })
    assert len(cfg.targets) == 2
    assert cfg.effective_max_parallel == 2  # default = len(targets)


def test_multi_target_config_explicit_max_parallel():
    cfg = MultiTargetConfig.model_validate({
        "output_base": "./out/",
        "max_parallel": 1,
        "targets": [
            {"name": "a", "remote": {"adapter": "http", "target": "http://a", "test_suites": []}},
            {"name": "b", "remote": {"adapter": "http", "target": "http://b", "test_suites": []}},
        ],
    })
    assert cfg.effective_max_parallel == 1


def test_multi_target_config_missing_output_base():
    with pytest.raises(ValidationError):
        MultiTargetConfig.model_validate({
            "targets": [
                {"name": "a", "remote": {"adapter": "http", "target": "http://a", "test_suites": []}},
            ],
        })


def test_multi_target_config_duplicate_names_rejected():
    with pytest.raises(ValidationError, match="duplicate target names"):
        MultiTargetConfig.model_validate({
            "output_base": "./out/",
            "targets": [
                {"name": "prod", "remote": {"adapter": "http", "target": "http://a", "test_suites": []}},
                {"name": "prod", "remote": {"adapter": "http", "target": "http://b", "test_suites": []}},
            ],
        })


def test_multi_target_config_empty_targets_rejected():
    with pytest.raises(ValidationError):
        MultiTargetConfig.model_validate({"output_base": "./out/", "targets": []})


def test_multi_target_config_from_yaml(tmp_path):
    p = tmp_path / "multi.yaml"
    p.write_text(
        "output_base: ./multi-report/\n"
        "targets:\n"
        "  - name: prod\n"
        "    remote:\n"
        "      adapter: http\n"
        "      target: http://prod\n"
        "      test_suites: []\n"
    )
    cfg = MultiTargetConfig.from_yaml(p)
    assert cfg.targets[0].name == "prod"
    assert cfg.output_base == Path("./multi-report/")


# ── TargetEntry ───────────────────────────────────────────────────────────────

def test_target_entry_output_resolves_from_output_base():
    entry = TargetEntry(
        name="prod",
        remote=RemoteConfig(adapter="http", target="http://x", test_suites=[]),
    )
    openclaw_cfg = entry.to_openclaw_config(Path("./multi-report/"))
    assert openclaw_cfg.output.dir == Path("./multi-report/prod")


def test_target_entry_explicit_output_preserved():
    entry = TargetEntry(
        name="prod",
        remote=RemoteConfig(adapter="http", target="http://x", test_suites=[]),
        output=OutputConfig(dir=Path("/custom/out/")),
    )
    openclaw_cfg = entry.to_openclaw_config(Path("./multi-report/"))
    assert openclaw_cfg.output.dir == Path("/custom/out/")


def test_target_entry_rejects_neither_remote_nor_server_audit():
    with pytest.raises(ValidationError, match="at least one of"):
        TargetEntry(name="empty")


def test_multi_target_config_rejects_unknown_key():
    with pytest.raises(ValidationError, match="bad_key"):
        MultiTargetConfig.model_validate({
            "output_base": "./out/",
            "bad_key": 1,
            "targets": [
                {"name": "a", "remote": {"adapter": "http", "target": "http://a", "test_suites": []}},
            ],
        })
```

- [ ] **Step 1.2: Run tests to confirm they fail**

```bash
.venv/bin/pytest tests/test_config_model.py -k "multi_target or target_entry" -v
```

Expected: `ImportError` or `AttributeError` — `MultiTargetConfig` not yet defined.

- [ ] **Step 1.3: Implement `TargetEntry` and `MultiTargetConfig` in `config.py`**

Add after the `OpenClawTargetConfig` class (keep the existing class unchanged):

```python
class TargetEntry(BaseModel):
    """Single target within a multi-evaluate config.

    Like OpenClawTargetConfig but `output` is optional; MultiTargetConfig
    resolves it to output_base/<name>/ when absent.
    """
    model_config = _FORBID
    name: str
    remote: RemoteConfig | None = None
    server_audit: ServerAuditConfig | None = None
    output: OutputConfig | None = None
    authorization: AuthorizationConfig = Field(default_factory=AuthorizationConfig)
    scoring: ScoringConfig = Field(default_factory=ScoringConfig)

    @model_validator(mode="after")
    def _at_least_one_half(self) -> "TargetEntry":
        if self.remote is None and self.server_audit is None:
            raise ValueError(
                f"target {self.name!r}: at least one of `remote` or `server_audit` "
                "must be configured"
            )
        return self

    def to_openclaw_config(self, output_base: "Path") -> "OpenClawTargetConfig":
        resolved = self.output or OutputConfig(dir=output_base / self.name)
        return OpenClawTargetConfig(
            name=self.name,
            remote=self.remote,
            server_audit=self.server_audit,
            output=resolved,
            authorization=self.authorization,
            scoring=self.scoring,
        )


class MultiTargetConfig(BaseModel):
    model_config = _FORBID
    output_base: Path
    max_parallel: int = Field(default=0, ge=0)
    targets: list[TargetEntry] = Field(min_length=1)

    @model_validator(mode="after")
    def _unique_names(self) -> "MultiTargetConfig":
        names = [t.name for t in self.targets]
        dupes = {n for n in names if names.count(n) > 1}
        if dupes:
            raise ValueError(f"duplicate target names: {sorted(dupes)}")
        return self

    @classmethod
    def from_yaml(cls, path: "Path | str") -> "MultiTargetConfig":
        text = Path(path).read_text(encoding="utf-8")
        data = yaml.safe_load(text)
        if not isinstance(data, dict):
            raise ValueError(f"{path}: config must be a YAML mapping")
        return cls.model_validate(data)

    @property
    def effective_max_parallel(self) -> int:
        return self.max_parallel if self.max_parallel > 0 else len(self.targets)
```

- [ ] **Step 1.4: Run tests to confirm they pass**

```bash
.venv/bin/pytest tests/test_config_model.py -k "multi_target or target_entry" -v
```

Expected: all new tests PASS; no existing tests broken.

```bash
.venv/bin/pytest tests/test_config_model.py -v
```

Expected: all tests PASS.

- [ ] **Step 1.5: Commit**

```bash
git add src/agentsec/config.py tests/test_config_model.py
git commit -m "feat(config): add TargetEntry and MultiTargetConfig for multi-evaluate"
```

---

## Task 2: Multi-Summary Renderer

**Files:**
- Create: `src/agentsec/reports/multi_summary.py`
- Modify: `src/agentsec/reports/__init__.py`
- Create: `tests/test_multi_summary.py`

- [ ] **Step 2.1: Write the failing tests**

Create `tests/test_multi_summary.py`:

```python
from pathlib import Path

import pytest

from agentsec.reports.multi_summary import TargetResult, render_multi_summary
from agentsec.scoring import CombinedScore, CoverageTriple, EvaluationScore


def _make_score(
    grade: str = "A",
    combined: float | None = 90.0,
    remote: float | None = 88.0,
    server: int | None = 92,
    findings: list | None = None,
) -> EvaluationScore:
    return EvaluationScore(
        category_scores=[],
        remote_score=remote,
        server_score=server,
        combined=CombinedScore(
            score=combined, weight_remote_used=0.5,
            weight_server_used=0.5, note=None,
        ),
        coverage=CoverageTriple(
            planned=5, executed=5, scored=5, error=0,
            execution_coverage=1.0, verdict_coverage=1.0, error_rate=0.0,
        ),
        low_coverage=False,
        grade=grade,
        deduped_findings=findings or [],
    )


def test_render_two_success_targets():
    results = [
        TargetResult("prod", Path("./out/prod"), _make_score(), None, 272.0),
        TargetResult("staging", Path("./out/staging"), _make_score(grade="C", combined=61.2), None, 198.0),
    ]
    md = render_multi_summary(results, generated_at="2026-05-04T12:00:00Z")
    assert "# Multi-Target Evaluation Summary" in md
    assert "2 success" in md
    assert "0 errors" in md
    assert "| prod |" in md
    assert "| staging |" in md
    assert "prod/combined-report.md" in md
    assert "staging/combined-report.md" in md


def test_render_one_success_one_error():
    results = [
        TargetResult("prod", Path("./out/prod"), _make_score(), None, 272.0),
        TargetResult("dev", Path("./out/dev"), None, "SSH connection refused", 5.0),
    ]
    md = render_multi_summary(results, generated_at="2026-05-04T12:00:00Z")
    assert "1 success" in md
    assert "1 error" in md
    assert "| dev | ERROR |" in md
    assert "SSH connection refused" in md
    # error target must not appear in Reports section
    assert "dev/combined-report.md" not in md


def test_insufficient_coverage_grade_abbreviated():
    results = [
        TargetResult("prod", Path("./out/prod"), _make_score(grade="INSUFFICIENT_COVERAGE"), None, 60.0),
    ]
    md = render_multi_summary(results, generated_at="2026-05-04T12:00:00Z")
    assert "LOW_COV" in md
    assert "INSUFFICIENT_COVERAGE" not in md


def test_finding_counts_in_table():
    from agentsec.audit.findings import Finding
    findings = [
        Finding(check="x", title="c1", severity="critical", evidence="e", fingerprint="fp1", threat_refs=[]),
        Finding(check="x", title="h1", severity="high", evidence="e", fingerprint="fp2", threat_refs=[]),
        Finding(check="x", title="h2", severity="high", evidence="e", fingerprint="fp3", threat_refs=[]),
    ]
    results = [
        TargetResult("prod", Path("./out/prod"), _make_score(findings=findings), None, 60.0),
    ]
    md = render_multi_summary(results, generated_at="2026-05-04T12:00:00Z")
    row = next(line for line in md.splitlines() if line.startswith("| prod |"))
    cells = [c.strip() for c in row.split("|")]
    # | Target | Grade | Score | Remote | Server | Critical | High | Elapsed |
    # cells[0]="" [1]=prod [2]=grade [3]=score [4]=remote [5]=server [6]=critical [7]=high [8]=elapsed
    assert cells[6] == "1"  # critical
    assert cells[7] == "2"  # high


def test_elapsed_formatting():
    results = [
        TargetResult("prod", Path("./out/prod"), _make_score(), None, 272.0),  # 4m32s
    ]
    md = render_multi_summary(results, generated_at="2026-05-04T12:00:00Z")
    assert "4m32s" in md


def test_generated_at_defaults_when_empty():
    results = [
        TargetResult("prod", Path("./out/prod"), _make_score(), None, 10.0),
    ]
    md = render_multi_summary(results)
    assert "Generated:" in md


def test_not_scored_shows_dash():
    results = [
        TargetResult(
            "prod", Path("./out/prod"),
            _make_score(grade="NOT_SCORED", combined=None, remote=None, server=None),
            None, 10.0,
        ),
    ]
    md = render_multi_summary(results, generated_at="2026-05-04T12:00:00Z")
    row = next(line for line in md.splitlines() if line.startswith("| prod |"))
    assert "NOT_SCORED" in row
```

- [ ] **Step 2.2: Run tests to confirm they fail**

```bash
.venv/bin/pytest tests/test_multi_summary.py -v
```

Expected: `ModuleNotFoundError` — `multi_summary` not yet created.

- [ ] **Step 2.3: Create `src/agentsec/reports/multi_summary.py`**

```python
"""Cross-target summary renderer for `agentsec multi-evaluate`."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from agentsec.scoring import EvaluationScore


@dataclass
class TargetResult:
    name: str
    output_dir: Path
    score: "EvaluationScore | None"
    error: str | None
    elapsed_seconds: float


def render_multi_summary(
    results: list[TargetResult],
    generated_at: str = "",
) -> str:
    if not generated_at:
        generated_at = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")

    total = len(results)
    success_count = sum(1 for r in results if r.error is None)
    error_count = total - success_count
    error_word = "error" if error_count == 1 else "errors"

    lines: list[str] = [
        "# Multi-Target Evaluation Summary",
        "",
        f"Generated: {generated_at}  ",
        f"Targets: {total} ({success_count} success, {error_count} {error_word})",
        "",
        "## Score Overview",
        "",
        "| Target | Grade | Score | Remote | Server | Critical | High | Elapsed |",
        "|--------|-------|-------|--------|--------|----------|------|---------|",
    ]

    for r in results:
        if r.error is not None:
            lines.append(
                f"| {r.name} | ERROR | — | — | — "
                f"| — | — | {_fmt_elapsed(r.elapsed_seconds)} |"
            )
        else:
            s = r.score
            assert s is not None
            grade = "LOW_COV" if s.grade == "INSUFFICIENT_COVERAGE" else s.grade
            combined = f"{s.combined.score:.1f}" if s.combined.score is not None else "—"
            remote = f"{s.remote_score:.1f}" if s.remote_score is not None else "—"
            server = str(s.server_score) if s.server_score is not None else "—"
            critical = sum(1 for f in s.deduped_findings if f.severity == "critical")
            high = sum(1 for f in s.deduped_findings if f.severity == "high")
            lines.append(
                f"| {r.name} | {grade} | {combined} | {remote} | {server} "
                f"| {critical} | {high} | {_fmt_elapsed(r.elapsed_seconds)} |"
            )

    success_results = [r for r in results if r.error is None]
    if success_results:
        lines += ["", "## Reports", ""]
        for r in success_results:
            lines.append(f"- [{r.name}]({r.name}/combined-report.md)")

    error_results = [r for r in results if r.error is not None]
    if error_results:
        lines += ["", "## Errors", ""]
        for r in error_results:
            lines += [f"### {r.name}", "", r.error, ""]

    return "\n".join(lines) + "\n"


def _fmt_elapsed(seconds: float) -> str:
    m = int(seconds) // 60
    s = int(seconds) % 60
    return f"{m}m{s:02d}s"
```

- [ ] **Step 2.4: Update `src/agentsec/reports/__init__.py`**

Replace the file content with:

```python
from .json_report import write_findings_json, write_threat_intel_snapshot
from .markdown import (
    render_combined_report,
    render_markdown_report,  # deprecated shim; removes in v0.3
    render_remote_report,
    render_server_audit_report,
)
from .multi_summary import TargetResult, render_multi_summary

__all__ = [
    "render_combined_report",
    "render_markdown_report",
    "render_multi_summary",
    "render_remote_report",
    "render_server_audit_report",
    "TargetResult",
    "write_findings_json",
    "write_threat_intel_snapshot",
]
```

- [ ] **Step 2.5: Run tests to confirm they pass**

```bash
.venv/bin/pytest tests/test_multi_summary.py -v
```

Expected: all tests PASS.

```bash
.venv/bin/pytest tests/ -x -q
```

Expected: all existing tests still PASS.

- [ ] **Step 2.6: Commit**

```bash
git add src/agentsec/reports/multi_summary.py src/agentsec/reports/__init__.py \
        tests/test_multi_summary.py
git commit -m "feat(reports): add TargetResult dataclass and render_multi_summary"
```

---

## Task 3: CLI Command — `_evaluate_one_target` + `multi_evaluate`

**Files:**
- Modify: `src/agentsec/cli.py`
- Create: `tests/integration/test_multi_evaluate.py`

- [ ] **Step 3.1: Write the failing integration tests**

Create `tests/integration/test_multi_evaluate.py`:

```python
"""Integration tests for `agentsec multi-evaluate`."""

from pathlib import Path

import pytest
from typer.testing import CliRunner

from agentsec.cli import app

runner = CliRunner()


@pytest.fixture
def multi_cfg(tmp_path: Path) -> Path:
    suite = tmp_path / "suite.yaml"
    suite.write_text(
        "- id: mt-1\n"
        "  category: ssrf\n"
        "  description: multi-evaluate smoke test\n"
        '  attack: "test"\n'
        "  expected_behavior: refusal\n"
        "  severity: low\n"
        "  judge_type: deterministic\n"
        "  assertions:\n"
        "    - type: response_not_contains_pattern\n"
        '      pattern: "secret"\n'
    )
    cfg = tmp_path / "multi.yaml"
    cfg.write_text(
        f"output_base: {tmp_path / 'multi-out'}\n"
        "targets:\n"
        "  - name: prod\n"
        "    remote:\n"
        "      adapter: stub-multi\n"
        "      target: http://prod.stub\n"
        f"      test_suites: [{suite}]\n"
        "  - name: staging\n"
        "    remote:\n"
        "      adapter: stub-multi\n"
        "      target: http://staging.stub\n"
        f"      test_suites: [{suite}]\n"
    )
    return cfg


def test_multi_evaluate_two_targets_creates_artifacts(multi_cfg, tmp_path):
    from agentsec.adapters import registry
    from agentsec.adapters.base import AgentAdapter, AgentResponse

    class _StubAdapter(AgentAdapter):
        def __init__(self, **_kw): pass

        async def send(self, message, session_id):
            return AgentResponse(content="ok", latency_ms=0.0, raw={}, tool_events=[])

    registry.register("stub-multi", _StubAdapter)
    try:
        result = runner.invoke(app, ["multi-evaluate", "--config", str(multi_cfg)])
    finally:
        registry._REGISTRY.pop("stub-multi", None)

    assert result.exit_code == 0, result.output
    out_base = tmp_path / "multi-out"

    # Per-target artifacts
    for name in ("prod", "staging"):
        assert (out_base / name / "combined-report.md").exists(), f"missing {name}/combined-report.md"
        assert (out_base / name / "findings.json").exists()
        assert (out_base / name / "threat-intel-snapshot.yaml").exists()
        assert (out_base / name / "audit-log.jsonl").exists()

    # Cross-target summary
    summary_path = out_base / "summary.md"
    assert summary_path.exists()
    summary = summary_path.read_text()
    assert "| prod |" in summary
    assert "| staging |" in summary
    assert "prod/combined-report.md" in summary
    assert "staging/combined-report.md" in summary


def test_multi_evaluate_one_error_still_writes_summary(multi_cfg, monkeypatch, tmp_path):
    """A target that errors at runtime is recorded in summary; exit code stays 0."""
    import agentsec.cli as cli_mod
    from agentsec.reports.multi_summary import TargetResult
    from agentsec.scoring import CombinedScore, CoverageTriple, EvaluationScore

    def _fake_evaluate_one_target(cfg, api_key, output_base):
        out = output_base / cfg.name
        out.mkdir(parents=True, exist_ok=True)
        if cfg.name == "staging":
            return TargetResult(cfg.name, out, None, "SSH: connection refused", 3.0)
        score = EvaluationScore(
            category_scores=[],
            remote_score=90.0, server_score=None,
            combined=CombinedScore(
                score=90.0, weight_remote_used=1.0, weight_server_used=0.0, note=None,
            ),
            coverage=CoverageTriple(
                planned=1, executed=1, scored=1, error=0,
                execution_coverage=1.0, verdict_coverage=1.0, error_rate=0.0,
            ),
            low_coverage=False, grade="A", deduped_findings=[],
        )
        (out / "combined-report.md").write_text("# Combined")
        return TargetResult(cfg.name, out, score, None, 10.0)

    monkeypatch.setattr(cli_mod, "_evaluate_one_target", _fake_evaluate_one_target)
    result = runner.invoke(app, ["multi-evaluate", "--config", str(multi_cfg)])

    assert result.exit_code == 0, result.output
    out_base = tmp_path / "multi-out"
    summary = (out_base / "summary.md").read_text()
    assert "1 error" in summary
    assert "| staging | ERROR |" in summary
    assert "SSH: connection refused" in summary
    assert "staging/combined-report.md" not in summary


def test_multi_evaluate_bad_config_exits_2(tmp_path):
    bad_cfg = tmp_path / "bad.yaml"
    bad_cfg.write_text("not_a_mapping: true\n")
    result = runner.invoke(app, ["multi-evaluate", "--config", str(bad_cfg)])
    assert result.exit_code == 2


def test_multi_evaluate_all_errors_exits_2(multi_cfg, monkeypatch, tmp_path):
    """All targets erroring → exit code 2."""
    import agentsec.cli as cli_mod
    from agentsec.reports.multi_summary import TargetResult

    def _always_error(cfg, api_key, output_base):
        out = output_base / cfg.name
        out.mkdir(parents=True, exist_ok=True)
        return TargetResult(cfg.name, out, None, "down", 1.0)

    monkeypatch.setattr(cli_mod, "_evaluate_one_target", _always_error)
    result = runner.invoke(app, ["multi-evaluate", "--config", str(multi_cfg)])
    assert result.exit_code == 2
```

- [ ] **Step 3.2: Run tests to confirm they fail**

```bash
.venv/bin/pytest tests/integration/test_multi_evaluate.py -v
```

Expected: `Error: No such command 'multi-evaluate'`.

- [ ] **Step 3.3: Add `_evaluate_one_target` and `multi_evaluate` to `cli.py`**

Add this import at the top of `cli.py` (alongside existing imports):

```python
import time
from concurrent.futures import ThreadPoolExecutor, as_completed

from .reports.multi_summary import TargetResult, render_multi_summary
```

Add this private helper anywhere after `_evaluate_server` in `cli.py`:

```python
def _evaluate_one_target(
    cfg: "OpenClawTargetConfig",
    api_key: str | None,
    output_base: Path,
) -> "TargetResult":
    """Run a full evaluate pipeline for one target. Safe to call from a thread.

    Never raises — all exceptions are caught and surfaced in TargetResult.error.
    """
    out = cfg.output.dir
    out.mkdir(parents=True, exist_ok=True)
    start = time.monotonic()
    try:
        remote_results, planned_count = (
            _evaluate_remote(cfg, api_key, out) if cfg.remote is not None
            else ([], 0)
        )
        try:
            server_findings, server_meta = (
                _evaluate_server(cfg, out) if cfg.server_audit is not None
                else ([], None)
            )
        except AuthorizationError as exc:
            raise RuntimeError(f"authorization rejected: {exc}") from exc

        score = score_evaluation(
            remote_results=remote_results,
            server_findings=server_findings,
            planned_count=planned_count,
            config=cfg,
        )

        if cfg.remote is not None:
            (out / "remote-report.md").write_text(
                render_remote_report(cfg.name, remote_results, score)
            )
        if cfg.server_audit is not None and server_meta is not None:
            (out / "server-audit-report.md").write_text(
                render_server_audit_report(
                    server_meta["report"],
                    profile_name=server_meta["profile_name"],
                    remote_home=server_meta["remote_home"],
                    low_assurance=server_meta["low_assurance"],
                    server_score=score.server_score,
                )
            )
        (out / "combined-report.md").write_text(
            render_combined_report(
                cfg.name, score,
                remote_present=cfg.remote is not None,
                server_present=cfg.server_audit is not None,
            )
        )
        write_findings_json(score.deduped_findings, out / "findings.json")

        referenced_refs: set[str] = set()
        for f in score.deduped_findings:
            referenced_refs.update(f.threat_refs)
        threat_intel_path = (
            (cfg.server_audit.ioc_db / "threat_intel.yaml")
            if cfg.server_audit is not None
            else Path(__file__).parent / "audit" / "ioc" / "threat_intel.yaml"
        )
        write_threat_intel_snapshot(
            referenced_refs, threat_intel_path, out / "threat-intel-snapshot.yaml"
        )
        (out / "audit-log.jsonl").touch(exist_ok=True)

        return TargetResult(
            name=cfg.name, output_dir=out, score=score,
            error=None, elapsed_seconds=time.monotonic() - start,
        )
    except typer.Exit as exc:
        return TargetResult(
            name=cfg.name, output_dir=out, score=None,
            error=f"aborted (exit code {exc.exit_code})",
            elapsed_seconds=time.monotonic() - start,
        )
    except Exception as exc:  # noqa: BLE001
        return TargetResult(
            name=cfg.name, output_dir=out, score=None,
            error=str(exc) or repr(exc),
            elapsed_seconds=time.monotonic() - start,
        )
```

Add this CLI command after the `evaluate` command:

```python
@app.command("multi-evaluate")
def multi_evaluate(
    config: Path = typer.Option(..., help="Path to multi-target.yaml"),
    output_base: Path = typer.Option(
        None, "--output-base",
        help="Output base directory (overrides config.output_base)",
    ),
    api_key: str = typer.Option(
        None, envvar="AGENTSEC_JUDGE_API_KEY",
        help="Anthropic API key for LLM/hybrid judges",
    ),
) -> None:
    """Evaluate multiple OpenClaw targets in parallel."""
    from datetime import datetime, timezone

    from .config import MultiTargetConfig

    try:
        cfg = MultiTargetConfig.from_yaml(config)
    except Exception as exc:  # noqa: BLE001
        console.print(f"[red]config error:[/red] {exc}")
        raise typer.Exit(code=2)

    base = output_base if output_base is not None else cfg.output_base
    base.mkdir(parents=True, exist_ok=True)

    target_cfgs = [t.to_openclaw_config(base) for t in cfg.targets]
    max_w = cfg.effective_max_parallel
    console.print(
        f"[bold]multi-evaluate:[/bold] {len(target_cfgs)} targets, "
        f"max_parallel={max_w}"
    )

    results_map: dict[str, TargetResult] = {}
    with ThreadPoolExecutor(max_workers=max_w) as pool:
        futures = {
            pool.submit(_evaluate_one_target, tc, api_key, base): tc.name
            for tc in target_cfgs
        }
        for future in as_completed(futures):
            name = futures[future]
            result = future.result()
            results_map[name] = result
            elapsed = f"{int(result.elapsed_seconds) // 60}m{int(result.elapsed_seconds) % 60:02d}s"
            if result.error is None:
                assert result.score is not None
                console.print(
                    f"  [green]done[/green] {name} "
                    f"(grade={result.score.grade}, {elapsed})"
                )
            else:
                console.print(f"  [red]ERROR[/red] {name}: {result.error}")

    ordered = [results_map[tc.name] for tc in target_cfgs]
    generated_at = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
    (base / "summary.md").write_text(render_multi_summary(ordered, generated_at=generated_at))

    error_count = sum(1 for r in ordered if r.error is not None)
    console.print(
        f"[green]wrote[/green] {base}/summary.md "
        f"({len(ordered) - error_count}/{len(ordered)} succeeded)"
    )
    if error_count == len(ordered):
        raise typer.Exit(code=2)
```

- [ ] **Step 3.4: Run tests to confirm they pass**

```bash
.venv/bin/pytest tests/integration/test_multi_evaluate.py -v
```

Expected: all 4 tests PASS.

- [ ] **Step 3.5: Run the full test suite**

```bash
.venv/bin/pytest tests/ -x -q
```

Expected: all tests PASS. Confirm count is `713 + new tests`.

- [ ] **Step 3.6: Lint check**

```bash
.venv/bin/ruff check src/ --fix
```

Expected: no errors.

- [ ] **Step 3.7: Commit**

```bash
git add src/agentsec/cli.py tests/integration/test_multi_evaluate.py
git commit -m "feat(cli): add multi-evaluate command with parallel target execution"
```

---

## Task 4: Fixtures, Changelog, Roadmap

**Files:**
- Create: `tests/fixtures/multi-target.example.yaml`
- Modify: `CHANGELOG.md`
- Modify: `ROADMAP.md`

- [ ] **Step 4.1: Create `tests/fixtures/multi-target.example.yaml`**

```yaml
# Example multi-target config for agentsec multi-evaluate
# Usage: agentsec multi-evaluate --config multi-target.example.yaml

output_base: ./multi-report/
max_parallel: 4

targets:
  - name: openclaw-prod
    remote:
      adapter: openclaw-gateway
      target: https://prod.example.com:18789
      token_env: OPENCLAW_PROD_KEY
      test_suites: [test_suites/openclaw/]
      concurrency: 5
    server_audit:
      host: prod.example.com
      user: ubuntu
      key: ~/.ssh/id_rsa
    authorization:
      consent_file: ./AUTHORIZATION-prod.txt
    scoring:
      weight_remote: 0.5
      weight_server: 0.5

  - name: openclaw-staging
    remote:
      adapter: openclaw-gateway
      target: https://staging.example.com:18789
      token_env: OPENCLAW_STAGING_KEY
      test_suites: [test_suites/openclaw/]
      concurrency: 3
    authorization:
      consent_file: ./AUTHORIZATION-staging.txt
    # output.dir defaults to ./multi-report/openclaw-staging/
```

- [ ] **Step 4.2: Add CHANGELOG entry**

Prepend a new version block at the top of `CHANGELOG.md` (after the `# Changelog` heading):

```markdown
## v0.6.0 (2026-05-04)

### Added
- `agentsec multi-evaluate --config multi-target.yaml` — evaluate multiple
  OpenClaw instances in parallel via `ThreadPoolExecutor`. Each target produces
  the standard six artifacts under `output_base/<name>/`; a cross-target
  `summary.md` (score table + per-target links + error details) is written last.
- `MultiTargetConfig` + `TargetEntry` Pydantic models in `config.py`
  (`extra="forbid"` enforced, duplicate target names rejected).
- `TargetResult` dataclass and `render_multi_summary` renderer in
  `src/agentsec/reports/multi_summary.py`.
```

- [ ] **Step 4.3: Tick off ROADMAP item**

In `ROADMAP.md`, under `## Phase 3 — OpenClaw 持续演进`, replace:

```
- [ ] 多目标并行评估（一次 `agentsec evaluate` 跑多个 OpenClaw 实例）
```

with:

```
- [x] 多目标并行评估（`agentsec multi-evaluate`，ThreadPoolExecutor 并行） — v0.6.0，2026-05-04
```

- [ ] **Step 4.4: Run full test suite one final time**

```bash
.venv/bin/pytest tests/ -q
```

Expected: all tests PASS.

- [ ] **Step 4.5: Commit**

```bash
git add tests/fixtures/multi-target.example.yaml CHANGELOG.md ROADMAP.md
git commit -m "chore: add multi-evaluate fixture, CHANGELOG v0.6.0, tick ROADMAP"
```

---

## Self-Review Against Spec

| Spec section | Covered by task |
|---|---|
| §2 CLI flags `--config`, `--output-base`, `--api-key` | Task 3 |
| §3.1 YAML schema with `output_base`, `max_parallel`, `targets` list | Task 1 |
| §3.2 `TargetEntry.to_openclaw_config`, `MultiTargetConfig.effective_max_parallel` | Task 1 |
| §4 `ThreadPoolExecutor`, `_evaluate_one_target` never raises | Task 3 |
| §5 Output layout: per-target subdir + `summary.md` | Task 3 |
| §6 Summary format: table, Reports section, Errors section | Task 2 |
| §6 `INSUFFICIENT_COVERAGE` → `LOW_COV` in table | Task 2 |
| §7 `render_multi_summary` pure renderer, no I/O | Task 2 |
| §8 `AuthorizationError` → `RuntimeError`, `typer.Exit` caught | Task 3 |
| §8 Exit 0 when ≥1 target succeeded; exit 2 when all errored | Task 3 |
| §9 Unit tests: `MultiTargetConfig` parse, `TargetEntry.to_openclaw_config` | Task 1 |
| §9 Unit tests: summary renderer (2 success, 1 error, LOW_COV, finding counts) | Task 2 |
| §9 Integration: two targets artifacts, one-error-summary, bad-config exit 2, all-error exit 2 | Task 3 |
| §10 File list complete | All tasks |
| §11 No cross-target diff, no aggregate scoring, no streaming progress | N/A — excluded |
| Example YAML fixture | Task 4 |
| CHANGELOG + ROADMAP | Task 4 |
