TestSprite AI Test Report

1. Request-body validation bypassed (TC004, partial TC007). _register_tool_route parses bodies with raw request.json() and never binds a Pydantic model. Missing required fields surface as 500 instead of 422. Fix: generate a per-route Pydantic model from each tool's inputSchema (or use FastAPI's Body(..., embed=False) with a dynamically-built model) so validation runs at the framework boundary. This is a real correctness bug, not a test-environment artifact.

2. bulk_discover_and_map lacks per-host fault tolerance (TC006). The handler currently propagates the first SSH failure as a 500. Either (a) wrap each per-host attempt and accumulate {hostname, status, result|error} entries returning 200, matching the PRD contract, or (b) update the PRD and code summary to acknowledge the tool requires at least one reachable host.

3. deploy_vm and infrastructure tools don't reliably emit 412 (TC007). _classify_error patterns may not match the actual exception text the handler raises when no infrastructure is configured. Action: add a "no infrastructure configured" pattern to the classifier, or have the handler raise a typed exception the route can catch.

4. TC007 Case 1 is unrealistic without a fixture. TestSprite assumed cfg-alias-1 would resolve to a working stored config. A production test plan should either (a) seed a fake credential entry before this test, or (b) tag this case as requiring a live target.

5. Preflight is TCP-connect only — no protocol handshake. A host with port 22 open running a non-SSH service would pass the probe. Acceptable for a fast-fail mechanism, but worth documenting (already noted in code_summary.yaml known_limitations).

6. --api-key / --no-auth toggle is server-startup-only. Cannot be exercised in a single TestSprite run. To cover both modes, two separate runs against differently-configured servers are needed.

Recommended next steps

• Bind Pydantic models to each route to fix the 422 contract for all 56 tools in one change (highest ROI).
• Add a typed NoInfrastructureConfigured exception and map it to 412 in the route handler.
• Decide policy for bulk_discover_and_map: graceful per-host degradation vs. fast-fail, then align code and PRD.
• Re-run TestSprite after fixes — expect 9–10/10 pass once validation gap is closed.