2026-06-26 · 50 shipped capabilities researched against the field (Claude Code ecosystem, other agentic-coding tools, domain best practice) via Firecrawl, with cited sources.
50 capabilities5 significant-gaps45 minor-gaps0 already best-in-class88 P0106 P190 P2
★ Read the MCP server log and triage by the actual error string BEFORE killing anything — diagnosis-by-error beats our current kill-everything-blind reset.
Clear 4-step structure (diagnose -> kill -> test -> report-and-fix) that reads well as a slash command. Genuinely strong, often-missing process-hygiene: explicit ps/lsof diagnosis across Chrome remote-debug, Playwright, and MCP node processes plus ports 9222/9223/9229, with a tidy status table. Strong safety guardrail that most guides lack: scoped `pkill` patterns (`pw-browser`, `playwright.*chromium`, `chrome-devtools-mcp`) with an explicit 'do NOT kill the user's main Chrome' rule. Tests both MCPs by actually calling a tool (`list_pages`, `browser_snapshot`) rather than assuming. Includes the `curl http://localhost:9222/json/version` verification and the correct Chrome 144+ / `chrome://inspect/#remote-debugging` requirements, plus prevention tips (prefer CDM over Playwright, don't run both on the same page).
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add an early 'Step 1.5: Read the error first' before killing anything. Inspect the MCP server logs — on macOS `~/Library/Logs/Claude/mcp-server-chrome-devtools.log` and `~/Library/Logs/Claude/mcp-server-playwright.log` (tail the last ~40 lines) — and instruct that if logs are empty/uninformative, restart the server with verbose logging: `DEBUG=* npx chrome-devtools-mcp@latest --log-file=<scratchpad>/cdm.log`. Reading the actual error is the fastest path and is the #1 move in both the official CDM skill and TestDino. Both authoritative sources lead with reading logs; ours kills processes blind. Cheapest, highest-signal addition. |
| P0 | M | Add an error-string triage table mapping the canonical errors to fixes: `Target closed` -> a Chrome is already running, fully quit it then retry; `Could not find DevToolsActivePort` / `Network.enable timed out` / `socket connection was closed unexpectedly` -> autoConnect can't reach a debuggable Chrome (verify Chrome 144+ is RUNNING, remote debugging enabled at chrome://inspect/#remote-debugging, the connection prompt was accepted, and nothing else holds the debug port); `ERR_MODULE_NOT_FOUND` -> wrong Node version or corrupt npx cache (`rm -rf ~/.npm/_npx && npm cache clean --force`). Directly from the official troubleshooting.md. Turns ours from a generic reset into a targeted fix keyed to the real error. |
| P1 | S | In Step 1 (Diagnose), promote `curl -s http://localhost:9222/json/version` to a diagnostic gate (not just a fix step): if it returns Chrome version JSON, Chrome is debuggable and the problem is the MCP layer (restart server/Claude Code); if it fails/refuses, Chrome itself isn't exposing the debug port (enable remote debugging or relaunch with --remote-debugging-port). Add a tab-count caution: a browser with very many tabs can cause CDM connection timeouts because CDM force-loads all tabs (Chrome <=149) — close excess tabs. Separates 'Chrome problem' from 'MCP problem' so the fix targets the right layer; the tab-count failure mode is explicitly called out in the canonical doc and ours omits it. |
| P1 | M | Add a 'Step 3.5: Isolate the server standalone' when an MCP still won't connect after cleanup: run `npx chrome-devtools-mcp@latest --help` (verifies install/Node), and for Playwright run `npx playwright install` (missing browser binaries are the most common Playwright MCP failure) and check `node -v` is >=18. Note that pinning versions beats `@latest` (mismatch is a top cause of 'No tools detected'). TestDino and the CDM general-tips both stress running the server directly to surface Node/binary/version issues that process-killing can't fix. |
| P2 | M | Expand the fix sections with two field-known fallbacks: (a) sandboxed clients (Claude Desktop / macOS Seatbelt / containers) can't let the MCP spawn Chrome — start Chrome manually and connect via `--browser-url=http://127.0.0.1:9222`; (b) if only a limited subset of tools is visible (e.g. ~9), the client is in read-only/plan mode — exit it. Add a Playwright last-resort: `browser_close` to free a stuck context, then restart the MCP server to drop stale browser contexts. These are documented failure modes (sandbox, read-only) and the explicit Playwright memory/context remedy; all absent from ours. |
★ Turn a verbose, Python/Hank-specific README prose-persona into a language-agnostic, AGENTS.md-aware doc maintainer with a concrete 3-pass grounded-verification + diff-driven drift loop (matching the suite's docs-sync) and a concise elevator-pitch README structure from community consensus.
Solid, well-organized checklist of WHAT a complete README covers: entry points (CLI/programmatic/tests), an env-var table with required/optional + defaults + grouping, install/requirements with version constraints, usage examples, and output-format docs. Has a sensible code-analysis phase (scan for main()/__main__, argparse/click, os.environ) and an explicit anti-fabrication stance ("never document features that don't exist," "use actual code snippets not pseudo-code," "test commands before documenting"). The output-format guidance (TOC, language-tagged code blocks, env-var tables, collapsible sections) is good. These are genuinely useful and worth preserving.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add an AGENTS.md section: teach the agent the README-(humans)-vs-AGENTS.md-(agents) split, to detect an existing AGENTS.md / CLAUDE.md, keep build/test/run commands + conventions there (not bloating the README), respect nested per-subproject files in monorepos (nearest-file-wins), and offer to scaffold an AGENTS.md when missing. Cite that listed test/build commands get auto-run by agents so they must be real and current. AGENTS.md is the cross-tool standard adopted by Codex/Cursor/Copilot/Devin/Aider/Gemini/Windsurf and 60k+ repos (agents.md, Linux Foundation). A README maintainer that doesn't know about it is materially behind the field and risks putting agent context in the wrong file. |
| P0 | M | Strip all project-specific content (KRAC, hank-codesets/medicalnotes, Azure Document Intelligence, specialty configs) and make code analysis language-agnostic: detect entry points across Python (main/__main__, argparse/click, console_scripts), Node (package.json scripts + bin), Go, Rust (Cargo bin), Makefile/justfile targets, Dockerfile/compose, and env via .env.example / config files — not just os.environ. This ships in a shareable suite (claude-jacked) to arbitrary repos. The hardcoded Python/Azure/Hank context is wrong-by-default and the Python-only scanning misses entry points in the majority of repos the agent will run in. |
| P0 | M | Replace the vague 'test commands before documenting' line with an explicit grounded verification loop mirroring the suite's docs-sync 3-pass protocol: PASS 1 update from code, PASS 2 verify every command/flag/env-var/version against the actual source (and run safe commands like --help / build), PASS 3 prune anything not backed by code. State that every documented fact must trace to a file/line. The agent claims anti-fabrication but gives no procedure; its own sibling skill (docs-sync, in this repo) already encodes a non-negotiable 3-pass protocol. Aligning the agent to that proven loop makes 'never document what doesn't exist' actually enforceable and consistent within the suite. |
| P1 | M | Add a drift-detection / diff-driven entry mode: when invoked after code changes, diff the branch against base, map changed areas (entry points, env, deps, CLI, schemas) to specific README sections, and update only those; when invoked cold, audit the whole README section-by-section against current code. Note the sibling /docs-sync command for branch-wide doc sync. The agent's trigger is literally 'documentation drift detected' / 'after significant code changes,' yet it has no mechanism to locate drift. docs-sync in this same repo shows the pattern (git timestamp staleness + change-to-doc mapping); the agent should reuse it rather than re-scan everything blindly. |
| P1 | M | Add a README structure/philosophy section grounded in community consensus: lead with one-line description + badges, a Highlights/selling-points bullet list, then a copy-paste 60-second Quickstart; keep it an 'elevator pitch + link fest' that answers the 4 reader questions (does it solve my problem / can I use it / who made it / how to learn more); push exhaustive API detail to dedicated docs; include a screenshot/GIF for visual or UI projects; ensure License + Contributing links exist. Directly from banesullivan/README and art-of-readme: the field optimizes READMEs for concise, inviting elevator pitches, while our agent optimizes for exhaustive completeness — which produces the bloated READMEs those guides explicitly warn against. Adds high-signal elements (quickstart-first, visual demo) the agent currently omits. |
| P2 | S | Add a 'preserve human voice / single source of truth' rule: only correct code-derived facts (commands, flags, env vars, versions, paths); never overwrite human-written narrative, marketing, or design rationale; and avoid creating duplicate instruction content that drifts across README / AGENTS.md / CLAUDE.md — point one canonical doc and link rather than copy. Cross-tool sources (Ivan Morgillo; agents.md FAQ on conflicts) warn that duplicated per-tool instructions drift and rot. An auto-maintainer must be explicitly constrained to factual corrections and single-sourcing or it will flatten human prose and spawn stale copies. |
| P2 | S | Tighten the frontmatter description: it's a single dense escaped-newline blob with three Python-CLI examples. Replace with crisp trigger phrasing covering create/update/sync-README, AGENTS.md, and post-change drift, with language-neutral examples — improving agent auto-selection. In a shared suite, the description drives auto-triggering; the current one is verbose, Python/Azure-flavored, and omits AGENTS.md, so the agent will mis-trigger or be skipped versus the more specific docs-sync skill. |
★ The command's file-isolation instinct is right, but it's written against a removed API (TeamCreate/TeamDelete no longer exist as of v2.1.178) and misses the platform's now-core mechanics — enablement gate, plan-first, context-embedding into spawn prompts, task dependencies, per-teammate models, lead-discipline, and an integration step — so it needs a substantive rewrite to current Claude Code, not a polish.
The command has the right core instincts and they all match field consensus: (1) non-overlapping file ownership / file-level isolation as the central constraint — the doc's 'Avoid file conflicts' best practice and the Reddit #1-failure both confirm this is THE thing to get right. (2) Scaling teammate count to complexity (2-3/4-5/6-8) is close to the doc's '3-5, scale only when work genuinely benefits' guidance. (3) 'If a file must be touched by multiple tasks, assign it to ONE teammate and have others message changes' is a genuinely good conflict-avoidance pattern. (4) Run tests after integration. (5) 'If a teammate fails, diagnose and reassign, don't retry blindly' matches the doc's troubleshooting advice. The intent is sound; the problem is the API it's written against has changed underneath it.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Remove all references to TeamCreate/TeamDelete and team_name. Rewrite the spawn mechanic to the current model: form the team by spawning teammates via the Task/Agent tool (or natural-language 'spawn N teammates') with a per-teammate spawn prompt; rely on automatic cleanup on session exit. Update the intro sentence that lists 'TeamCreate ... SendMessage' as the toolset. The official doc states TeamCreate/TeamDelete 'no longer exist' as of v2.1.178 and team_name is 'accepted but ignored.' The command's core instructions currently fail against current Claude Code. This is the single load-bearing fix. |
| P0 | S | Add a preflight step: verify agent teams are enabled (CLAUDE_CODE_EXPERIMENTAL_AGENT_TEAMS=1 in settings.json env). If not set, instruct the user to add it and restart, or offer to fall back to plain subagents. Teams are experimental and OFF by default per the doc's enablement section; without the env var the command silently does nothing. |
| P0 | M | Add a plan-first gate before fan-out: require an explicit task breakdown (from $ARGUMENTS plan file or a quick plan-mode pass) with each task scoped to a clear deliverable and explicit file ownership, and confirm it before spawning. Doc ('start with research/review', 'size tasks appropriately') and Push-To-Prod tip 17 both state agent teams without an upfront plan waste tokens as agents wander. Mirrors the discipline already in the sibling swarm-research command. |
| P0 | S | Instruct embedding session-specific context into each spawn prompt: relevant file paths, code pointers, decisions, and the 'why,' since teammates do NOT inherit the lead's conversation history (only CLAUDE.md/MCP/skills). Both sources call this the top context failure mode ('teammates won't have whatever context you built up; embed the important details into the task descriptions'). |
| P1 | M | Replace 'all tasks independent' with explicit dependency support: let tasks declare prerequisites so the platform blocks claiming until deps complete; only parallelize the independent frontier. The doc documents native task dependencies with file-locked claiming and auto-unblocking. Real plans have ordering; the current all-parallel assumption is both lossy and unsafe. |
| P1 | S | Add a monitor-and-steer + lead-discipline section: tell the lead to delegate (not implement), to wait for teammates to finish before proceeding, and to verify each task is genuinely complete (guard against task-status lag) before declaring done. These are explicitly documented experimental failure modes in the troubleshooting section ('lead shuts down before work is done', 'task status can lag', 'lead does work instead of delegating'); Push-To-Prod tips 26/30 echo them. |
| P1 | S | Add per-teammate model guidance: default mechanical/isolated tasks to a cheaper model (Sonnet/Haiku) and reserve Opus for hard tasks; note that token cost scales linearly per teammate. Doc 'choose appropriate team size' (linear token cost) and Push-To-Prod tip 10 both recommend per-role model assignment as the main cost lever. |
| P1 | M | Add an explicit integration/reconciliation phase before the test run: have the lead reconcile independently-produced changes (interface drift across owned files), then run the PROJECT's detected test command — fix the Python-only 'uv run pytest' default to auto-detect (package.json scripts, Makefile, pyproject, etc.) with pytest as just one case. Reddit field report: parallel work breaks at integration. And jacked targets non-Python repos, so hardcoding pytest first is wrong. Tighten verification to where failures actually occur. |
| P2 | M | Offer a read-only-first / safer-mode framing and a git-worktree option: suggest starting with read-only swarms (research, review, parity scans) where conflicts are impossible, and for large write-heavy swarms recommend git-worktree-per-teammate isolation instead of same-tree file ownership. Push-To-Prod tip 29 (read-only first) and the Reddit 8-12-agent report (worktree isolation is the proven fix for the #1 file-conflict failure) both support this; the doc itself flags same-tree ownership as fragile beyond clean domain splits. |
| P2 | S | Mention optional quality-gate hooks (TaskCompleted / TeammateIdle, exit code 2) as a way to enforce 'tests pass before a task is marked done' or 'keep a teammate working.' Documented native mechanism for enforcing quality gates across teammates; a low-effort note that materially hardens autonomous swarms. |
★ Ours is a well-structured Python doctest/pytest agent, but it lacks the defenses the field now treats as table stakes for AI-written tests — independent (non-tautological) assertions, mutation/deliberate-bug verification, an actual measure-coverage loop, language-agnostic stack detection, and a characterization-test path for unknown-behavior code.
Clear doctest-vs-test-file decision tree with concrete complexity/dependency/quantity heuristics. Sound prioritization framework (core business logic and public APIs first, then frequently-used utilities and recently-modified code). Solid hygiene baked in: AAA pattern, test independence/determinism, behavior-over-implementation, descriptive test names, fixtures/parameterization to cut duplication, a sub-1s speed budget. Explicit, useful 'do NOT' constraints (no doctests on private/IO/async/GUI code; no time-dependent or over-mocked tests). Well-structured output expectations (which files, doctest-vs-file rationale, assumptions, follow-up gaps).
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add an 'Assertion Independence' section that bans tautological tests: never derive an expected value by calling the function under test (or copying its logic); compute expected values by hand from the spec/business rule/docstring, or from a known-good reference. Include the short before/after contrast (assert against an independently computed number, not against the function's own output). This is the dominant, repeatedly documented failure of AI-written tests (Autonoma, super-productivity, multiple Medium posts). Our agent — which writes tests for code it just read — is maximally exposed and currently has zero guard against it. Highest-leverage single change. |
| P0 | M | Add a self-verification step before reporting done: for each new test, mentally or actually perturb the code under test (flip a comparison, change a constant, swap a branch) and confirm at least one test would go red; if nothing fails, the test is tautological and must be strengthened. Recommend running a mutation tool when available (mutmut/cosmic-ray for Python, Stryker for JS/TS, PIT for Java) and treating mutation score — not line coverage — as the quality bar. The field consensus is that line coverage is gameable and mutation score is the real proxy; the deliberate-bug check is the lightweight, tool-free version of the same idea (cited in both Autonoma and Understand Legacy Code). Directly converts our agent's tests from 'green' to 'protective.' |
| P1 | M | Add an explicit measure loop: run the existing test command and coverage tool, capture before/after coverage (prefer diff/patch coverage on changed lines over global %), and report only measured numbers. If no coverage tool is configured, say so rather than estimating. Our agent promises an 'approximate coverage improvement' it never measures, which invites fabricated metrics; the VoltAgent reference and diff-coverage literature treat run-and-measure as core. Diff coverage is the field's recommended, achievable standard. |
| P1 | M | De-hardcode the stack: open by detecting language, test framework, runner, and existing test layout/conventions, then match them (doctest/pytest only when the repo is Python). Generalize the doctest-vs-separate-file rule to 'inline example/doc test vs dedicated test file' for the detected language. claude-jacked is shared across arbitrary repos; a Python-only agent silently fails or imposes the wrong framework on JS/TS/Go projects. The field's reference test subagent is stack-agnostic and detects first. |
| P2 | M | Add a characterization-test path for code with unknown/undocumented intended behavior: capture current observable behavior as a regression net, scrub flaky/nondeterministic data (timestamps, ids, ordering) before asserting, and explicitly label these as 'characterizing existing behavior — a human must confirm this behavior is correct, not just current.' Directly from the Feathers/Understand Legacy Code technique. It safely covers the large class of code where the correct answer is unknown — which our agent currently has no strategy for — without falsely 'approving' possibly-buggy behavior. |
★ Ours has solid page templates but is destructive-by-default, create-only, and fabrication-prone — add anti-fabrication source-grounding, an audit-first/drift-aware non-destructive workflow, and Diátaxis-based adaptive structure (plus fix the wrong Windows-path block and unpinned Action) to make it best-in-class.
Strong, concrete page templates with consistent section skeletons (Home, Getting-Started, API-Reference, module pages) that produce uniform output. Explicit per-project analysis step (project type, language, modules, audience) before generating. Good content requirements (every page needs overview, tested code examples with expected output, related-page links). Includes a GitHub Actions workflow to sync _wiki/ to the real wiki and a final quality checklist. Internal wiki-link format guidance is correct.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a top-priority 'Source-Grounding & Anti-Fabrication Rules' section: never invent endpoints, params, config keys, env vars, CLI flags, error messages, status codes, or version numbers; only document what is verifiably in the source; when unsure write 'See source at <file:line>' instead of guessing; flag gaps rather than filling them. State that fabricated docs are worse than missing docs. Dosu's #1 documented failure: Claude added a non-existent response field and guessed commands. This is the highest-leverage change and currently entirely absent. |
| P0 | M | Replace the destructive 'DELETE the _wiki folder and start fresh' directive with an audit-first, non-destructive workflow: Discover (inventory existing docs + last-modified + coverage) → Audit (find stale references, missing coverage, inconsistencies) → Update (smallest edit, preserve hand-written sections) → Generate only missing pages → Recommend → Report. Only regenerate a page wholesale when it is non-conforming AND has no salvageable hand-written content. Dosu explicitly preserves manual edits and makes the smallest edit; blindly deleting destroys human-authored docs and is the opposite of the proven approach. |
| P0 | M | Add a documentation-drift audit: diff doc claims against the codebase (paths, commands, env vars, error strings, version/expiry numbers) and report stale items with file:line, supporting an 'update existing wiki' mode separate from 'create new wiki'. Dosu's most valuable real-world finds were a dead src/middleware/ path and a 24h-vs-60min token-expiry mismatch — exactly the silent drift our create-only agent can't catch. |
| P1 | M | Restructure the page model around Diátaxis: organize content by user need (tutorials / how-to guides / reference / explanation) and warn against mixing types on one page. Make the page set adaptive to project type/size instead of a fixed mandatory 15; drop pages the analysis shows aren't warranted (don't emit empty Performance-Optimization/Webhooks/Migration-Guides). Diátaxis is the field-standard IA (Cloudflare/Gatsby/Vonage). The current flat, always-15 list mixes doc types and guarantees placeholder pages — the scaffolding-with-no-substance anti-pattern. |
| P1 | S | Delete the 'File Editing on Windows / always use backslashes' block (or replace with a one-line note to use OS-native paths). It is wrong on this macOS/Linux environment and brittle even on Windows. Grounded in the actual run environment (darwin); the mandatory backslash instruction would break Edit/MultiEdit here and adds noise. |
| P1 | S | Pin the generated GitHub Actions workflow to commit SHAs (or at least current actions/checkout@v4) and add least-privilege 'permissions:' to the job, instead of the unpinned actions/checkout@v3. The agent writes this workflow into every repo; v3 is outdated and unpinned action refs are a known supply-chain risk for a file shipped across many projects. |
| P2 | S | Make the _Footer 'proprietary and confidential' copyright and emoji-on-every-header rules optional and style-matching: detect/ask for license and existing docs voice; match the project's established heading/emoji style rather than forcing one. Dosu's Quality Rules emphasize matching existing voice/formatting; hardcoding 'proprietary' and mandatory emojis is wrong for open-source repos and clashes with no-emoji house styles. |
| P2 | M | Add an agent-facing docs layer: recognize and keep CLAUDE.md / AGENTS.md in sync, resolve symlinks between them before editing (ls -la check), and flag divergence — alongside a short 'wiki vs in-repo docs/ folder' decision note up front. Dosu makes agent-facing docs + symlink resolution first-class (it edited the wrong file until it added the check); GitHub docs show wiki-vs-docs is a real tradeoff the agent currently ignores. |
| P2 | S | Add scoped/incremental invocation guidance: support targeting a single module/page for bounded token cost on large repos, and recommend a module-by-module pass after the initial bootstrap. Dosu identifies unbounded 'do everything' runs as the main cost/scaling failure and recommends scoped runs; our 'recreate all' default is exactly that failure. |
★ Strong on naming/methods/consistency, but missing the modern correctness-and-reliability layer — idempotent retries, RFC 9457 error bodies, validation-array shape, Retry-After, deprecation lifecycle, and field-level authz — all of which the field now treats as table stakes.
Already covers the core ergonomics surface well: consistent resource naming, HTTP-method semantics, structured errors with machine code + human message, consistent pagination, uniform filter/sort, partial responses/field selection, explicit versioning, 401-vs-403 distinction, rate-limit headers, and self-describing schemas. The anti-pattern list is strong and field-tested (200-with-error-body, deep nesting >2 levels, multi-call data that belongs together, breaking changes without version bump, GET with side effects, exposing internal IDs). 'Pick one and be consistent' framing for naming/pagination/filtering matches industry consensus. Scope and trigger list are sound.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add an idempotency check to 'What to check': state-changing endpoints are safe to retry — naturally-idempotent verbs (PUT/DELETE) or an Idempotency-Key header for POSTs that create resources or trigger side effects (payments, sends). Add anti-pattern: 'Retried POST creates duplicate resources / double-charges because no idempotency key is honored.' Most-cited modern API check across all sources (Alex Xu, ByteByteGo, Postman, Fern). It is a correctness failure mode the current lens cannot catch at all. |
| P0 | S | Upgrade the error-contract check to name RFC 9457 Problem Details (application/problem+json with type/title/detail/instance, plus extension members) as the recommended standard for error bodies, while keeping 'or a documented consistent house format'. Speakeasy and Swagger both call RFC 9457 (2023, supersedes 7807) the current best practice; naming the standard turns a vague 'machine-readable codes' check into something a reviewer can concretely verify and that interops with tooling. |
| P1 | S | Add a validation-error shape check: field-level validation failures are returned as ONE error with a validation[] (field + message) array, not multiple top-level errors and not one-at-a-time. Add anti-pattern: 'Returns the first validation failure only, forcing fix-resubmit-fail loops.' Direct from Speakeasy's 'Single or Multiple Errors?' section; this is a concrete, common real-world case the lens currently has no opinion on. |
| P1 | S | Add a Retry-After/backoff check: 429 (rate limited) and 503 (overloaded) responses include Retry-After so clients back off correctly. Fold into the existing rate-limit-headers item. Speakeasy explicitly flags Retry-After; it pairs naturally with the rate-limit headers we already check and closes the loop on graceful failure. |
| P1 | S | Sharpen the pagination item with the tradeoff rationale: prefer cursor-based for large or append-heavy/real-time collections (avoids skip/duplicate drift under writes); offset acceptable for small, stable sets. Frame it as a deliberate choice, not just 'pick one'. Fern (March 2026) and Alex Xu treat offset-vs-cursor as a reasoned decision; adding the WHY lets the reviewer judge whether the choice fits the data, not just whether it's consistent. |
| P1 | S | Expand the versioning item into a deprecation-lifecycle check: prefer additive, non-breaking changes (add fields, don't repurpose/remove); when breaking, bump the version AND signal end-of-life via Deprecation/Sunset headers with advance notice. ByteByteGo names evolving schemas without breaking old clients as the #1 source of long-term API pain; our lens checks that a version exists but not how change is managed over time. |
| P2 | S | Add a field-level / object-level authorization check to 'Common anti-patterns': responses expose fields or objects the caller isn't authorized to see (over-fetching, IDOR via guessable IDs), even when 401/403 are otherwise correct. ByteByteGo lists weak authorization paths as a top failure mode; complements the existing authn-vs-authz check without overlapping the dedicated security lens. |
| P2 | S | Add a brief async/long-running-operation note: operations that can't complete synchronously return 202 Accepted with a status/polling URL or webhook, rather than blocking or timing out. A common modern endpoint shape (corroborated across REST guides) that the lens does not address; keeps the lens current with real-world API surfaces. |
★ Solid AA-fundamentals checklist that is branded 'WCAG 2.2' but contains none of the 9 new 2.2 criteria, carries a wrong target-size figure (44x44 vs the AA 24x24), and lacks the live-region, data-table, link-text, landmark, and reduced-motion domains the best-in-class Claude Code competitor treats as first-class.
Tight, technique-dense, and well-structured in the repo's lens format (What to check / Common anti-patterns / When to apply). Already covers the durable AA fundamentals correctly: contrast ratios (4.5:1 / 3:1), keyboard access + focus indicators, label association (not placeholder-as-label), alt text incl. empty alt for decorative, 'don't sprinkle ARIA', error announcement, no color-alone, focus management after dynamic changes, skip link, lang attribute, unique page titles, logical heading hierarchy. Anti-patterns list is genuinely good: div-as-button, outline:none without replacement, tabindex>0, hover-only essential info, modals that don't trap focus, non-keyboard custom selects, toasts that vanish before announcement. This already matches or exceeds most generic single-file a11y rule sets.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add the 6 most impactful new WCAG 2.2 AA/A criteria as concrete 'What to check' bullets: focus not obscured by sticky/fixed elements (2.4.11/2.4.12); focus indicator >=2px thick at >=3:1 contrast (2.4.13); any drag interaction has a single-pointer alternative (2.5.7); auth allows paste + password-manager autofill and no cognitive-function test/CAPTCHA without alternative (3.3.8); previously-entered info is auto-populated or selectable, not re-typed (3.3.7); help mechanisms appear in consistent location/order across pages (3.2.6). Lens is named 'WCAG 2.2' but contains none of the 2.2-specific criteria. These are exactly the checks AI-generated UI fails and are the highest legal-risk AA items per the TestParty migration guide. |
| P0 | S | Fix the target-size figure: state WCAG 2.2 AA 2.5.8 minimum as 24x24 CSS px (with the noted exceptions: inline links, user-agent-controlled, essential), and optionally cite 44x44 as the stronger AAA/mobile target rather than presenting 44x44 as the AA bar. The current 44x44 claim is factually mismatched to the stated standard — 24x24 is the 2.2 AA requirement; 44x44 is AAA 2.5.5. Grounded directly in the WCAG 2.2 criteria comparison. |
| P1 | S | Add a live-region / dynamic-announcement check: dynamic status (loading, search results, async validation, success/toast) is exposed via aria-live / role=status (polite) or role=alert (assertive), and content swaps don't silently update for screen-reader users. The best-in-class competitor treats this as a dedicated domain; ours only alludes to it via 'errors announced' and toast timing. This is a top failure mode for SPA / async UIs. |
| P1 | S | Add concrete data-table and landmark checks: tables use th with scope (or headers/id), a caption, and proper header association; sortable columns expose aria-sort state; page uses landmark regions (header/nav/main/footer or ARIA landmarks) with a single main. Competitor has dedicated tables-data and landmark coverage; ours lists tables only under 'when to apply' with no actionable checks, and omits landmarks entirely. |
| P1 | S | Add a link-text + ARIA-correctness check: links/buttons have meaningful text out of context (no bare 'click here'/'read more'/'learn more'); links opening new tabs warn the user; and elevate the existing ARIA note to the 'first rule of ARIA' — prefer a native HTML element over an ARIA-retrofitted div, and every custom control exposes a correct name/role/value/state. Both are standalone competitor domains (link-checker, aria-specialist) and are classic AI-codegen failure modes flagged in the Copilot-risks article. |
| P2 | S | Add a prefers-reduced-motion / animation check: non-essential animation and parallax respect prefers-reduced-motion, and nothing auto-moves/auto-updates for >5s without a pause control (WCAG 2.2.2 / 2.3.3). Animations are common in AI-generated UI and entirely absent from the current lens; grounded in WCAG and the competitor's contrast-master/motion scope. |
| P2 | S | Add a verification-expectations note to 'When to apply': call out that automated tools (axe/Lighthouse) cover only ~30% of criteria, so keyboard-only and screen-reader (NVDA/VoiceOver) manual passes are required for focus order, announcements, and reading sequence; and note that 4.1.1 Parsing was removed in WCAG 2.2 (don't flag well-formedness as a WCAG failure). Sets correct reviewer expectations (the ~30%-automatable framing is the core thesis of the competitor and the '33 skills' article) and removes a potential stale/incorrect HTML-validity expectation. |
★ Strong on exception-handling fundamentals, but missing the modern resilience layer every source treats as table stakes: bounded/single-layer retries + circuit breakers, idempotency-key-based safe retries, granular timeouts with deadline propagation, DLQ/saga for async + multi-step work, graceful fallbacks, and resilience-specific observability.
Solid, well-scoped coverage of the fundamentals: specific exception types over bare catch-all; preserving error context on re-raise (raise ... from e); helpful user messages that don't leak internals; retry with exponential backoff AND jitter; resource cleanup via finally/context managers; error boundaries at system boundaries (API handlers, consumers, job runners); collecting validation errors together; distinguishing expected (user/network) from unexpected (bug) errors; async timeout/cancellation; and correlation ID + timestamp + error code in responses. The anti-pattern list is genuinely strong and matches field consensus: silent swallow, log-but-return-success, exceptions-for-flow-control, retrying non-idempotent operations, hiding catch-all, string-matching on error messages, and missing timeouts on external calls. These map cleanly onto the GitHub Copilot review guide and the basics every source agrees on.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a 'Resilience for external calls' cluster of checks: (1) retries are bounded (max attempts AND max total elapsed time) and happen at exactly ONE layer, not stacked across layers; (2) a circuit breaker (or equivalent fail-fast) protects calls to a dependency that may be down; (3) connection timeout and request timeout are set separately, with the request timeout derived from the downstream's p99/p99.9, and inbound deadlines are propagated to downstream calls. Retry amplification (Amazon Builders' Library 243x; AWS Well-Architected High-risk anti-pattern), circuit breakers, and granular timeouts + deadline propagation are treated as table stakes by all three sources, yet the lens covers none of the three. These are the single biggest gap between OURS and the modern field. |
| P0 | S | Replace the bare 'retrying non-idempotent operations' anti-pattern with a positive check: writes that may be retried carry an idempotency key (client-supplied token / Idempotency-Key header), the server dedupes on it (store-with-TTL, return the original result on replay, reject same-key-with-different-body), and a timeout on a write is treated as an INDETERMINATE outcome requiring reconciliation, not a definitive failure. Every source names idempotency as the prerequisite to safe retries (the 2026 reference and Temporal both lead with it; IETF is standardizing the header). The lens currently only states the negative. The 'timeout is indeterminate' nuance (Kislay Verma) closes the duplicate-charge failure mode the lens's own 'don't retry non-idempotent' rule is gesturing at. |
| P1 | M | Add async/queue and workflow checks: message consumers cap retries and route poison messages to a Dead Letter Queue (with depth/age monitoring and a reprocessing plan) rather than retrying forever; consumers dedupe because delivery is at-least-once; and multi-step workflows that partially succeed have a defined rollback via compensating transactions (saga). The lens lists consumers and multi-step workflows as in-scope but gives no async-specific failure handling. DLQs, at-least-once dedup, and saga/compensating transactions are core in Temporal and the distributed-systems checklist. |
| P1 | S | Add a graceful-degradation check: when retries/breaker exhaust, there is a defined fallback (serve stale cache, simplified/generic response, disable non-essential feature, or fail fast), and the fallback path is simpler and more reliable than the primary path. Fallbacks/graceful degradation are a named pillar in Temporal and implied by the partial-failure mindset; absent from OURS. The 'fallback must be simpler' constraint prevents the common mistake of a fallback that itself depends on three more services. |
| P1 | S | Add an observability check tailored to error handling: emit and alert on retry rate vs success rate (and max-attempt-cap hits), circuit-breaker state transitions/flapping, DLQ depth and message age, timeout rate, and fallback invocation/success rate, so a retry storm is visible before customers feel it. Both the 2026 reference and Temporal ship a metrics-per-pattern alerting table; OURS's observability is limited to per-error correlation IDs. These signals are how you know the error handling is working rather than silently amplifying failure. |
| P2 | S | Add two concrete failure-mode anti-patterns from the distributed-systems checklist: 'invoking a remote/network call while holding an open DB transaction or pooled connection' (exhausts the pool under downstream slowness), and 'no defined unit-of-failure for batch/partial-success operations' (caller can't tell what succeeded). Optionally note structured machine-readable error bodies (RFC 9457 application/problem+json: type/title/status/detail/instance) as the standard form of the lens's existing 'error code' field. The remote-call-in-transaction and batch unit-of-failure items are concrete, high-frequency production bugs from Kislay Verma's checklist that the lens doesn't catch. RFC 9457 generalizes the lens's lone 'error code' into a parseable contract and is now shipped natively by Spring Boot 6+ / ASP.NET Core 7+. |
★ Bolt a keyboard/axe accessibility pass (focus-order walk + focusable-without-focus-ring detection) and a 44px touch-target check onto the existing measure.js — that's the one dimension where every leading design-review skill clearly beats our contrast-only a11y coverage, and it fits our measure-not-eyeball doctrine perfectly.
Ours is genuinely differentiated and best-in-class on several axes the field's checklists do NOT match: (1) Measure-driven truth — measure.js computes type-scale sprawl, gap dupes, top-level-sibling edge misalignment, tabular-num currency, contrast ratios, ragged wraps, page-scroll/overflow/clipped-table detection in-browser, so a screen passes only when the measure is clean AND the shot passes — far more rigorous than the eyeball-plus-checklist competitors. (2) Exhaustive crawl discipline — every route from router config (not just nav), EVERY persona with the explicit 'never audit as admin alone / RBAC-gated UI' rule, every state/expandable/modal/dark+light, list->detail link-following. Most competitors visit '5-8 pages from homepage'. (3) A best-in-class false-positive section that competitors largely lack: resize_page clamps to physical display (read innerWidth after every resize), fullPage clips right edge, modal z-index overlaps are false positives, in-container overflow-x scroll is fine vs page-scroll/clipped-parent is a defect, white-on-dark is correct, padding-inset children aren't misaligned, grep-a-class is noisy so verify computed style on the rendered element. (4) Fix-at-source doctrine (one token/theme-class/component change cascades; outline not box-shadow for focus rings so overflow-hidden doesn't clip). (5) Crisp differentiation vs /qa and /ux. These are real, hard-won, and must not regress.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a keyboard + axe accessibility step to the loop and a measure block. In measure.js add an a11y section: enumerate focusable interactive els, report count without a visible focus-visible outline (compare :focus computed outline/box-shadow), and flag any element with outline:none and no replacement. In SKILL.md 'The loop', add a step: tab through the page capturing document.activeElement at each stop to verify logical order + no focus trap, and (if available) run an axe-core scan against the live DOM. This is the field's #1 consensus addition (arkhe Phase 4, Larn, accessibility-agents) and the biggest single gap. Every leading design-review skill pairs the visual pass with an axe-core + keyboard-focus walk; contrast-ratio alone is the weakest part of our otherwise strong measure. Grounded in arkhe, gstack, and the Larn/accessibility-agents consensus. |
| P0 | S | Add a 44px touch-target SIZE check to measure.js alongside the existing 8px-proximity check: filter button,a,input,[role=button] where rect.width>0 && (width<44||height<44), return count + sample labels. Note it as a mobile-width (~375) lens in the SKILL design bar. Cheapest high-value miss; a one-line getBoundingClientRect filter that the gstack skill includes and ours omits. Directly serves the mobile width we already audit. |
| P1 | S | Add a noise-control clause to the Output section: keep 'flag everything' as the SCAN posture, but in the REPORT cap to the highest-signal items — group by fix-at-source, lead with Blocker/High, and gate borderline aesthetic items behind a confidence bar ('report a finding only when you can name the measured failure or cite a specific token/rule; suppress pure preference'). Add an explicit severity ladder (Blocker / High / Medium / Nit) rather than just MED/S. arkhe's confidence-1-10 + 8-finding-cap + self-reflection is the field's answer to the exact failure mode our 'nothing is fine' posture risks on a large app: a wall of low-signal nits burying real defects. Adopt the discipline without losing our measure-first rigor. |
| P1 | M | Add a motion/animation lens to measure.js and the design bar: flag elements with transition-property:'all', flag transitions/animations on layout properties (width/height/top/left/margin) rather than transform/opacity, and check matchMedia('(prefers-reduced-motion: reduce)') is respected. Add 'Motion' as a measured sub-lens under existing lens 8. Lens 8 currently asserts 'smooth transitions' but is unmeasured; gstack codifies concrete, computable motion rules (no transition:all, only transform/opacity, honor reduced-motion) that fit our measure-not-eyeball doctrine. |
| P2 | M | Extend measure.js with three cheap computed-style probes: (a) distinct font-FAMILY count (flag >3) added to the typeScale report; (b) copy hygiene — flag straight quotes/apostrophes and literal '...' where '…' belongs in user-facing text; (c) color-only status encoding — flag red/green status text/badges that carry no icon or text label. All three are standardized in the gstack 80-item checklist, computable in-page, and close real fit-and-finish gaps (font-family drift, typographic polish, colorblind-safe status) that our size/weight-only and contrast-only checks miss. |
| P2 | M | Add an optional baseline/regression note: on first run, persist the measure.js JSON per page (e.g. a design-baseline file in the run dir); on re-run after fixes, diff against it to surface new vs resolved defects and per-lens deltas instead of re-judging from scratch. Both gstack (design-baseline.json regression mode) and arkhe support cross-run regression diffing; it turns our one-shot audit into a trackable gate. Lower priority because our fix->re-measure->confirm-zero-regression loop already covers the within-session case. |
★ Add the two research-backed checks ours lacks — Lint Leakage (delete rules a linter already enforces, the #1 real-world smell) and Enforcement mismatch (promote hard 'never' prohibitions to deterministic hooks) — to turn a solid duplicate/contradiction auditor into a full config-smell auditor grounded in the UFMG taxonomy.
Ours is already well-scoped and matches the field on its core job. It covers the four checks most articles lead with: Duplicates (semantic, not just exact match), Contradictions (ALWAYS/NEVER opposing pairs), Vague rules (non-actionable), and Stale rules (verifies referenced files/tools exist via Glob). It uniquely handles Scope conflicts (project vs global CLAUDE.md, plus ~/.claude/CLAUDE.md), which most community commands ignore. It has strong anti-fabrication discipline ('Don't invent problems... say it's clean'), a clean structured report with file:line citations and quoted rule text, a consolidation nudge at 50+ rules, and firm READ-ONLY safety rails. The note that /learn is append-only (so fixes need manual editing) is an honest, useful constraint.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a 'Lint Leakage' check to Step 2: flag any rule that merely restates what a linter/formatter/type-checker already enforces (indentation, quote style, import order, line length). Suggest deleting it and letting the tool own it. Cite it as the most common smell so users take it seriously. 62% prevalence in the UFMG study (arXiv:2606.15828) makes it the highest-frequency real issue, and ours misses it entirely. Pure token-waste, cheap to fix. |
| P0 | S | Add an 'Enforcement mismatch' check: flag rules phrased as hard 'never/must-not' prohibitions (especially around destructive ops) and suggest promoting them to a deterministic PreToolUse hook, noting instructions are followed only ~70% of the time. Multiple sources + Anthropic's own steering guidance: 'when something absolutely must not happen, an instruction is the wrong tool.' This is the field's sleeper insight and a genuinely new check. |
| P1 | S | Add a 'Wrong layer / Skill Leakage' check: flag rare, task-specific workflows (release process, migration recipe, one-off setup) living in always-loaded CLAUDE.md and suggest moving them to a skill or path-scoped rule that loads on demand. 35% prevalence; directly reduces context cost and buries fewer important rules. Fits jacked's skill/command ecosystem cleanly. |
| P1 | S | Upgrade the vague-rule check to also flag negative phrasing: when a rule says 'never/don't X', suggest reframing as a positive 'always Y' directive, since negative directives are measurably weaker and force the model to dwell on the prohibited action. Consistent practitioner consensus (AI in Plain English, dev.to '5 Patterns'). Small lint that improves rule eff-icacy, complements the existing actionability check. |
| P2 | S | Add a 'Blind References' check: flag external files/paths/links cited in a rule with no explanation of when to read them or why they matter; suggest adding a one-line purpose+timing annotation. 16% prevalence; cheap annotate-don't-delete fix that improves how the agent uses referenced docs. |
| P2 | M | Broaden SCOPE to also read AGENTS.md and, if present, .cursor/rules + .github/copilot-instructions.md, and run cross-file duplication/contradiction across all of them (not just project vs global CLAUDE.md). AGENTS.md is now read by Claude Code/Codex/Cursor/Aider; cross-tool rule duplication is a documented real-world failure (Ivan Morgillo). Extends ours' existing scope-conflict strength. |
| P2 | S | Add a bloat/token signal to the report: show total line count and flag if the always-loaded file materially exceeds ~200 lines, and add Anthropic's per-line keep/cut test ('would removing this cause the agent to make mistakes? If not, cut it') as the framing for the consolidation nudge. ETH Zurich + UFMG: every context file carries a 20–23% inference tax and bloat makes Claude ignore real instructions; the 200-line figure and one-line test are concrete, citable, and sharpen the existing 50+ nudge. |
★ Capture LCP/CLS for real via chrome-devtools performance_start_trace (with recorded throttling) instead of the raw Performance API — today the command budgets Core Web Vitals it never actually measures.
Genuinely strong fundamentals that we must not regress: (1) Browser-health preflight with chrome-devtools->playwright fallback and a /browser-reset escape hatch. (2) 3-sample median to fight noise — exactly what LH-CI's 'median' aggregation does. (3) Dual evaluation: relative (vs baseline, warn/regression two-tier thresholds) AND absolute (industry budgets) — matches LH-CI's warn/error + budget model. (4) Timestamped baselines in ~/.claude/jacked-benchmark/ with multi-baseline trend analysis. (5) Resource breakdown by type + top-10 slowest resources. (6) READ-ONLY discipline. (7) Clean report table format with explicit verdict. This is already above a naive 'run Lighthouse once' command.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a trace-based capture path as the PRIMARY method when chrome-devtools MCP is available: call mcp__chrome-devtools__performance_start_trace (reload=true, autoStop=true), then read LCP and CLS from the returned summary, and optionally call mcp__chrome-devtools__performance_analyze_insight for LCPBreakdown/LCPDiscovery/CLSCulprits/RenderBlocking. Keep the existing evaluate_script Navigation-Timing capture as the fallback (and to enrich resource breakdown). This makes the LCP/CLS budgets actually measurable and adds free, actionable 'why' insights. The command budgets LCP but never measures it; performance_start_trace is the supported, already-available tool that returns LCP+CLS+subparts in a compact AI-ready form (DebugBear; Chrome DevTools MCP blog). |
| P0 | M | Fix the evaluate_script fallback to actually capture LCP and CLS via buffered PerformanceObserver (largest-contentful-paint + layout-shift entry types, summing CLS for non-recent-input shifts) before reading the navigation entry, so even the Playwright/no-trace path reports the headline CWV instead of only FCP. Raw getEntriesByType cannot return LCP/CLS; without a PerformanceObserver the fallback silently omits the two most important metrics it claims to budget. |
| P0 | M | Set and RECORD throttling for every run via mcp__chrome-devtools__emulate (or document Playwright CDP throttling): default to a 'Slow 4G + 4x CPU' profile, stamp the profile into the baseline JSON, and refuse/warn on compare if the stored baseline profile differs from the current run. Unthrottled local runs report falsely-good numbers and baseline-vs-compare is invalid if conditions drift — both sources call this out explicitly as the #1 measurement pitfall. |
| P1 | M | Report a noise floor: alongside the 3-sample median, surface min/max (or stdev) per metric, and suppress a 'REGRESSION' verdict to 'WARN (within noise)' when the delta is smaller than the observed run-to-run spread. Optionally bump to 5 samples for load-time metrics. LH-CI's median/optimistic/pessimistic aggregation exists because deltas inside measurement noise are false alarms; we median but never expose variance, so we can't tell a real regression from jitter. |
| P1 | S | Extend the budget + delta tables to the full Core Web Vitals set and per-resource-type bytes: add CLS (<=0.1), TBT (<=200ms), Speed Index (<=3400ms) to budgets, and add per-type byte budgets (script 300KB, css 100KB, image 500KB, font 100KB, third-party 300KB) reusing the byType breakdown already computed in Step 1. Directly mirrors LH-CI's individual-audit and resource-summary budgets; CLS/TBT are core gates and third-party byte weight is a common silent regression we already have the data to flag. |
| P2 | S | Add one line to the report: a lab-vs-field caveat noting these are synthetic lab numbers, that a page can pass here yet fail field CrUX, and that INP is interaction-driven and not captured by a passive page-load benchmark (suggest the qa/ux flow for interaction timing). LH-CI and the CWV guide are explicit that lab passing != field passing and that INP needs interaction; stating it prevents over-trusting a green verdict. |
★ Give the loop a durable on-disk progress ledger it re-reads each iteration (plus a get-your-bearings smoke check before new work) so the build-out survives context resets and never stacks a cell on a broken main — the one thing every primary source treats as the spine of a long-running agent and the one thing ours currently keeps only in-context.
The auto-merge SAFETY ARCHITECTURE is genuinely best-in-class and exceeds the external art: a two-gate model (declared Lifecycle maturity + verbatim explicit human authorization), default-to-STAGED, hard production refusal, cross-checking for live-user signals even when Lifecycle says pre-production, and treating a stale Lifecycle label as not-eligible. The CI discipline is stricter than anything found externally: merge ONLY when local tests green AND every CI check 'passed' (never pending/skipped/neutral/failed), treat no-CI as unverified, true merge commit only (never --squash/--rebase/--admin), never force-push or bypass branch protection. The one-cell -> one-branch -> one-PR -> one-merge invariant (no two cells at once, no carrying an unmerged cell forward) makes every change a clean revertable unit. Existing guardrails: turn/merge/failure backstop, BLOCKED reporting, scope/destructive-action stops, brief size budget (wc -c < 4000), treating read-in issue/doc text as DATA only, and being a deliberately-named command (not an auto-triggering skill).
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a durable on-disk worklist/progress ledger to the brief. In Step 1-6 / the loop intro, instruct: write the ordered matrix cells to a tracked file (e.g. BHAG_PROGRESS.md or .bhag/worklist.json) with a status per cell (todo/in-progress/done + the merged PR number), and make step 1 of every iteration re-read that file to pick the next 'todo' cell and step 5 update it after merge. This is the field's most-cited durability pattern (Anthropic's feature-list.json + claude-progress.txt; Osmani 'the filesystem isn't amnesiac') and it survives compaction/context resets that the in-brief worklist does not. Without external state, a long autonomous loop loses track of done-vs-remaining cells on any context reset — the exact problem every primary source designs around. |
| P0 | S | Add a 'get your bearings' first sub-step to each iteration: before implementing, pull latest main, read the progress ledger + recent git log, and run the repo's build/start + a quick smoke (or the existing test command) to confirm main is GREEN; if main is already broken, fix that first or STOP — do not stack a new cell on a broken base. Anthropic's #1 documented fix for looped agents building on a half-broken state and for wasting a session re-discovering context; cheap and directly prevents compounding breakage. |
| P1 | S | Add an explicit test ratchet to the brief's verify step and STOP rules: 'Never delete, skip, weaken, or loosen existing tests/assertions to make an iteration green; a feature only counts as done when it passes end-to-end, not when the test was removed. Removing or gutting a test to pass is a BLOCKED condition, not a merge.' Closes a known reward-hacking escape hatch; Anthropic uses near-verbatim 'it is unacceptable to remove or edit tests.' Our current 'add NEW tests / never merge red' leaves the delete-the-test loophole open. |
| P1 | M | Add a checker step to the done-judgment: before opening the PR, have the loop verify the cell against an explicit acceptance line it wrote for that cell at step 1 (end-state + evidence shown), and prefer the existing jacked review gates as the checker (e.g. run /dcr, or /cso for security-sensitive and /qa for UI) so the 'this cell is now best-in-class' call isn't purely self-graded by the maker. Maker-grading-its-own-work is the field's named #1 failure mode (Anthropic, Cursor, Osmani); writing the acceptance criterion up front + a separate review gate makes 'done' a contract, not a vibe. |
| P2 | S | Extend the backstop to include a cost/budget ceiling: 'STOP after N merges, N failed iterations, N total turns, OR an approximate token/time budget the user sets at Step 7 (default to a conservative cap), whichever comes first.' Surface the budget in the Step 7 plan presentation alongside the mode. Osmani lists unbounded spend as a top unsolved risk for overnight loops ('burn a week's budget in an afternoon'); our backstop currently has no economic circuit-breaker. |
★ Require N consecutive failures before alerting (and pair it with absolute success-rate + latency floors) — this single change kills our biggest real-world flaw, single-sample false alarms, and aligns us with how every production canary tool (Argo/Flagger) actually decides pass/fail.
Already strong and should be preserved: (1) Browser-tool health check in Step 0 with Chrome DevTools -> Playwright fallback and a clean abort path — most monitoring scripts assume the harness works. (2) Explicit dual-MCP tool mapping table so the loop is tool-agnostic. (3) A real baseline/compare model (capture BEFORE deploy, diff AFTER) with persisted ~/.claude/jacked-canary/baseline-latest.json. (4) Strict new-vs-pre-existing error discrimination so it only flags actionable deltas. (5) Five concrete check classes (load, console, DOM-render, performance, network 4xx/5xx). (6) 'Never stop on first alert — finish the duration to catch cascading issues.' (7) READ-ONLY discipline and an explicit revert recommendation. (8) Auto-discovery of prod URL (Railway/Vercel/homepage/CNAME). This is genuinely above-average for the niche.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a consecutive-failure budget instead of single-sample alerting. Track per-check-class failure streaks; only ALERT after the same check fails N consecutive intervals (default N=2, configurable via `--failure-limit`). A single bad sample becomes a logged blip, not an alert. Mirror Argo's failureLimit / Flagger threshold semantics. Eliminates the biggest real-world failure mode of our current loop: false alarms from one transient 5xx, cold-start latency spike, or flaky network sample. This is THE consensus practice in every canary tool we found. |
| P0 | M | Add absolute thresholds alongside the baseline deltas: a success-rate floor (default: 0 new 5xx AND >=99% of tracked requests non-failed) and a latency ceiling (default: FCP <= 3000ms / LCP <= 4000ms absolute, in addition to the existing >100% delta). Bake the explicit note 'check BOTH success-rate and latency — error rate alone misses performance degradations.' Directly from the OneUptime/Flagger best-practices section and the canonical request-success-rate>=99 / request-duration<=500ms config. Our delta-only model misses a deploy that is uniformly slow if the baseline was also slow, and has no success-rate metric at all. |
| P1 | M | Add an optional Check F: 'critical user journey' smoke check. If the user passes `--journey` (or a journey is discoverable), script one real flow: navigate -> wait for a named selector -> click -> assert a post-action element/text exists. Treat a failed journey assertion as an ALERT-tier signal. Keep the bare-navigate as the fallback when no journey is given. Synthetic-monitoring consensus (Datadog/New Relic/Elastic): a 200 + clean console can still be a functionally dead page. DOM element-count alone passes a half-broken render. |
| P1 | S | Add a pre-loop health gate (Step 2.5): immediately after first navigate, run all checks ONCE; if the page is already down, already throwing new console errors, already serving 5xx, OR not serving the new build, report a FAIL-FAST result and recommend revert before spending the full monitoring duration. Flagger's pre-rollout acceptance test. No reason to burn a 10-30 minute loop when the very first load is already broken — fail fast and recommend revert in seconds. |
| P2 | S | Introduce a third terminal verdict INCONCLUSIVE alongside HEALTHY/DEGRADED/FAILING, used when signal is insufficient (e.g. <2 successful samples collected, baseline missing AND no absolute floor tripped, or mixed flapping). Recommendation for INCONCLUSIVE = 'hold, do not auto-trust or auto-revert; extend monitoring or capture a baseline.' Argo Rollouts explicitly models Inconclusive precisely so noisy data neither auto-promotes nor auto-aborts. Our binary verdict over-claims confidence when baseline is absent or data is sparse. |
| P2 | S | Formalize the WARN vs ALERT tiers as a small table at the top of the loop: ALERT-tier (page down, NEW console error, NEW 5xx, failed journey, render collapse >30%) = abort-worthy / recommend revert; WARN-tier (perf delta, resource-count drift, memory growth) = informational, never the sole trigger for a revert recommendation. Cite this as the dry-run/abort distinction. Argo's dryRun pattern — some metrics inform but must not drive rollback. Right now WARN and ALERT severity is implied per-check but never stated as a contract, so the model may over- or under-react. |
| P2 | M | Add a build-verification step to baseline + monitoring: capture a build fingerprint (e.g. main JS/CSS asset hash from network requests, a <meta name=build>, or a /version|/healthz body) at baseline, and in the loop assert the live build fingerprint has CHANGED from baseline (confirming the new deploy is actually being served, not a stale CDN/cache). Prevents the silent false-HEALTHY where a cache serves the old artifact. None of the page/console/perf checks would catch 'we are monitoring the wrong build.' |
| P2 | S | Make the inter-check wait real and turn-safe: since the harness blocks foreground sleep, instruct the loop to use the Monitor/background until-condition pattern to wait `--interval` minutes between checks rather than implying an in-turn sleep, and have each interval append to a single rolling status log so streaks are tracked across turns. The current 'for each check interval' instruction has no actual wait mechanism under this harness, so intervals can collapse to back-to-back calls, defeating the point of duration-based canary monitoring. |
★ Make resume goal-directed (Amp-style): let the user pass the next task so /checkpoint resume extracts only the relevant context and proposes a focused starting brief, instead of always replaying the full generic snapshot.
Ours is already well above the field's median and matches or exceeds the Claude Code #11455 community spec on content structure. Specific strengths to preserve: (1) Rich, well-chosen section taxonomy — Session Context is explicitly called out as 'the most important section' for capturing interactive/verbal knowledge and the WHY of decisions, which the #11455 thread independently confirms is the load-bearing value. (2) Engineering rigor the community patterns lack: atomic temp-file-then-rename writes, append-only checkpoint history, single-writer assumption, and a documented legacy .md migration path. (3) Resume is robust: branch-mismatch warning (no auto-switch), priority-ordered file auto-load (plans > research > key files) with a ~3000-line budget and graceful 'also referenced (not loaded)' overflow, plus missing-file handling for renamed/deleted refs. (4) Machine-introspectable metadata via HTML <meta> tags (status/branch/timestamp/plans/research/lenses) enabling list/resume/complete without a DB. (5) active_lenses tracking and separated HTML research files — both go beyond the flat single-file approach Cline/Amp/#11455 use.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add goal-directed resume: support `/checkpoint resume {slug} --goal "<next task>"` (and an optional free-text goal prompt on plain resume). When a goal is supplied, after loading the checkpoint, filter/rank the auto-loaded files and the Remaining Work toward that goal, and synthesize a focused 'next-task brief' (goal + the subset of context/files that matter for it) instead of replaying the whole snapshot. Mirror Amp: present it as a proposed starting plan the user can edit before you begin. This is the single biggest delta vs best-in-class. Amp deliberately replaced compaction with goal-directed handoff because focused threads outperform full-context replays. Ours currently can only do the generic full replay. |
| P0 | S | Add a secrets-hygiene rule to the Rules section and a guard in Save Flow Step 3/5: before writing, scan the drafted Session Context / Gotchas / Key Files content for likely secrets (API keys, tokens, passwords, .env values, connection strings) and redact to placeholders like {REDACTED: openai key}. State plainly 'never write credentials or secrets into a checkpoint or research file.' Directly from the #11455 spec's Security & Privacy section. Ours captures free-form verbal context and 'things tried' that frequently include pasted keys/env values, and these HTML files can be committed to git — a real leak vector with zero current mitigation. |
| P1 | M | Add a resume-time freshness check (new Resume Step between branch check and file load): compare the checkpoint's saved branch HEAD/commit and a captured `git status` fingerprint against current `git rev-parse HEAD` and `git status --short`. If HEAD advanced or the tree diverged since save, surface a short 'State has moved since this checkpoint: N new commits, M files changed — Current State may be stale' note so the model and user don't trust a stale snapshot. To enable this, also capture `<meta name="jacked:head_sha">` in Save Flow Step 5. Avoiding silently-stale restored context is the explicit motivation behind Amp's handoff redesign. Ours presents 'Current State' verbatim with no signal that the repo moved on, which is exactly the failure mode multi-day workflows hit. |
| P2 | M | Add an optional persistent 'project journal' note: on `/checkpoint complete`, offer to promote durable, checkpoint-independent learnings (recurring gotchas, user preferences, non-obvious patterns) into a single long-lived `.claude/checkpoints/PROJECT-NOTES.md` (or append to CLAUDE.md), and read it on every resume. Keep it small and curated, not append-everything. Cline's .clinerules learning-journal is the field's mechanism for cross-session intelligence that outlives one task. Ours loses all hard-won project knowledge the moment a checkpoint is completed; a thin durable layer closes that gap without the overhead of Cline's full multi-file memory bank. |
| P2 | S | Add one line of git-hygiene guidance near the checkpoint-dir setup: note that `.claude/checkpoints/` can be committed to share session handoffs with teammates, or gitignored for local-only use — and that if committed, the secrets-hygiene rule above is mandatory. #11455 explicitly flags commit-vs-gitignore as a decision users must make; ours is silent on it, leaving a foreseeable 'I committed my checkpoint with a key in it' outcome. |
★ Add the path-scoped `.claude/rules/` (`paths:` frontmatter) lever and measure in tokens via /context — that one upgrade turns a strong line-budget audit into a Claude-Code-native token-efficiency tool that matches the verified 41% / 91.9% reduction techniques the field has converged on.
Our skill is structurally more rigorous than almost everything published. Keep/protect: (1) The two-axis Quality(50)+Efficiency(50) scoring rubric with letter grades — no external source has a comparable audit instrument. (2) The KEEP/EXTRACT/DEDUPE/DELETE per-section classification framework with explicit lists of what belongs in each bucket. (3) Report-BEFORE-changes discipline (Phase 5 output gate) — prevents destructive trimming. (4) Strong-pointer language patterns WITH explicit weak-pointer anti-patterns ('See X for details' gets ignored) — this is genuinely ahead of the field. (5) The doc-chain hierarchy concept (always / when-area / task-specific) with a verify phase that runs executable checks for missing references and weak pointers. (6) The 'restructure, don't trim' principle (size-tiered line budgets) and the named restructure patterns (Everything-In-One-File, Weak-Pointers, Duplicated-Tables, Stale-Reference, No-Doc-Chain). None of this should regress.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a path-scoped rules lever to Phase 4/6. Introduce a fifth classification SCOPE: content that is a real rule but only applies to files under a path → move to `.claude/rules/<area>.md` with `paths:` frontmatter (e.g. `paths: ["src/api/**/*.ts"]`) instead of (or alongside) a pointer. Add a one-line explainer: rules files WITHOUT `paths:` frontmatter load every session like a second CLAUDE.md; WITH it they load only when a matching file is touched (~41% always-loaded overhead reduction). Note Skills as the home for extracted multi-step procedures (progressive disclosure, 30-100 tokens at startup). This is the highest-leverage technique in the entire field that ours completely lacks, and it's Claude-Code-native and strictly better than a pointer for path-specific rules (auto-loads, no reliance on Claude deciding to read). Verified 41% reduction. |
| P0 | S | Quantify in tokens, not just lines. In Phase 2 (Measure), instruct running `/context`, `/memory`, and `/usage` to get the real per-turn cost, and add a token target alongside the line table (e.g. aim <500 tokens / ~60-200 lines; cite Anthropic <200 lines). In Phase 7 (Verify) report before/after token delta, not just line delta. The whole premise of the skill is per-turn token cost, yet it never measures tokens or uses the built-in commands that show them. The field consistently targets and reports tokens; the headline 91.9%-reduction proof is a token number. |
| P1 | M | Add a content-quality (not just structure) audit dimension. Add anti-pattern lint for weak RULES: prose-without-commands, ambiguous directives ('be careful'/'gracefully'/'where possible'), contradictory priorities without numbered ordering, and style rules with no enforcement command. Add a rule: every command must be a copy-paste invocation WITH FLAGS, and high-stakes workflows should have a 'Definition of Done' (specific passing checks/exit codes). Fold into the Quality score. Two independent studies (2,500 repos; 10+-run behavior tests) find vagueness — not length — is the #1 reason instruction files get ignored. Ours optimizes structure and token cost but never lints the rules for actionability. |
| P1 | S | Add a Boundaries check using the three-tier model: every A-grade CLAUDE.md should have explicit Always-do / Ask-first / Never-do rules (never commit secrets, never edit vendor/generated dirs, never force-push) plus escalation-when-blocked guidance. Add it to the KEEP list and as a scored criterion. Field consensus across GitHub's analysis and behavior tests: boundaries + escalation rules are the most common helpful constraint and prevent destructive workarounds. Ours only mentions one example in passing. |
| P1 | S | Sharpen the DELETE rule with the inference heuristic and add the mechanics tips. State the one-line test: 'If Claude can infer it from reading the codebase — or a senior dev could in ~20 min — cut it (framework-default conventions, generic stack facts, globbable structure).' Add a short 'Mechanics' note: HTML comments are stripped before injection (free teammate notes); @import splitting is organizational only — all imported files still load at session start (NOT a token saving); use CLAUDE.local.md (gitignored) for personal prefs. The inference heuristic is the field's crispest DELETE test and catches bloat ours only implies. The @import misconception is widespread and worth pre-empting; the HTML-comment trick is a genuine free win. |
| P2 | S | Add a 'recite test' to Phase 7 Verify: after restructuring, ask Claude (fresh context) to reproduce the build/test commands and the top constraints verbatim — if it can't, the file is too verbose or too vague; iterate. A near-free empirical validation the field recommends; complements our existing static checks with a behavioral one. |
| P2 | S | Add a short AGENTS.md interop note. If the repo (or team) uses other agent tools (Codex/Cursor/Copilot/Windsurf), recommend AGENTS.md as the canonical source and have CLAUDE.md mirror/import the shared sections rather than maintaining parallel files that drift. Mention it's a Linux-Foundation standard read by 30+ tools. Multi-tool teams are now common; without this they silently maintain drifting duplicate instruction files — exactly the duplication our skill exists to prevent, one level up. |
★ Ours is a strong, mature simplicity reviewer that already matches the field on substance and exceeds it on before/after rigor and trade-off nuance — it mainly needs guardrails the field has converged on: a diff-scope fence, an anti-nitpick/clean-exit cap, a named over-engineering+atomicity checklist, and a merge-readiness verdict.
Ours is already a well-shaped, mature simplicity reviewer. Clear core principles (simplicity-first, human readability as prose, future-ready without over-engineering). A real review process: focus on recent changes, named simplification levers (combine parameterized fns, prefer stdlib/built-ins, guard clauses + early returns + lookup tables, remove intermediate transforms, async over callback chains), and SOLID/duplication/control-flow checks. Good trade-off discipline (acknowledges when complexity is justified, never sacrifices correctness/type-safety/error-handling for brevity). Strong communication norms (constructive, before/after snippets, lead with what's good, explicit 'if it's already good, say so'). A concrete output template with prioritized sections. This already covers the substance of hamy.xyz's Agent 9 and exceeds it on before/after-snippet rigor and trade-off nuance.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a 'Scope discipline' rule to the review process: review ONLY the changed/added lines in the diff; do not flag or re-design pre-existing untouched code; if a simplification requires touching code outside the change, note it as an explicit out-of-scope 'optional follow-up', not a primary finding. arXiv 2605.20668 and CodeAnt identify out-of-scope, over-harsh findings as the #1 reason AI reviewers lose trust. Ours says 'focus on recent changes' but never fences the agent to the diff. |
| P0 | S | Add an anti-nitpick / calibration clause: cap findings (e.g. top ~5 by impact); skip anything a linter/formatter/type-checker already catches (whitespace, import order, naming nits); and when the change is genuinely simple-and-clean, say exactly that in one line and stop — do not manufacture findings. hamy.xyz's reviewer caps at 5, explicitly skips linter-catchable nits, and has a one-line clean-exit ('complexity is proportionate'). The arXiv paper shows uncalibrated volume is the core AI-reviewer failure. Ours only softly hints at this. |
| P1 | S | Add an explicit over-engineering / YAGNI checklist to the 'Identify Simplification Opportunities' section: premature abstraction; helper or wrapper used exactly once; framework/config/plugin machinery for a one-off problem; indirection that doesn't pull its weight; speculative generality ('solving a problem we don't actually have'); clever code that sacrifices clarity. The BuiltIn six-month study found all four agentic tools over-engineer and expand scope; hamy.xyz Agent 9 enumerates exactly this list. Ours mentions 'over-engineered solutions' abstractly but gives the agent no concrete pattern vocabulary to detect it. |
| P1 | S | Add a change-atomicity check: is this diff one logical unit of work? Is unrelated cleanup/refactoring/feature work bundled in that should be a separate commit? Is it sized to be reviewable? Flag scope creep and suggest splitting. BuiltIn names scope expansion ('one change requested, three made, regression slips in') as the actual failure mode of AI-authored diffs; hamy.xyz Agent 9 has a dedicated atomicity block. Ours has no check for this entire dimension. |
| P2 | S | Add a one-line merge-readiness verdict to the output template (e.g. 'Verdict: Clean / Minor simplifications / Needs simplification') and rank findings by impact AND effort rather than impact alone, so high-impact/low-effort wins surface first. The hamy.xyz suite closes the loop with a Ready-to-Merge/Needs-Attention/Needs-Work verdict and impact+effort ranking; this makes output actionable for an orchestrating agent deciding whether to iterate. Ours stops at prioritized suggestions with impact-only ranking. |
★ Add an explicit Confidence discount to the lever-ranking formula and anchor the whole matrix on 1-3 measurable target outcomes — together they turn 'a defensible feature-parity audit' into 'a defensible impact-driven roadmap,' which is what best-in-class gap analysis (RICE + Opportunity Solution Tree) actually requires.
The dual-axis capability-vs-experience model is a genuine differentiator — nothing in the surveyed Claude Code PM ecosystem or the classic frameworks (RICE, OST, JTBD) collects 'experienced workflow' evidence the way our required Experience Walkthrough stream does (click-cost of the core loop, lifecycle visibility, metric honesty / counters-with-zero-writers, lying success states, repeated-context cost). The hard rule that feature-inventory evidence caps a cell at 7 and 8+ requires walkthrough evidence is a strong, original anti-over-scoring guardrail. The Roles × Domains INTERSECTION framing (vs a flat feature list) correctly captures combinatorial gaps that a 1-D backlog misses. Cross-cutting levers ranked by cells-lifted-per-effort is exactly the right ROI lens. Mandatory source-URL tracking + competitor profiles with clickable links + inline superscript citations is more rigorous than the ecosystem norm. The self-contained dark-theme HTML heatmap + PDF export is a polished, presentable deliverable. The Red Flags and Rationalizations tables are excellent self-correction scaffolding. Do not regress any of these.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | In Step 6, add a Confidence column (0.1-1.0, or High=1.0 / Med=0.7 / Low=0.4) to the cross-cutting levers table and change the ranking formula to (cells_lifted × avg_score_gain × confidence) / effort. State the rule: a lever's confidence is High only when its cells_lifted estimate is backed by walkthrough or competitor evidence, not inference. Mirror RICE's explicit discount on guesses. Ours currently rewards confident-sounding guesses over evidenced ones — the exact false-precision failure RICE's Confidence factor exists to prevent (Dovetail, Anthropic roadmap-management). One-line formula change, immediately more defensible rankings. |
| P0 | M | Add a new early step (between Step 2 and Step 3, or fold into Step 1): 'Anchor on the target outcome.' Require naming the 1-3 measurable business/product outcomes (a metric with a current and target value, e.g. 'coder throughput', 'denial rate', 'time-to-first-value') that 'best-in-class' is supposed to move. Then in Step 6/7, every cross-cutting lever and every roadmap phase must state which outcome it ladders up to. Add a Red Flag: 'Matrix is fully scored but no cell maps to a business outcome — you've measured feature parity, not impact.' Teresa Torres' Opportunity Solution Tree anchors all gap analysis at a measurable outcome root; without it our roadmap optimizes feature coverage in a vacuum and can't be defended to a stakeholder asking 'what does this move?' |
| P1 | S | In Step 4's competitor stream and the Step 2/3 discovery, require classifying competitors into the 4 tiers from Anthropic's competitive-analysis skill: direct, indirect, adjacent, and substitute (explicitly including the manual/spreadsheet/status-quo the product replaces). Score the matrix's 'best-in-class' ceiling against the strongest tier per domain, and call out substitutes by name. Ours says 'top 5-10 competitors' flat; the substitute tier (the manual process you're displacing) is usually the real bar and the most common miss. Direct lift from Anthropic's official skill. |
| P1 | M | In Step 7, reframe phases with explicit time-horizon labels (Now / Next / Later or quarterly) and add two required fields per phase: (a) dependencies — which earlier phase/lever must land first; (b) a one-line capacity sanity-check (does the effort estimate fit a realistic team/sprint window). Keep cell-lift-per-effort as the ordering driver. Now/Next/Later + dependency mapping + capacity planning are the standard the Anthropic roadmap-management skill ships; ours phases by ROI but hides sequencing constraints, so a top-ranked phase can be un-startable because its dependency is in a later phase. |
| P2 | M | Add an optional bundled helper-script step (or at minimum a note) to compute lever rankings deterministically: feed a small CSV/JSON of (lever, cells_lifted, avg_score_gain, confidence, effort) to a tiny script that emits the sorted table, instead of doing the arithmetic in-context across dozens of cells. Reference the pattern (alirezarezvani rice_prioritizer.py). At matrix scale (10 roles × 8 domains = 80 cells), in-context cells_lifted counting and ranking is error-prone; the ecosystem has standardized on deterministic scripts for exactly this. Optional so it doesn't bloat the core flow. |
★ cso already meets or beats the field on scope, exclusions, and structure — close the gap by adopting Anthropic's official /security-review techniques it's missing: a dedicated adversarial per-finding FP pass, concrete precedent rulings, baseline-deviation analysis, and an OWASP-LLM threat lens.
cso is already strong and largely matches the field. It has: an explicit 8/10+ confidence gate; a mandatory concrete exploit scenario per finding ('an attacker could...'); a thorough 17-category false-positive exclusion list (broader than Anthropic's ~5 hard exclusions); full OWASP Top 10 walk-through (A01-A10) with stack-specific grep patterns; a STRIDE threat model focused on the 3-5 most critical data flows; multi-mode scoping (--diff/--code/--infra/--supply-chain/--owasp/--skills); tech-stack detection, attack-surface census, git-history secret scan, and dependency audit phases; a clean structured Security Posture Report format; and hard READ-ONLY guarantees with node_modules/vendor/dist exclusions. The --skills mode (auditing Claude Code skills/commands for prompt injection) is ahead of the official tool, which has no equivalent. The phased pipeline (detect stack -> map surface -> scan history -> deps -> OWASP -> STRIDE -> report) is more structured than Anthropic's diff-only command.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add an explicit two-stage adversarial verification pass: after the OWASP/STRIDE detection phase produces candidate findings, run a SECOND per-finding adjudication that re-applies the full exclusion list + precedents to each candidate independently and drops anything that doesn't survive at 8/10+. State that each finding must be 'verified by re-reading the code as a skeptic trying to disprove it.' Mirrors anthropics/claude-code-security-review's 3-step (identify -> filter-per-finding -> drop <8) design and Datadog's separate-classification-pass finding. The biggest lever on report quality is FP suppression, and the field's consensus is that a dedicated adversarial second pass beats a single bundled judgment. Also aligns with claude-jacked's own modernize-harden-scan skill which already does 'adversarial per-finding verification'. |
| P0 | S | Add a 'Precedent Rulings' subsection to the False Positive Exclusions with ~10 concrete adjudications lifted/adapted from the official command: UUIDs assumed unguessable; env vars + CLI flags are trusted inputs; React/Angular XSS only via dangerouslySetInnerHTML/bypassSecurityTrustHtml; SSRF requires host/protocol control (path-only is out); client-side auth/permission checks are not vulns (server-side responsibility); user content in AI system prompts is not itself a vuln; logging URLs is safe but logging secrets/PII is a finding; memory-safety findings invalid in Rust/Go/managed languages; report command injection in shell scripts only with a concrete untrusted-input path. Ours lists false-positive CATEGORIES but gives the model no way to resolve the ambiguous middle cases that drive most noise. These precedents are the highest-signal content addition and come straight from Anthropic's own tuned-in-production prompt. |
| P1 | S | Insert a 'Baseline / Comparative Analysis' step before the OWASP walk: have cso first identify the repo's existing security frameworks, sanitization helpers, and secure-coding conventions, then flag NEW code that DEVIATES from those established patterns rather than judging each file in isolation. Reduces false positives on code that is intentionally consistent with project conventions, and catches the highest-signal bugs (deviations from the team's own secure patterns). Directly from the official command's Phase 1 + Phase 2 methodology. |
| P1 | M | Add an 'A11: LLM / AI-Component Threats (OWASP LLM Top 10)' category, applied only when the stack includes LLM calls, agents, or MCP tools: prompt injection into model inputs, insecure handling of model OUTPUT (eval/exec/SQL built from completions), excessive agency / over-broad tool permissions, and system-prompt / secret leakage. Keep the existing --skills mode as the static-file variant of this. claude-jacked operates in an LLM-heavy ecosystem and audits agentic artifacts; the OWASP LLM Top 10 is the recognized taxonomy and is currently absent from the application-code analysis. Gated on stack detection so it adds zero noise for non-LLM repos. |
| P2 | S | Split the single 8/10 gate into two independent axes already half-present in the report format: SEVERITY (HIGH/MEDIUM/LOW by blast radius) and CONFIDENCE (0.7-1.0 by exploit certainty), with explicit anchors (0.9-1.0 certain exploit path; 0.8-0.9 known pattern; 0.7-0.8 needs specific conditions; <0.7 don't report). Report HIGH/MEDIUM only. Conflating severity and confidence loses triage information; the official command keeps them orthogonal with calibrated anchors, which produces a more actionable findings list. Low effort since the report template already has both fields. |
★ Strong fundamentals checklist, but missing the modern Postgres migration-safety mechanics the field treats as table stakes — CONCURRENTLY indexing, NOT VALID/VALIDATE constraints, lock_timeout, named expand-contract, volatile-default rewrites, and CI migration linting.
Solid, well-scoped checklist that already covers most of the durable fundamentals: NOT NULL discipline, FK ON DELETE behavior, indexes for WHERE/JOIN/ORDER BY, composite-index selectivity order, DECIMAL/integer-cents for money, timestamptz over timestamp, string enums over magic ints, unique constraints, created_at/updated_at, and key anti-patterns (NOT NULL-without-default on a populated table, missing FK indexes, unindexed LIKE '%x%', N+1 lazy loading, JSON blobs vs normalized columns, runaway cascade deletes, non-idempotent migrations). It also already names backward-compatible/rollback-safe migrations and batched updates to avoid locking — so the conceptual frame is right; what it lacks is the specific Postgres operational mechanics the field now treats as table stakes.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a check + anti-pattern: indexes on large/production tables must be created with CREATE INDEX CONCURRENTLY (and dropped with DROP INDEX CONCURRENTLY); plain CREATE INDEX takes an ACCESS EXCLUSIVE lock and blocks all writes for the duration. Note CONCURRENTLY cannot run inside a transaction. Single most common migration outage and the universal first recommendation across strong_migrations, squawk, and the dev.to 2026 patterns piece. Our lens is silent on it. |
| P0 | S | Add a check: foreign keys, CHECK constraints, and SET NOT NULL on existing columns should be added as NOT VALID first (instant), then VALIDATE CONSTRAINT in a separate step (non-blocking ShareUpdateExclusive lock). For NOT NULL on PG12+, validate a CHECK (col IS NOT NULL) NOT VALID then promote to NOT NULL. Validated-in-place constraints scan the whole table under a blocking lock; the NOT VALID/VALIDATE split is the documented safe path in strong_migrations and dev.to. Our lens checks for the constraints but not the safe way to add them to a populated table. |
| P1 | S | Add a check: migrations should set a short lock_timeout (and a statement_timeout) so a migration blocked behind a long transaction fails fast instead of queuing all traffic; on Postgres, prefer lock-timeout retries so a timed-out non-concurrent operation doesn't leave a half-applied/invalid index. strong_migrations sets lock_timeout/statement_timeout by default precisely because a lock-wait queue is how 'safe-looking' migrations cause outages. Currently absent from our lens. |
| P1 | M | Name and operationalize expand-contract: renames, column drops, and type changes must be split into add-new -> dual-write/backfill -> drop-old across separate deploys, and the application code must work against BOTH the old and new schema during rollout (drain old app instances before dropping a column). Our lens says 'backward-compatible' but never names the pattern or the app-compatibility/drain requirement, which is the part engineers actually get wrong. dev.to and strong_migrations both center this. |
| P1 | S | Refine the existing 'default for new non-null column' guidance: flag that a CONSTANT default is instant on PG11+, but a VOLATILE default (now(), gen_random_uuid(), random()) rewrites the entire table — use add-nullable-column -> batched backfill -> SET NOT NULL instead. Our current advice ('specify a default') can itself cause a full-table-rewrite outage for volatile defaults; strong_migrations calls this out explicitly as 'adding a column with a volatile default value'. |
| P2 | S | Add a check: after any CONCURRENTLY operation, verify the index is valid (pg_index.indisvalid) and DROP INDEX CONCURRENTLY any leftover INVALID index before retrying. A timed-out/failed CONCURRENTLY silently leaves an unused INVALID index that blocks re-creation; documented in both strong_migrations and the dev.to piece. Not in our lens. |
| P2 | S | Recommend wiring a migration linter (squawk for Postgres, or strong_migrations/safe-pg-migrations in Ruby) into CI to fail PRs containing unsafe DDL automatically. Structural, low-effort safety net the field treats as standard; turns this lens's checks into an enforced gate rather than a manual review item. |
| P2 | S | Expand the batched-backfill anti-pattern with the operational detail: commit per batch, sleep between batches, and watch WAL bloat / replication lag; never backfill in one giant UPDATE. Our lens says 'batched updates' but omits why (WAL bloat, replication lag, lock contention) and how (per-batch commit + sleep) — the concrete mechanics in the dev.to backfill pattern. |
★ Add a required Intent & Requirements lens (with an explicit negative-requirements check) — it is the only thing that catches the ~50% of bugs, including the worst security flaws like missing-authorization, that ours' purely-structural lenses provably cannot see.
Ours already exceeds the field on several axes worth protecting from regression: (1) PHASE AUTO-DETECTION (planning / implementation / post-impl / grill) with an explicit AMBIGUOUS->ask-don't-guess fallback — HAMY and others review a static diff only; (2) the PRE-MORTEM ANALYST agent that ASSUMES FAILURE HAS HAPPENED and works backward, with a rotating bank of operational/design/integration scenarios — no comparable command has this, and it is a genuine differentiator that catches a different bug class than structural review; (3) interactive GRILL MODE (one Socratic question at a time, push on weak answers) — unique; (4) the findings -> writing-plans -> review-the-fix-plan -> wait-for-approval pipeline for impl phase (does NOT blindly auto-fix code); (5) the recursive STOP CONDITION where ANY fix resets the pass tracker and forces a full fresh re-review (not just verify-the-fix); (6) deep PRE-REVIEW CONTEXT DISCOVERY across CLAUDE.md/AGENTS.md/.cursorrules/ADRs/design docs in both .md and .html; (7) an 11-lens taxonomy that already includes Observability and Data-Integrity/Schema-Safety lenses HAMY lacks as first-class items.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a new required lens 'Intent & Requirements' (make it required alongside Guardrails). Reviewer prompt: derive the intended behavior from the conversation, plan/spec/ADR files, commit messages, and CLAUDE.md, then check the diff AGAINST that intent — flag code that is structurally correct but does the wrong thing. Explicitly include a NEGATIVE-REQUIREMENTS sub-check: 'List what this code must NEVER do, which states must be impossible, and which data must never be exposed; then verify the diff actually enforces each.' Call out missing authorization / unspecified trust boundaries as the canonical example. Directly closes the ~50% intent-violation blind spot documented by NIST SATE + Charoenwet 2024 (O'Reilly/Stellman). It is the one class of bug NO structural lens — ours included — can ever catch, and it includes the most dangerous security defects (CWE-862). Highest-value gap by far. |
| P0 | S | Add a 'SCOPE RESOLUTION' section before context discovery with an explicit priority ladder: (1) user-specified scope in $ARGUMENTS (branch/SHA/PR/paths); else (2) on a feature branch -> `git diff main...HEAD`; else (3) staged changes -> `git diff --staged`; else (4) last commit -> `git show HEAD`. Feed the resolved diff to every reviewer so phase-detection augments rather than replaces a concrete target. HAMY's command opens with exactly this ladder. It guarantees the review always has a real diff even in a fresh/compacted session, where ours' 'infer from recent conversation' heuristic silently degrades. Low risk, removes a whole failure mode. |
| P1 | M | Add a determinism pre-step (or a 13th 'Build & Test' lens) that actually runs the project's tests + linter/type-checker on the changed files BEFORE the reasoning lenses, and feeds failures in as findings. Make a clean pass impossible while tests or the type-checker fail. HAMY dedicates 2 of 9 agents to this and the author's 'close the loop' point is that giving the AI a real success signal massively improves outcomes. Ours can currently bless code that does not compile. Reuse the existing parallel-spawn machinery. |
| P2 | S | Add a final 'VERDICT' to the merge/report step: one of READY (no Critical/High, suggestions optional) / NEEDS ATTENTION (mediums or important suggestions) / NEEDS WORK (criticals/highs or failing tests), and rank suggestion-level findings by impact x effort. Also instruct: collapse every clean lens to a single one-line summary. Gives a scannable bottom line that maps cleanly onto the existing CRITICAL/MEDIUM/LOW gate and onto downstream commands (/pr, /land-and-deploy). Matches HAMY's proven verdict + impact/effort format and fixes the all-clean wall-of-text. |
★ Add a false-positive validation gate (with an evidence requirement and a 'do NOT flag' list) between reviewer findings and the fix phase — it's the one technique the entire field has converged on that ours lacks, and it matters most precisely because ours runs fixers in a recursive loop where an unvalidated false positive causes real regressions and wasted waves.
Ours already exceeds every external source on the hardest parts and must not regress: (1) RECURSIVE review-until-clean loop with fix-then-re-verify waves — neither Anthropic's nor hamy's command re-checks that a fix actually worked or introduced regressions; ours mandates a fresh full re-review of each failed lens plus verification of prior fixes, and a hard 'do NOT stop early / do NOT ask should-I-continue' rule. (2) Structural randomness for genuine coverage diversity: shuffled personas, wild cards, and a one-shot Pre-Mortem Analyst that assumes-failure-and-works-backward (a reframing none of the external commands have). (3) Lens-PAIRING (2 lenses per reviewer, depth over breadth) plus relevance-based lens SELECTION with announced reasoning, vs the field's fixed agent roster. (4) Phase detection (PLANNING/IMPLEMENTATION/POST-IMPLEMENTATION) and full plan-mode support where the parent edits the plan file. (5) Pluggable specialist lenses from ~/.claude/lenses with trigger matching and project-override. (6) Read-only reviewers + single sequential holistic fix phase by the parent — cleaner than parallel editors and lets the parent see cross-cutting concerns. (7) Conditional frontend-design reviewer and UX discoverability sub-checklist. (8) Repo-config acceleration via /jacked-setup. Already carries severity tiers (CRITICAL/MEDIUM/LOW) with LOW non-blocking, and already scans CLAUDE.md/AGENTS.md/.cursorrules/.windsurfrules for context — matching the field's context-discovery step.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a 'Finding Validation' micro-step between the wave results and the FIX PHASE (after step 8, before step 9). For each CRITICAL/MEDIUM finding, the parent (or a cheap parallel validation subagent per finding for large waves) re-confirms the issue is real against the actual code BEFORE fixing: does the cited file:line exist, does the claimed trigger path actually occur, is a CLAUDE.md rule actually in-scope and violated. Drop any finding that can't be confirmed; note dropped findings in the report as 'unconfirmed, not actioned'. Mirror Anthropic step 5 and dev.to Gate 5. Closes the single biggest gap and the highest-cost one for a recursive fixer: an unvalidated false positive makes the parent edit working code, which can introduce a regression and spawn another wave. Both Anthropic's official command and dev.to independently make this a mandatory gate. |
| P0 | S | Add an evidence requirement to the SPAWNING INSTRUCTIONS: every reported CRITICAL/MEDIUM finding MUST include (a) exact file:line, (b) the concrete trigger — input, state, or call path that produces the failure, and (c) one sentence on why it's wrong. Instruct reviewers: 'If you cannot point to the specific code path that exhibits the problem, do not report it as CRITICAL/MEDIUM — downgrade to LOW or drop it.' Match dev.to 'no evidence, no report.' Forces reviewers to self-filter speculation before it reaches the parent, and makes the P0 validation step fast (the evidence is already attached to confirm or refute). Field-standard noise control. |
| P0 | S | Add a 'DO NOT FLAG' section to the command, injected into every reviewer prompt: pre-existing issues not introduced by the change under review; formatting/style a linter catches; pedantic nitpicks a senior engineer would not raise; patterns used consistently elsewhere in the codebase; rules explicitly silenced inline (e.g. lint-ignore); purely subjective preferences. State 'False positives erode trust and trigger wasted fix waves.' All three external sources ship this exclusion list as their primary signal-to-noise mechanism; ours has no suppression guidance at all. Directly reduces wasteful re-check waves in the recursive loop. |
| P1 | S | Add a final actionable VERDICT line to both the clean-pass and cap-reached reports: Ready to Merge (all selected lenses clean, no confirmed CRITICAL/MEDIUM) / Needs Attention (LOW or advisory items remain) / Needs Work (cap reached with confirmed CRITICAL/MEDIUM still open). Include one-sentence next-step guidance. Makes dcr usable as a programmatic self-check gate in autonomous/long-running agent loops (the exact use hamy highlights: 'tell AI agents to run this and iterate before marking done'). Cheap, high-leverage, and fits jacked's autonomous commands like /bhag and /goal-maker. |
| P2 | M | In POST-IMPLEMENTATION phase, add an optional deterministic 'Diagnostics' reviewer (or a parent pre-step) that collects linter + type-checker + IDE diagnostics for changed files and runs the relevant tests, feeding concrete pass/fail/error output into the review as ground truth alongside the LLM lenses. Skip gracefully if no toolchain is detected. hamy dedicates 2 of 9 agents to this; it's cheap deterministic ground truth that complements probabilistic lens review and catches type/import/test breakage the lenses may rationalize away. Pairs naturally with the existing 'run tests in fix phase' step. |
★ Ours is best-in-class on single-process Python defensive coding but stops at retry+timeout; add the resilience tier the field treats as table stakes — retry-safety rules, circuit breakers, recoverable-vs-unrecoverable termination, swallowed-exception hunting, and partial-failure rollback.
Genuinely strong on single-process Python defensive coding and already matches or exceeds the field on those axes: None/NoneType guards, getattr-with-default, dict.get vs KeyError, guarding iteration over None/empty, fail-fast input validation at boundaries, a clean custom exception hierarchy (base AppError + specific subclasses), no-bare-except-without-reraise, retries with exponential backoff + jitter, mandatory timeouts (no infinite waits), idempotency mentioned, context managers for resources, the mutable-default-argument trap, structured logging, no leaking sensitive data, and a concrete review process with critical/high/medium/low priority ranking and per-finding fix examples. The four-pass review structure and explicit prioritized output format are better than most community 'error handling' subagents, which are vague prompt blobs.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a 'Retry safety rules' block to the resilient-I/O section: never retry non-idempotent operations (use/require an idempotency key first), never retry 4xx/client errors, never retry an already-overloaded service, and cap total attempts to avoid retry amplification in deep call chains. The agent actively recommends retries but omits the guardrails; an unguarded retry on a payment/POST is a double-charge and a thundering-herd amplifier. Directly from the Temporal 'golden rule of retries: idempotency' and 'know when to quit' sections. |
| P0 | S | Add 'circuit breaker' as a named pattern: when a dependency fails persistently, trip a breaker (closed/open/half-open) to fail fast instead of retrying forever; pair it with the retry logic so retries handle transient blips and the breaker handles sustained outages. Closes the single biggest resilience gap — retries with no breaker is the documented cascading-failure mode. Temporal devotes a full section to it as a baseline pattern. |
| P0 | S | Add an explicit 'recoverable vs unrecoverable' decision rule to the Pragmatism section: missing/invalid config, missing critical resource, or violated invariant => fail fast and terminate (non-zero exit / re-raise to top), never log-and-continue. Recoverable transient/peripheral failures => degrade and continue. The agent's 'log and continue when safe' currently lacks the criterion for 'safe', which is exactly where junior devs swallow fatal errors. SonarSource best-practice #3 names the rule precisely. |
| P1 | S | Add 'swallowed exceptions' as a first-class finding to hunt in the review passes: flag every `except: pass`, log-less swallow, and over-broad catch that drops the error. Require each caught exception to be re-raised, handled, or logged with context — never silently dropped. This is the field's #1 anti-pattern (Sonar flags empty catch blocks); making it an explicit review target raises catch rate. Our agent only addresses the narrower 'bare except without re-raise' case. |
| P1 | S | Add a user-facing-vs-diagnostic rule: at boundaries, return a safe non-sensitive error message/code to the caller while logging full diagnostic context (with a correlation/request ID) internally; never expose raw stack traces or internal paths to end users. Extends the existing 'don't leak sensitive data' line into an actionable pattern and folds in correlation-ID logging for traceability. Grounded in SonarSource (#1/security) and Temporal's structured-logging+correlation-ID section. |
| P1 | M | Add a 'partial-failure / state consistency' check for multi-step state-mutating operations: when step N fails, ensure prior side effects are compensated/rolled back (saga-style) or the operation is made atomic, so the system isn't left half-updated. The agent claims to 'fail fast when data integrity is at risk' but gives no mechanism; compensating transactions are the field's standard answer for distributed/multi-step rollback (Temporal saga section). |
| P2 | S | Add a brief 'graceful degradation / fallback' tier: when a non-critical dependency is unavailable, prefer serving stale cache, a simplified result, or disabling the feature over hard-failing the whole request — and keep fallback logic simpler than the primary path. Rounds out the resilience ladder (retry -> breaker -> fallback) that the agent currently truncates at retry. Direct from Temporal's 'Fallbacks: Plan B and C' section. |
| P2 | S | Make language-generality explicit: keep the rich Python examples but add a one-line note that the same checks map to other languages (e.g., try/catch, null/undefined, defer/ensure, error-return checking), or retitle the agent's scope as Python-focused. The description advertises generic 'error handling audit' while the body is Python-only; either widen with mapped equivalents or set scope honestly. Low-risk clarity fix. |
★ Add the 0.0.0.0/$PORT binding gotcha plus configuring Railway's healthcheckPath — the single most common 'deploy says SUCCESS but the app never serves' failure, which our skill's verify loop catches but never explains how to fix.
Best-in-class on the hard, non-obvious traps that actually eat hours: NIXPACKS can't provision bleeding-edge Node (set NIXPACKS_NODE_VERSION); npm 11 vs npm 10 lockfile optional-dep skew on `npm ci`; RLS bypass via the superuser ${{ Postgres.DATABASE_URL }} (the single best security insight here — most guides get this wrong); app-role password rotation in the migrator pre-deploy step; 'Unauthorized that actually succeeded' CLI behavior with re-verify-via-service-list; MCP-unauthenticated-while-CLI-authed; GitHub App authorization being a separate manual step from `add --repo`; `railway up` deploys the local dir not GitHub; and a strong verify-don't-trust loop (poll deployment status to terminal, then curl /health). Also strong: explicit service-topology decision table (monolith vs split FE/BE), reference-variable model, the bucket/S3 cred-sharing recipe, and a clean user hand-off output spec. Structure (model -> topology -> sequence -> env recipe -> storage -> auth -> gotchas -> output) is excellent and should be preserved.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a top gotcha (place it as the new #2, right after the Node-version one): 'App binds 127.0.0.1 or a hardcoded port -> deploy goes SUCCESS but healthcheck/crash loops.' State the rule: the server MUST listen on 0.0.0.0 and read the Railway-injected PORT env var (e.g. `app.listen(process.env.PORT, '0.0.0.0')`, `--host 0.0.0.0 --port $PORT`). Note the Dockerfile twist: exec-form CMD (JSON array) does NOT expand $PORT — use shell-form CMD with ${PORT:-8000}. Vendor-confirmed (Railway healthchecks doc) and cross-platform-confirmed (SnapDeploy/Fly/Django threads) as the single most common 'deployed but not serving' failure. It's a different root cause than our existing Nixpacks gotchas and is currently absent. |
| P1 | S | In the deploy sequence step 7 (Deploy + VERIFY), add: set the service's healthcheckPath (e.g. /health or /api/health) so Railway gates traffic cutover on a 200 before going live; note the default 300s timeout and the RAILWAY_HEALTHCHECK_TIMEOUT_SEC override for slow first boots/migrations; and warn that the check runs only once at deploy (not continuous) so it is not uptime monitoring. Railway's healthchecks doc is explicit that this is how zero-downtime cutover works. Our skill manually curls /health but never has the user configure the healthcheckPath, so a broken new deploy can replace a working one. Concrete and vendor-grounded. |
| P1 | S | Add a gotcha: 'Host-allowlist apps reject Railway healthchecks.' Railway healthcheck requests originate from healthcheck.railway.app — Django ALLOWED_HOSTS, Rails, Helmet, and similar must allow that host (or `*` for the health route) or the deploy fails with 400 / service unavailable despite a healthy app. Directly from the Railway healthchecks doc; a real, non-obvious failure mode that looks like a healthcheck bug but is an app-config issue. Saves a confusing debug. |
| P1 | M | Extend the topology table's 'Truly separate FE + BE (same repo)' row and add a monorepo note: each service needs a per-service Root Directory set (dashboard/MCP-only, like the cron config-file-path), and the railway.json/.toml config file does NOT follow Root Directory — pass its absolute path (e.g. /backend/railway.toml). Also note Railway can auto-stage one service per package for pnpm/npm/yarn/bun monorepos when imported via railway.com/new. From Railway's monorepo doc. Our split-FE/BE row says 'two services from the same repo' but omits the Root Directory step, which is mandatory for isolated monorepos and is a dashboard-only setting users will miss. |
| P2 | S | In the GitHub auto-deploy section / gotcha #6, add Watch Paths: for multi-service (monorepo) projects, set gitignore-style watch paths per service (e.g. /backend/**) so a push only rebuilds the affected service instead of redeploying every service on every merge. From Railway's monorepo doc. Without it, the 'auto-deploy on merge to main' feature our skill sells causes redundant full-project rebuilds — a real cost and noise problem for the exact split-service setup the skill targets. |
| P2 | S | Add a short gotcha on build-method precedence: any Dockerfile in the repo overrides Nixpacks and DOCKER_IMAGE; a railway.json/.toml startCommand overrides the Dockerfile CMD. If a build behaves unexpectedly, check for a stray Dockerfile or config startCommand before debugging Nixpacks. From the Medium debug write-up; explains a class of 'I set X but it ignores it' confusion distinct from our Nixpacks gotchas. Low effort, prevents a wrong-rabbit-hole debug. |
★ Add a deterministic symbol-drift pre-pass (extract the functions/classes/env-vars/CLI-flags a doc references, grep the codebase, and flag the missing ones) — it is the field's single highest-signal drift check and it turns our already-strong 3-pass verification from 'agent must notice everything' into a concrete, exhaustive worklist.
Our docs-sync is already ahead of the field on rigor and should not be regressed: (1) The 3-pass verification protocol — Pass 1 update, Pass 2 cross-check 'lying detector' (verify EVERY remaining claim: file paths via ls/Read, symbols via grep, env vars, CLI flags, code examples, cross-refs), Pass 3 'fresh reader' misdirection audit — is materially more rigorous than every external tool found, all of which are single-pass. (2) The 'misdirection is worse than absence — silence is honest, wrong instructions are not' principle is a genuinely sharp idea nobody else articulates. (3) Unverifiable claims get an explicit `<!-- docs-sync: unable to verify -->` marker surfaced to the human rather than silently dropped. (4) Staleness uses git's last-commit timestamp (not filesystem mtime, which changes on checkout) with a stat fallback for untracked files — exactly the correct, non-obvious choice the field's git-age signal also lands on. (5) Parallel one-agent-per-doc dispatch with branch-driven / fresh-audit / both modes is a clean architecture. (6) Per-doc-type agent templates (README, Wiki, CLAUDE.md, generic) with type-specific scope/style constraints. (7) Strong 'What NOT to Update' carve-out (internal refactors, auto-generated docs, scratch scripts, WIP, test-only) — which the DeepDocs auto-generated-vs-narrative distinction independently validates. (8) Repo-config overlay + repo-scoped override resolution for speed. (9) Suggest-don't-auto-create for new doc gaps, and stage-don't-commit handoff.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a deterministic 'Symbol drift' sub-step to Step 2.5 (and feed its output to the agents): from each candidate doc, extract backticked tokens and code-fence-referenced identifiers (functions, classes, env vars, CLI flags, file paths), then grep/ls the repo to confirm each still exists. Emit a per-doc list of missing/renamed symbols. Use it two ways — (a) any doc with missing symbols is added to the audit queue even if untouched and not 30d stale, and (b) the missing-symbol list is passed into the agent prompt as a concrete Pass-2 worklist ('these references could not be found in the codebase — verify or fix each'). Note the regex-on-backticks starter and that tree-sitter/LSP is the upgrade path for polyglot repos. Guard against false positives: dotted paths like `Cls.method` count as a hit if all components resolve; skip docs tagged as changelog/historical. Highest-leverage gap. It is the field's #1 deterministic drift signal (Dosu freshness; 2024 ESE study found 28.9% of repos reference a symbol that no longer exists), it is cheap, and it makes our already-strong Pass 2 concrete and exhaustive instead of relying on the agent to notice every claim unprompted. |
| P0 | S | In Step 1 discovery, run `ls -la` (or `readlink -f`) on every root-level markdown file and resolve symlinks before dispatch. Record the canonical (real) file vs aliases; only ever dispatch an agent against the canonical path; if a user/config targets an alias, redirect to the target. Report the symlink relationship in Step 6. If CLAUDE.md and AGENTS.md exist as separate non-symlinked files, treat both independently AND flag content divergence between them. Concrete failure mode /doc-it actually shipped a fix for: editing the alias leaves the canonical agent-instructions file stale, or the two diverge silently. CLAUDE.md→AGENTS.md symlinks are now common. |
| P1 | S | Add AGENTS.md to the doc inventory everywhere CLAUDE.md appears (Step 1 globs, Step 2.5 DOC_CANDIDATES, Step 3 mapping table, and add an AGENTS.md agent template — or generalize the CLAUDE.md template to 'agent-instruction files (CLAUDE.md / AGENTS.md)'). Adopt the field's 'three doc layers' framing in Step 1 — user-facing, contributor-facing, agent-facing — so inventory completeness is checked against a model. AGENTS.md is now a first-class, cross-tool agent-facing doc surface; omitting it means a whole category of drift goes unaudited. The three-layer model is a clean completeness check. |
| P1 | S | Change the diff baseline in Step 2 from the base-branch tip to the merge-base: `MERGE_BASE=$(git merge-base origin/${BASE_BRANCH} HEAD)` and diff `${MERGE_BASE}...HEAD`. (Three-dot already approximates this, but compute and reference merge-base explicitly and use it for the age-delta comparison too.) Dosu explicitly documents that comparing against the moving base tip inflates drift on long-lived branches and re-flags changes already merged to main; merge-base is the correct, stable reference. |
| P1 | M | Replace the single hard-coded 30-day staleness cutoff with a tiered/configurable threshold: honor an optional per-doc `ttl_days` (or `freshness:`) frontmatter contract if present; otherwise fall back to externally-grounded defaults (age 90d / drift 30d, per docvet) rather than one global 30d. Let the existing Repo Config overlay set repo-wide defaults. One global cutoff over-flags slow-moving architecture docs and under-flags fast-moving quick-starts. Per-doc TTL contracts + docvet defaults are the field-standard, calibration-friendly approach (Giant Swarm runs this in production). |
| P2 | S | Add changelogs / historical / migration docs to 'What NOT to Update' (and exclude them from the staleness + symbol-drift queues), since they legitimately reference removed code and old versions. Allow an explicit `freshness: { exclude: true }` frontmatter opt-out. Dosu calls this out as a concrete false-positive source: scoring CHANGELOG.md like a living doc flags every 'replaced X with Y' line as drift. Cheap correctness win. |
| P2 | M | (Optional) Add a lightweight per-doc freshness summary to Step 6 — e.g. for each audited doc print fresh/drifted/needs-human plus the count of missing symbols and days-since-edit — so a human (or a future CI wrapper) has a trackable signal. Keep it advisory; do not gate. The field's main 'make aging visible over time' surface (Dosu's 0-100 score / README badge / SLO). A minimal version costs little and turns one-shot output into a trend a team can watch. |
★ Already structurally ahead of the public field (dual-mode, tenant-aware, lens-assignment, read-only) — the only real upgrades are operationalizing tenant-isolation per the OWASP Multi-Tenant cheat sheet, adding deployment/observability checks to the code deliverable, and an Impact/Effort anti-noise filter that real-world setups credit for going from <50% to ~75% useful findings.
Ours is already ahead of the public field on structure and depth. Dual-mode operation (Design Review = Principal Architect with independent research + first-principles + risk; Code Review = combined CTO+CSO) is more sophisticated than any community reference found. The multi-tenant / cross-org data-leak section is called out as CRITICAL with data-flow tracing — the VoltAgent canonical reviewer has no tenant concept at all. Domain-specific lens groups (Web/API, CLI, Data Pipelines, Libraries, Infra/DevOps) give breadth the flat community checklists lack. The /dcr lens-assignment protocol (depth-over-breadth, per-lens report structure, strict READ-ONLY discipline, footnote-only out-of-scope) is genuinely best-in-class and absent from every external source. Severity bucketing (Critical/Warning/Win, APPROVE/NEEDS CHANGES/BLOCK) and the project-context discovery fallback are solid. It also fits a strong surrounding ecosystem: /dc and /dcr dispatchers and a separate /cso skill that already owns deep OWASP Top 10 + STRIDE, so the agent correctly does not try to re-implement a full threat-model engine.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Expand the 'Cross-Organization Data Leaks' subsection into an operational tenant-isolation checklist grounded in the OWASP Multi-Tenant cheat sheet: (a) tenant context MUST come from the verified token/session, never from client headers or request params; (b) use composite-key lookups (tenant_id + resource_id) at the data-access layer, not just the API layer; (c) verify tenant prefixing on cache keys, blob/file-storage paths, and presigned URLs; (d) flag any admin or 'internal service' path that bypasses tenant filters; (e) confirm not-found returns 404 not 403 (no existence disclosure). Keep it as inline review prompts, not a full engine (the /cso skill owns deep OWASP+STRIDE). Ours flags the right risk category but at a level too abstract to reliably catch real leaks; the OWASP cheat sheet enumerates the exact vectors (cache, storage, header-trust, admin bypass) ours silently misses. Grounded directly in cheatsheetseries.owasp.org Multi-Tenant cheat sheet. |
| P1 | S | Add a 'Deployment & Migration Safety + Observability' block to the CODE REVIEW MODE deliverable (not just the Infra lens group): table-locking / non-reversible migrations, backwards-compat with existing prod data, rollback safety, feature-flag rollout for risky changes, and 'if this fails in prod, how would we know?' (logs/metrics/alerts on critical paths, no silent failures). Both real-world references make this a first-class reviewer; it is the most common gap between 'tests pass' and 'safe to ship'. Sourced from hamy.xyz Agent 8 (deployment safety) and Agent 4/observability patterns. |
| P1 | S | Add a signal-quality / anti-noise directive to GENERAL PRINCIPLES: rank findings by Impact AND Effort (not severity alone), cap routine reviews to the most important findings, and explicitly skip formatting/naming nitpicks and anything a linter or type-checker already catches. Preserve full severity buckets for security/data-isolation findings (never suppress a Critical). Both real-world setups attribute their jump from <50% to ~75% useful suggestions to exactly this discipline; AI-reviewer noise is the top complaint in the sources. From hamy.xyz Agent 3 prompt ('up to 5, ranked by impact and effort, skip things linters catch'). |
| P2 | S | Change frontmatter from 'model: opus' to 'model: inherit' (or document the Opus choice deliberately), noting that /dcr spawns multiple reviewers in parallel waves. The 22k-star canonical community reviewer made exactly this change; hardcoding Opus per-reviewer is costly in parallel-wave dispatch and inherit lets the session pick the tier. From the VoltAgent code-reviewer.md commit history ('switch subagent model from opus to inherit'). |
| P2 | S | In DESIGN REVIEW MODE, instruct the agent to prefer primary/authoritative sources (official docs, standards, OWASP) when doing independent research and to cite the specific sources behind each challenged assumption. Code Review mode is rigorously specified but Design Review's research step is vague ('web search or your knowledge'); citing sources makes the fresh-eyes validation auditable. Grounded in how the surrounding /cso and research skills in this repo treat sourced, confidence-rated findings. |
★ Solid, fast, symlink-correct single-dir edit guard — to be best-in-class it should fail CLOSED on corruption, honestly disclose the Bash-write escape hatch, and grow to multi-path/per-project scope like the native permission system it shadows.
Resolves symlinks via Path.resolve() before the boundary test, so a symlink inside the frozen dir pointing outside is correctly caught (matches the official 'deny applies to target' posture). Forward-slash normalizes for Windows (matches the doc's POSIX normalization). Deterministic <1ms boundary check that runs UNCONDITIONALLY — before the gatekeeper's enabled/LLM gates — so it works even when the LLM gatekeeper is off (default). Reads/Grep/Glob/Bash-inspection stay unrestricted, which is the right scope for 'focused work'. Clean one-command UX (/freeze, /unfreeze) is lower-friction than hand-editing settings.json deny rules. The confirmation message already enumerates what's blocked vs. not — good instinct, just incomplete (see gaps).
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Make the freeze hook fail CLOSED on a corrupt/unreadable freeze file: if the file exists but can't be parsed into a valid directory, DENY Edit/Write/NotebookEdit with a clear 'freeze file corrupt — run /unfreeze to reset' message instead of the current pass/allow. Update freeze.md/unfreeze.md to state the fail-closed behavior. The freeze is a deliberately-enabled safety boundary; silently evaporating it on corruption is the opposite of what the user asked for. Matches blocking-hook security consensus and the fail-closed pattern in the official managed-settings (forceRemoteSettingsRefresh). |
| P0 | S | Disclose the Bash escape hatch in the /freeze confirmation message: add to 'What's NOT blocked' that file writes performed via Bash subprocesses (sed -i, redirects, python/node scripts) are NOT caught by the freeze boundary, with a one-line pointer to enable the OS sandbox or a Bash deny rule for true isolation. The official permissions doc states Edit deny rules don't cover subprocess writes; the Cursor 'External-File Protection' thread is a live example of users over-trusting a protection whose real scope was narrower than its name. Honest scope disclosure prevents that exact failure. |
| P1 | M | Support multiple frozen paths and optional sub-path excludes: let /freeze accept more than one path (append, not stomp) and an optional 'except <subpath>' so a file is allowed only if it's under an included path AND not under an excluded one. Store as a small JSON list rather than a single line; the hook checks each entry. Native Claude Code permissions already express this via multiple Edit(/glob/**) rules and gitignore semantics, and issue #28321 shows demand for scoped, multi-path edit rules. A single dir is the most common but the field's baseline is multi-path + excludes. |
| P1 | M | Scope the freeze per project/session instead of one global file: key the stored boundary by the resolved project root (CLAUDE_PROJECT_DIR) so a freeze in repo A doesn't govern repo B opened elsewhere, and /freeze in a new repo doesn't silently overwrite an existing one. The hook already computes repo_path — match the active freeze entry to it. The official permission model scopes per project; a single global value causes cross-repo bleed and silent stomping that the user never sees until a confusing denial. |
| P2 | M | Have /freeze verify the gatekeeper hook is actually installed/registered before reporting success; if not, warn that the boundary will not be enforced and tell the user how to install jacked hooks. Also record a timestamp and surface an 'active since <date>' line in the confirmation and on each subsequent denial. Writing a freeze file that does nothing (hook not wired) is a silent-no-op trap; the awesome-claude-code/hooks ecosystem treats 'is the hook wired' as a first-class check. A timestamp addresses the no-staleness gap so week-old freezes are obvious rather than mysterious. |
★ Solid PR-authoring/strategy agent that already nails sizing and stacked-PR retargeting, but it misses the field's core 'split commits, not just PRs' interactive-staging technique, has no stacking-tool awareness or cascading-rebase warning, uses fragile parent/default-branch detection, and lacks staging-safety and AI-attribution guards.
Already covers the core lane well: change analysis/categorization via git status+diff, the <200-300 line reviewability threshold, branch-prefix conventions, one-logical-change-per-PR, a stacked-PR section with base-branch retargeting and post-merge rebase, a structured PR template, Conventional Commits type/scope/subject format, a red-flags checklist (mixing features+fixes, 'various fixes' messages, >10 files), and proactive-intervention framing. Its stacked-PR retargeting flow (open PR2 with base=feature-1, then gh pr edit --base main after PR1 merges) matches field practice. Scope is cleanly distinct from sibling agents pr-workflow-checker (post-coding state/stash/worktree triage) and issue-pr-coordinator (issue grouping/linking), so it owns authoring + splitting strategy.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add an 'Atomic commit construction' section: when a working tree mixes concerns, use interactive staging (git add -p for hunk-level, git add -i, git restore --staged to unstage) to build one focused commit per logical change, so a single push can still be reviewed piecewise. State the principle explicitly: 'the reviewable unit is the commit, not just the PR — split commits first, then decide if they warrant separate PRs.' This is the single highest-consensus technique in the field (r/git top answer 'split commits not PRs'; Pragmatic Engineer 'every commit becomes a code review'; Gerrit/spr one-commit-per-PR model) and it's the concrete mechanic behind the agent's own vague promise to 'suggest logical groupings.' Currently absent. |
| P1 | M | Add stacked-PR tooling awareness: detect Graphite (gt), git-town, Sapling (sl), spr, or ghstack and prefer the tool's stack-sync over raw rebases; if none present, keep the manual flow but add an explicit warning that when an earlier PR in the stack changes you must rebase EVERY descendant in order (cascading rebase) and force-push each, and that the risk grows with stack depth. Pragmatic Engineer and awesomecodereviews both stress native-git stacking is tedious/error-prone and that dedicated tools exist precisely for this; the agent's hand-rolled rebase guidance under-warns about the cascading failure mode that bites real users. |
| P1 | S | Replace the git show-branch parent-detection bash with robust signals: try gh pr view --json baseRefName for the current branch, then git rev-parse --abbrev-ref @{upstream}, then merge-base against candidate branches; and resolve the default branch dynamically via gh repo view --json defaultBranchRef (fallback git symbolic-ref refs/remotes/origin/HEAD) instead of hardcoding 'main'. The show-branch|grep heuristic is a known-flaky hack and the agent hardcodes main everywhere; both produce wrong base branches on real repos (master/develop/trunk), which silently mis-targets PRs — the exact mistake this agent exists to prevent. |
| P1 | S | Add a pre-commit staging-safety guard to operational guidelines and red flags: never use git add -A/git add .; stage only explicitly named/confirmed files; scan the staged set for likely secrets (.env, *.pem, key/token patterns) and large/binary blobs and refuse to include them, pointing the user to .gitignore. Medium FAQ and broad ecosystem consensus treat 'don't over-stage, never commit credentials' as table stakes; the current red-flags list omits secrets, large files, and over-staging — a meaningful safety gap for an agent that runs git add. |
| P2 | S | Add a short 'AI attribution' note: whether to include a Co-Authored-By: Claude trailer (or [Claude] prefix) is governed by the repo — read CLAUDE.md / existing git log for the team's convention and follow it; do not invent or strip it unilaterally. It's a live, contested ecosystem topic (deployhq guide, GitHub issue #29999, r/git) and our own sibling pr-workflow-checker already emits the trailer, so the suite is internally inconsistent; deferring to CLAUDE.md resolves both. |
| P2 | S | Add a 'verify before submit' step to Workflow Enforcement: self-review the full diff and run the project's build/test/lint before committing or opening the PR; surface failures rather than pushing red. The Claude-Code git playbook (Medium) frames 'review the diff before committing' as the cheapest safety net, and 'missing tests for new functionality' is already in our red flags but never enforced as a gate — closes that loop. |
| P2 | S | Extend the PR template with 'Risk / rollback plan' and 'Out of scope' lines. PR-description guidance from the scraped sources emphasizes telling reviewers what to watch out for and bounding the change; cheap addition that improves review focus. |
★ Already best-in-class on evidence-based DONE, bounded loops, and the LLM-judge mechanics; three small, grounded additions — force proof the test runner actually ran, grep-for-reuse before writing new code, and observable per-milestone acceptance signals — close the exact failure modes documented in real autonomous-agent runs.
Already best-in-class on the fundamentals the field converged on: (1) evidence-not-asserted DONE — 'Success is shown as evidence, never asserted' (line 47), which is precisely the verification-gap Nate's piece names. (2) Converge-in-one-run framing with the LLM-judge-reads-transcript mechanic spelled out (line 61), matching the Ralph-loop consensus. (3) Bounded backstop with a 'BLOCKED:' halt distinct from DONE, and a three-condition stop in MERGE mode (line 74) — directly the 'objective stop condition, not vibes' the overnight threads demand. (4) Per-milestone commits for resumable state; ordered independently-verifiable milestones; green-CI-gated merge with true merge commits and no protection bypass. (5) Hard 4,000-char measure-don't-eyeball gate with a file-backed pointer-goal fallback. (6) Prompt-injection hygiene (read-in content treated as DATA). (7) UI/UX polish + TDD quality bars. This already exceeds most published guidance.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | In the Verify section (Step 5 templates + Step 3 Tests bar), add an explicit 'the runner actually ran' check: the test command must print a real pass/fail summary with a non-zero test count — an empty, errored, or 'no tests collected' output counts as FAILED, never DONE. Add one sentence to the DONE line: 'If the test command did not actually execute (empty output, runner/env error, 0 tests), that is a BLOCKED halt, not completion.' Directly closes the Martin Fowler Codex failure: a PR that looked done but carried two failing tests the agent never ran. The judge only sees the transcript, so without forcing a non-empty pass/fail summary, an unrun suite reads as satisfied — the exact verification gap Nate's piece names. |
| P1 | S | Add one line to the Approach block: 'Before writing new code for a milestone, grep/search for an existing implementation or helper that already does this and reuse/extend it — do not create a parallel duplicate.' Fowler's experiment quantifies the cost: 4 of 6 autonomous runs duplicated code that already existed because they never searched first. This is the cheapest, highest-yield anti-duplication step and the one low-supervision agents most reliably skip; our brief currently only says 'match existing patterns,' which doesn't trigger the search. |
| P1 | S | Tighten the milestone-authoring guidance in Step 2/Step 4: for any milestone whose success isn't a binary command exit, state the acceptance criterion as an observable before→after / input→output outcome (exact status code, error string, visible state) rather than a vague verb like 'handle' or 'improve.' One added sentence, plus swapping the template comment to e.g. '<milestone — concrete deliverable + its observable acceptance signal>'. The micro-specs source documents that abstract criteria ('handle errors properly') yield flaky, interpretation-heavy tests, while naming exact observable outcomes makes each milestone judge-verifiable — the 'turn taste into checkable constraints' skill Nate's piece calls the scarce one. This makes the transcript-verifiable DONE the brief already aims for actually achievable. |
★ Ours already beats the field on multi-issue grouping/PR portfolio planning, but it misses the field's table-stakes triage primitives — duplicate detection, acceptance-criteria-as-contract, a human-not-author review gate, and draft-PR-with-live-checklist — and is broken for the shareable suite by a hardcoded 'jackneil'/'jack_' username and branch convention.
Ours is genuinely strong on the dimension most of the field ignores: strategic MULTI-issue grouping into coherent PRs by component affinity, root-cause similarity, feature complementarity, and sequential dependency, with scope caps (5-7 issues/PR). Copilot and Dosu are essentially single-issue agents; ours plans a portfolio. It also has a clean structured status report format, issue->PR linking with 'Fixes #' auto-close, testing strategy differentiated by issue type (BUG reproduce-then-regression-test, FEATURE/REFACTOR/ENHANCEMENT), an explicit prioritization decision framework (critical bugs > security > blocked deps > high-value features > tech debt), a pre-merge QA checklist, and reads existing Claude plans via the has-claude-plan label. These match or exceed field practice for the planning/coordination half of the job.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a 'Duplicate & relationship detection' step before grouping: compare each open issue's title/body against other open AND recently-closed issues; flag likely duplicates and propose closing them with a cross-reference comment ('Duplicate of #NN'), and surface blocks/blocked-by and related links. Make this an explicit first-line triage output alongside grouping. Deduplication is the single most-cited triage capability across all three sources (Dosu lists it as a core module; both GitHub community answers say 'close duplicates immediately with references'). Ours has zero dedup logic despite scanning all open issues — a clear, grounded gap. |
| P0 | S | De-hardcode the user/branch conventions. Replace the literal 'jackneil' PR filter with `gh api user --jq .login` (or an optional parameter), and replace the prescribed 'jack_YYYYMMDD_<n>' branch pattern with a detected/configurable convention (inspect existing branch names; fall back to a neutral default like '<login>/<issue#>-<slug>'). claude-jacked is explicitly a SHAREABLE suite; a hardcoded username and personal branch prefix silently break grouping/PR-listing for every other adopter. Concrete and load-bearing for the product's stated purpose. |
| P1 | S | Add an explicit human-review gate to the PR-management section and QA checklist: AI-authored PRs must be reviewed/approved by a human who is NOT the agent or the issue creator before merge; request reviewers per repo CODEOWNERS/rules; never self-approve or auto-merge. GitHub's own coding-agent guidance enforces 'the issue creator can't be the final approver' precisely because AI-generated code shouldn't merge unreviewed. Ours' QA checklist checks tests/CI but has no who-approves guardrail — a named failure-mode the field handles. |
| P1 | S | Make acceptance criteria the input contract: in IMPLEMENTATION PLANNING, extract acceptance criteria / definition-of-done from the issue (or its template); if absent, ask the user to confirm them before coding. Mirror them as a checklist in the PR description and verify each before marking ready. GitHub blog and both community answers identify issue templates with 'description + acceptance criteria' as the #1 driver of good agent output. Ours plans implementation but never anchors on a testable 'done', risking scope drift. |
| P2 | M | Switch to draft-PR-early with a live task checklist: open a draft PR linked to the issue(s) at the START of implementation containing a task checklist derived from the plan/acceptance criteria, tick items off and push iteratively, then mark ready-for-review when the QA checklist passes. Both Copilot's workflow and the community answer (draft PRs for WIP) converge on this for visibility and reviewability. It's a concrete structural improvement over ours' create-PR-only-at-the-end model. |
| P2 | M | Adopt a standard label/hygiene vocabulary and optional Project-board move: priority labels (P0-critical..P3-low), status labels (needs-reproduction, waiting-for-response, ready-for-review) instead of ad-hoc 'in progress', and — when a Project board exists — move grouped issues Backlog->In Progress->Review->Done in step with PR state. Directly from GitHub community consensus (#163134) and standard gh/Actions practice; gives ours a portable, conventional taxonomy and board sync rather than the single ad-hoc 'in progress' label it currently prescribes. |
★ Add a written spec (acceptance criteria + explicit non-goals) before planning AND an evidence-backed end-to-end 'prove it actually runs' gate (wiring in the existing /qa or /verify) before shipping — turning a strong artifact-review loop into a strong behavior-verified one.
jack-it-up is already a strong orchestrator and matches the field's architecture. It nails: (1) the canonical Explore->Plan->Implement->Verify->Ship loop the Anthropic docs prescribe, with a real iterate-until-clean cycle (phases 4-5 repeat until /dc passes with no CRITICAL/MEDIUM); (2) adversarial, fresh-context review delegated to /dc, which already does multi-lens review + a pre-mortem analyst + the findings-to-reviewed-fix-plan pipeline (this is best-in-class and exceeds most peer commands); (3) subagent-driven execution giving fresh context per task with two-stage (spec then quality) review and per-task TDD+commit — exactly the 'fresh context for review' / 'reviewer isn't the author' pattern Anthropic recommends; (4) explicit anti-'good enough' framing (Red Flags table, the 10/10 mindset, 'thoroughness vs scope creep' distinction) that most peer workflows lack; (5) a clear 'When NOT to use this' section matching Anthropic's 'if you could describe the diff in one sentence, skip the plan'; (6) reviewed-plan-before-execution gate (phase 3) — the planning gate the spec-driven crowd considers essential. Do not regress any of these.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a written spec artifact to Phase 1. After brainstorming, require capturing a short spec (user-facing goal, acceptance criteria as testable bullets, explicit NON-GOALS / out-of-scope, and the files/interfaces involved) — either inline at the top of the Phase-2 plan or as docs/superpowers/specs/{YYYY-MM-DD}-{slug}.md. State that every plan task must trace to an acceptance criterion, and that /dc's plan review (Phase 3) checks the plan against the acceptance criteria. Mirror Anthropic's 'let Claude interview you with AskUserQuestion' as the way to elicit it when the feature is large. Both authoritative sources make the written spec/acceptance-criteria+non-goals the central artifact that prevents AI scope creep and gives the reviewer something concrete to verify against. Ours has the mindset ('thoroughness vs scope creep') but no artifact to enforce it. |
| P0 | S | Add an explicit end-to-end verification gate as a new step between Phase 5 (clean /dc) and Phase 6 (Ship). Instruct: before /pr, actually run the feature and capture evidence — invoke the sibling /qa or /ux skill for UI changes (or /verify / run-the-app for non-UI), confirm the original acceptance criteria pass against the running system, and paste the evidence (test output, command+result, or screenshot). Do not ship on green tests + clean review alone. Anthropic explicitly says the spec should end with an end-to-end step 'that proves the feature works' and warns about the 'trust-then-verify gap' where code looks done but isn't exercised. /dc reviews artifacts but never runs the app; claude-jacked already ships qa/ux/qa-video/verify that this skill simply doesn't reference. |
| P1 | S | Add context-hygiene guidance at phase boundaries. After the brainstorm/spec is captured and again after the plan is reviewed-and-saved, instruct the main thread to /clear (or start a fresh session) and re-anchor on the saved spec+plan files before executing — since those artifacts persist on disk, nothing is lost. Add 'context rot / kitchen-sink session' to the Red Flags table with 'fix: /clear and re-load the spec+plan' as the remedy. The Anthropic docs name the kitchen-sink session and context rot as top failure modes and prescribe /clear between phases; the cookbook/research consensus confirms quality degrades as the window fills. A multi-phase orchestrator run entirely in one thread is the textbook trigger. Ours gets fresh context for execution/review via subagents but never resets the orchestrating thread. |
| P1 | S | Require evidence at the 'done' gate. Change 'Do NOT declare done until the final /dc review passes' to also require attaching evidence: the passing test command + its output, and for the feature the end-to-end run result/screenshot from the new verification gate. Frame as 'show evidence, do not assert success.' Anthropic: 'Have Claude show evidence rather than asserting success... it works for sessions you weren't watching.' Directly raises trust for unattended/overnight use and closes the gap where a run claims completion without proof. |
| P2 | S | Optional: in Phase 2 planning guidance, add a one-line preference for vertical-slice task decomposition (smallest user-visible behavior end-to-end, UI-first-with-mocks then wire down) over horizontal layer-by-layer steps, when the feature spans multiple layers. The spec-driven .NET library calls horizontal slicing 'the one most teams get wrong with AI' because integration mismatches surface only after building all layers in isolation. Cheap to add as guidance; superpowers:writing-plans handles the mechanics. Lower priority since it's a refinement, not a missing gate. |
★ Add meta-conversation filtering + a JIT freshness step so recall stops surfacing its own prior /jacked sessions and never silently misses the work you just finished — the two failure modes the field's most mature competitor built explicit features for and ours is fully exposed to.
Ours is the most architecturally advanced tool in the field and must not regress these: (1) TRUE SEMANTIC/vector search via Qdrant — every competitor is keyword/FTS5 only and even lists 'vector embeddings for semantic similarity' as a future wishlist item; ours already has it. (2) CROSS-MACHINE + TEAM sharing — Qdrant Cloud index surfaces teammates' sessions as @username with a YOU/@teammate distinction; no competitor does multi-user. (3) CONTENT-TYPE-AWARE indexing — separately indexes plan files (📋), subagent summaries (🤖), summary labels, and user messages, not just raw text. (4) SMART-MODE retrieval with explicit token budgeting (~5-10K vs 50-200K) to prevent context explosion — a genuine differentiator vs dumping transcripts. (5) AGE-BANDED STALENESS WARNINGS (7-30/30-90/90+ days) telling Claude how much to trust recalled code. (6) LOCAL-vs-REMOTE handling: prefers native `claude --resume` for local sessions (full memory state) and only injects context for remote ones. (7) Honest prerequisite gate that STOPs and degrades gracefully when the search extra/Qdrant isn't configured instead of silently failing.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a 'Meta-conversation filtering' note to the SKILL instructing Claude to drop, or visibly de-prioritize, search hits that are themselves /jacked recall sessions (e.g. preview dominated by 'jacked search'/'jacked retrieve' output or matching the jacked repo's own session-search content), so the picker shows real work, not prior recalls. Ideally back this with an indexer/searcher exclusion, but at minimum add the instruction so Claude filters in-prompt. Ours indexes every session via the Stop hook, including /jacked sessions themselves — so recall pollutes its own results. This is a known failure mode akatz-ai built an explicit feature for; ours has the exact same exposure and no mitigation. |
| P0 | S | In Step 1, instruct Claude to ensure the index is fresh before searching — run `jacked index` (or `jacked backfill --repo .`) for the current repo first, or explicitly note that the most recent in-flight session may not yet be indexed and to fall back to local transcript/git if a just-discussed item isn't found. akatz-ai's commit history documents the precise silent-miss bug: a fixed index window means searching a wider range returns stale/missing recent sessions. Ours leans entirely on the Stop hook, so the current session and very recent ones can be invisible at search time with no fallback guidance. |
| P1 | S | Broaden the frontmatter description to add the fix-recall and temporal trigger phrasings the field converges on: 'how did we fix X', 'where did I debug Y', 'what did I work on yesterday/this week', 'search my history', 'what was that command/approach'. Keep it under the description length budget. Definite and alexop both show the dominant real queries are fix-recall ('where did I debug the K8s OOM') and temporal ('what did we do yesterday'). Ours under-triggers on both, so the skill silently doesn't activate when it's most useful. |
| P1 | M | Add a short 'Temporal / digest queries' branch: when the user asks what they did in a time window rather than about a topic, run a recency-ordered listing (jacked search with a broad query or a repo filter, sorted by age) and summarize per session as: repo, branch, files touched, key decisions — instead of forcing a topic match. Both competitors ship a daily-digest/standup path as a headline feature; it's a distinct query class ours doesn't handle. The data (repo, age, plan, summaries) is already indexed, so this is mostly a SKILL-instruction addition. |
| P2 | M | In the smart-mode retrieval description (Step 3 / Retrieval Modes table), add that when present the output should surface 'files touched' and 'key commands run' from the past session as a compact recall block, and tell Claude to lead its post-injection summary with them. alexop's writeup identifies files-touched + commands-run as the cheapest, highest-signal artifacts for actually recreating a past solution — the core 'no more re-solving' value. Ours already parses tool uses during indexing, so exposing them is low-cost, high-payoff. |
| P2 | S | Add a one-line hygiene note to skip Claude Code sub-agent sidecar transcripts (files named like `agent-*` / agent sub-sessions) when enumerating or any local-fallback path, so subagent noise doesn't appear as standalone results. Definite explicitly skips `agent-` files for exactly this reason; if any jacked path enumerates local transcripts (search fallback, recover sibling), the same noise applies. |
★ Already ahead of the field on scope and safety; the one real, field-validated gap is drift — it generates a frozen repo snapshot with no staleness signal and a destructive (non-merge) regenerate, so add a repo-state fingerprint + drift nudge and turn regeneration into a config-preserving merge.
jacked-setup is already well ahead of the public field on scope and safety. The field's prior art is almost entirely single-CLAUDE.md generators (ClaudeForge, /init); jacked-setup instead profiles the repo and emits per-command standalone files with pre-filled config, which nothing in the ecosystem does. It already nails several things the field only talks about: (1) gatekeeper-safe, bounded discovery (every find/grep uses -maxdepth/--exclude-dir); (2) credential safety (variable names only, never values, with infra-cred exclusions); (3) a floor check that refuses to generate useless config on empty repos; (4) explicit consent before overwriting existing files, with OLD_FORMAT vs STANDALONE detection; (5) a .gitignore check so teammates actually receive the files; (6) reads the engine via the Read tool at generation time rather than from memory (avoids hallucinated engine bodies); (7) a stable ## Repo Config header contract for re-runs. It also already stamps each file with generation date + engine version and documents the upgrade/re-run path. This is a mature command.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a staleness self-check in the engine's ## Config Override path and/or in Step 6. Bake a lightweight repo fingerprint into ## Repo Config at generation time (e.g. generation commit SHA, plus the detected stack/manifest set). When a generated command later runs, or when /jacked-setup is re-invoked, compare current HEAD/manifests against the stamped fingerprint and surface a one-line 'this config was generated N commits ago and the stack/paths may have changed — re-run /jacked-setup <target>' nudge. Keep it advisory, never blocking. Directly closes the field's #1 documented failure mode ('an outdated CLAUDE.md is worse than a lean one'; the r/ClaudeAI staleness thread). jacked-setup already stamps a date + engine version, so adding a comparable repo-state stamp + a drift nudge is a natural, grounded extension — not an invented feature. |
| P1 | M | Make regeneration of a STANDALONE file a merge, not a clobber: when the target already has a ## Repo Config the user may have edited, re-read the existing config, show the user a diff of changed inferred values, and preserve any user-added lines (custom lens weights, extra wild cards, corrected paths) rather than discarding them. Mirror /init's 'suggest improvements rather than overwrite' behavior. Claude Code's own /init explicitly does this (per the ahumanengineer field report), and the ecosystem treats existing config as something to improve, not overwrite. jacked-setup's current binary regenerate-or-skip silently destroys hand-tuning, which discourages exactly the periodic re-runs the tool needs users to do. |
| P1 | S | Add a confirmation/verification step for high-risk inferred values before writing. After Step 3 inference, echo the key inferred values back (detected dev-server port, build/test commands if surfaced, browser tool, component paths, lifecycle stage) and either let the user correct them in one pass, or cheaply validate them (e.g. confirm the inferred component/path dirs still exist via the ls results already gathered) before baking them into the committed file. The 'Never Run /init' / init-gotcha discourse is specifically about auto-generated config asserting wrong commands and paths that then mislead the agent. jacked-setup commits these files for the whole team, so a wrong inferred port or path propagates. A confirm-before-bake pass is the field's standard mitigation. |
| P2 | S | Reframe the command's framing and Step 6 announcement to position it as a recurring refresh, not one-time setup: add an explicit 'when to re-run' trigger list (stack change, new planning docs, lifecycle shift, major refactor of component/API paths) alongside the existing engine-upgrade path and young-repo note. Field consensus (living-document maintenance; the reflection-command pattern) is that generated config must be refreshed as the project evolves. jacked-setup already hints at this for young repos and engine upgrades; generalizing it to codebase-evolution triggers is a small, grounded copy change that nudges the maintenance behavior the tool depends on. |
★ Ours is already competitive with the best public land-and-deploy command and has a deeper standalone canary, but it should fix three correctness/UX gaps the field handles — revert the captured merge SHA (not blind HEAD), actively wait on pending CI, and warn that DB migrations make revert non-symmetric — plus add scope-aware verification depth and a health-endpoint smoke check.
Ours already covers the core pipeline well and is competitive with the best public equivalent: identify-PR auto-detection with PR-number/skip-canary/revert arg modes; pre-merge readiness reporting (CI + reviews + mergeable); squash-by-default with clear merge-failure causes; broad platform detection (Railway/Vercel/Fly/Heroku/GH Pages/Netlify); per-platform deploy-wait with polling intervals; always-on HTTP smoke test plus a condensed single-pass canary that hands off to a dedicated /canary command for the full monitoring loop; git-revert (history-safe) revert mode; a structured deploy report; and explicit Hard Rules (never force-merge, always wait for deploy). The dedicated canary.md is notably deeper than the competitor's inline health check (baseline capture, console/DOM/perf/network deltas, new-vs-pre-existing distinction).
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Capture the merge-commit SHA and timestamp at merge time (gh pr merge ... then record the resulting SHA), and make revert mode target that specific SHA (git revert <merge-sha> -m 1) instead of HEAD. Fall back to a revert PR if branch protection blocks a direct push to main. Ours reverts HEAD blindly; if anything merged after, the revert hits the wrong commit. The competitor records and reverts the actual merge SHA — a correctness fix, not just a nicety. |
| P0 | S | In Step 2, when required checks are PENDING, actively wait via 'gh pr checks --watch --fail-fast' (or poll gh pr checks) with a bounded timeout (e.g. 15 min) before deciding; only stop on FAIL or timeout. Record CI wait duration for the report. Ours stops on pending CI, forcing the user to re-run. The competitor watches CI to completion. This is the single biggest UX gap between 'one command from approved to verified' and 'stops halfway'. |
| P0 | S | Add a DB-migration / irreversibility caveat to the Hard Rules and to revert mode: detect migration files in the diff (migrations/, alembic/, prisma/migrations, *.sql, db/migrate) and, if present, warn that 'git revert reverts code but NOT applied schema changes — verify the migration is backward-compatible or has a down-migration before reverting.' Domain sources (BetterBugs) stress DB rollbacks are asymmetric and higher-risk. Ours currently asserts revert is unconditionally 'safe', which can cause a reverted app to hit a forward-migrated DB. |
| P1 | M | Classify the merged diff scope (frontend / backend / docs / config) and scale verification: skip deploy-wait + visual verification for docs-only diffs, HTTP-200 smoke only for config/backend-only, full canary for any frontend change. The competitor's scope-aware decision tree avoids spending minutes waiting on a deploy and running a visual check for a README change, and avoids a pointless visual check on backend-only PRs. |
| P1 | S | Extend the smoke test beyond the root curl: also probe a health endpoint (/health then /api/health) and add a content/non-blank check (response body byte size > a small threshold) before declaring success. A 200 on '/' can still serve a blank SPA shell or an error page; the competitor and the Unleash smoke-vs-canary guidance both check a critical/health path plus non-empty content. |
| P1 | S | At each failure point (deploy failed, smoke failed, canary alert), present revert as an explicit numbered option alongside investigate/continue, rather than relying on the user to re-invoke with --revert. Both the competitor and the rollback-best-practices source make revert a first-class escape hatch at every failure gate; ours only offers it as a separate top-level mode. |
| P2 | S | Add a review-staleness check to Step 2: count commits since the approving review (git log <review-point>..HEAD) and flag CURRENT (0–3) / STALE (4+ touching code) / NOT RUN, surfaced in the readiness report. An approval that predates later commits is a real merge hazard the competitor explicitly guards against; ours reports approval state but not whether the approval still reflects the code. |
| P2 | M | Handle merge queues / required auto-merge: if 'gh pr merge --squash' is queued or rejected for queue reasons, fall back to 'gh pr merge --auto' and poll PR state until merged (bounded timeout) before proceeding to deploy detection. Repos with merge queues or required linear history will not merge synchronously; ours has no poll-until-merged path and would mis-report or stall. The competitor polls explicitly. |
★ /learn already nails capture, graduation, anti-fabrication, and concrete drafting; the real upgrade is a third path-scoped `.claude/rules/` destination plus awareness of native auto-memory and a sharper scope-routing test.
/learn is already well-aligned with field best practice and in several places exceeds it. It already has: (1) a graduation/promotion lifecycle (lessons.md -> permanent CLAUDE.md) that mirrors the self-improving-agent skill's promotion model; (2) explicit anti-fabrication ('do NOT invent a lesson just to have output'); (3) conflict detection with in-place merge rather than duplication; (4) generalize-the-principle extraction; (5) lead-with-ALWAYS/NEVER + lead-with-WHY + concrete-not-vague drafting rules with good/bad examples that match the mindstudio guidance; (6) machine-specific-path safety (portable references only) which most field examples omit; (7) append-only writes, 1-3 line cap, and a 50+-rule consolidation trigger (/audit-rules). This is a solid, opinionated implementation; the gaps below are additive, not corrective.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a third write destination: path-scoped `.claude/rules/<topic>.md` with `paths:` YAML frontmatter, for lessons that only apply to a specific file type or directory (e.g. API handlers, test files, migrations). Update Step 4 routing from a 2-way (project vs global) to a 3-way decision and show the frontmatter format. Both grounded sources (self-improving-agent skill and agentrulegen comparison) treat `.claude/rules/` with `paths:` as the canonical home for scoped lessons; it keeps the always-loaded CLAUDE.md lean by loading scoped rules only when matching files are open. Ours currently has no way to express a scoped lesson. |
| P1 | S | Make the graduation source pluggable: in Step 2/Step 4, check native Claude Code auto-memory (MEMORY.md / entries saved via `#`/'remember') as an additional source to graduate from, not only a project-root lessons.md. State that if neither auto-memory nor lessons.md exists, treat the conversation itself as the source. The self-improving-agent skill and the agentrulegen comparison both document that native auto-memory (v2.1.32+) is where corrections now land automatically; the agentrulegen piece confirms the `#`/'remember' shortcut. Hardcoding only lessons.md misses the real-world capture path on current Claude Code. |
| P1 | S | Add an explicit scope-decision test to Step 4: static personal preference/style -> global ~/.claude/CLAUDE.md; project-specific mechanics, conventions, or architecture -> project ./CLAUDE.md; applies only to a file type or path -> scoped .claude/rules/. Keep 'default global' only as the tie-breaker when none clearly applies. The cross-tool comparison and the learnings.md FAQ both distinguish static-personal vs project vs file-scoped knowledge; ours gives the instruction ('default global unless project-specific') but no operational test, so borderline lessons get mis-filed. |
| P2 | S | Optionally tag a graduating rule's provenance/firmness: when a lesson graduates after recurring 2-3x it is high-confidence/enforced; a first-time observation may be written as a softer note (or left in lessons.md/auto-memory rather than promoted yet). Borrow the high/medium/low confidence idea but keep it lightweight (one inline marker, not a schema). The mindstudio article makes a concrete case that a confidence signal lets the agent calibrate how firmly to apply a rule and prevents premature promotion of one-off hypotheses into hard ALWAYS/NEVER rules. Ours promotes everything to the same firmness. |
★ Add pnpm trustPolicy: no-downgrade (plus refresh pnpm v11 config keys to allowBuilds) — it's the one genuinely newer, real-world-validated consumer-side control ours lacks, closing the stolen-maintainer-credential downgrade-republish gap that our provenance checks don't catch at install time.
Ours is already substantially more rigorous than the closest direct competitor (latiotech's Claude Code plugin) and meets or exceeds every domain best-practice surfaced. Specifically load-bearing strengths to NOT regress: (1) per-package transitive hash verification in uv.lock (even one unhashed transitive dep = CRITICAL) — competitor only checks 'lockfile present'; (2) the corrected SHA-pinning positive-assertion regex with the documented @beta false-negative note; (3) frozen-install CI enforcement with a counter-check for non-frozen installs; (4) multi-scanner CVE coverage (pip-audit + osv-scanner + npm audit signatures) plus npm provenance verification; (5) Phase 11 release-tarball-vs-git diff (the xz-utils lesson) — competitor has no equivalent; (6) zizmor + Harden-Runner + persist-credentials + permissions block; (7) OpenSSF Scorecard integration; (8) coverage-gated 0–100 scoring that refuses 'Hardened' band when scanner coverage <90% (defeats false-sense-of-security from missing tools); (9) prompt-injection defense treating audited content as data; (10) a hard read-only safety model that NEVER mutates manifests/lockfiles so locking down repo A can't break repo B, plus the validated, opt-in --workspace blast-radius scan; (11) HIPAA mapping + paranoid tier. This is genuinely best-in-class for a deps/CI-focused audit.
Best-in-class sources found
Gaps
Recommended changes
| P1 | S | In Phase 2 (Node) and the auto-fix/baseline lists, add a check + recommendation for pnpm trustPolicy: no-downgrade (grep pnpm-workspace.yaml for trustPolicy; flag HIGH if a pnpm repo doesn't set it). State the attack: 'an attacker who steals a maintainer token republishes a version with weaker/no provenance than prior releases; no-downgrade blocks the install.' Note it's pnpm-only and opt-in (default off). This is the single newest, concrete, real-world-validated consumer-side control (pnpm v10.21+) that ours genuinely lacks; it closes the exact s1ngularity / credential-downgrade gap our provenance checks don't cover at install time. Grounded in pnpm.io/supply-chain-security and the Mondoo 2026 matrix. |
| P1 | S | Update Phase 5 and Phase 14 pnpm guidance: prefer the v11 unified allowBuilds map and treat onlyBuiltDependencies as the legacy (pnpm <11) form. Detect pnpm major version (packageManager field in package.json / corepack) and emit the correct key; if version unknown, recommend allowBuilds and note the legacy alias. Prevents ours from writing a deprecated config key on pnpm v11 repos. pnpm v11 (April 2026) replaced onlyBuiltDependencies/neverBuiltDependencies/ignoredBuiltDependencies with a single allowBuilds map. |
| P2 | S | Add pnpm blockExoticSubdeps: true and verifyDepsBeforeRun to the Node hardening checks and the auto-fixable pnpm-workspace.yaml config set (these are config, not deps, so they stay inside the read-only-on-manifests rule). blockExoticSubdeps closes a real transitive-injection vector (git/tarball URLs sneaking in via subdeps) and is a pnpm v11 default; ours doesn't mention either. Low-risk config additions consistent with our existing pnpm-workspace.yaml fixes. |
| P2 | S | Where enableHardenedMode appears in the Yarn auto-fix list, add a one-line threat statement: 'defends against lockfile poisoning — a malicious PR rewriting a lockfile entry to point at a compromised package; hardened mode re-validates lockfile content against the registry.' Ours already recommends the flag but never explains it, violating our own 'concrete attack scenario per finding' rule. Naming the threat (lockfile poisoning) makes the recommendation actionable. Grounded in the Mondoo matrix + Yarn docs. |
| P2 | S | Add a short 'Out of scope (by design)' note near the coverage/taxonomy section listing AI/ML model deserialization (pickle/torch.load) and IDE-extension supply chain as domains this audit does NOT cover, with a one-line pointer to dedicated tooling (picklescan/modelscan; VS Code publisher allowlist). Consistent with our coverage-honesty posture (the 'unknown' status + coverage% machinery): explicitly stating non-coverage prevents a false sense of completeness. The competitor covers both, so users may assume a 'supply chain audit' includes them. |
★ Ours already exceeds the field on pre-flight hygiene, lint/docs/guardrail gates, and stacked PRs — the real upgrades are conforming to the repo's own PR template, adding Breaking-Changes/Release-Note + Conventional-Commit titles, draft/CI-status awareness, and de-hardcoding the personal tone/username so it's truly shareable.
Ours is a thin command (one line) that delegates to a deep `pr-workflow-checker` agent, and that agent is already ahead of the field in several ways. Phase-0 pre-flight verification (stashes, worktrees, untracked files, [gone] branches, MEMORY.md freshness) with verify-before-clean and never-auto-delete is something none of the external sources offer. It has a lint gate (ruff/eslint/clippy/go vet with auto-fix), integrates /dc and /docs-sync (auto-running docs-sync when the diff touches code/CLI/config), checks JACKED_GUARDRAILS/DESIGN_GUARDRAILS compliance, and has clean decision logic for the four branch states (uncommitted / no-commits / commits-no-PR / commits-with-PR). It reads ALL commits not just the latest, links issues aggressively with `Fixes #`, and the sibling `git-pr-workflow-manager` adds stacked-PR detection and retargeting. Its generated description (Summary / Changes / Fixed Issues / Test Plan) already matches Graphite's Purpose/Changes/Impact/Testing recommendation.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Before generating a description, check for and read the repo's `.github/PULL_REQUEST_TEMPLATE.md` (also `docs/` and root). If one exists, fill IT OUT (preserve its sections and HTML comments) rather than emitting the agent's own structure. Only fall back to the built-in Summary/Changes/Fixed-Issues/Test-Plan template when no repo template is found. Both scraped sources establish the repo template as the source of truth; conforming to it is the single highest-signal way a reviewer recognizes a well-formed PR. Currently ours ignores it entirely. |
| P1 | S | Add a 'Breaking Changes' line and an optional 'Release Note' (one-line changelog entry, e.g. `- Added X (#NN)`) to the built-in description template, and have the agent infer breaking-ness from the diff (removed/renamed public APIs, env vars, CLI flags, entry points). The GitHub PR-template guide names both as essential: an unflagged breaking change can merge silently, and a ready release-note line makes changelog generation automatic. Ours omits both. |
| P1 | S | Tie the generated PR title to Conventional Commits — derive a type prefix (fix:/feat:/docs:/refactor:/chore:) from the commits/diff so the title reads e.g. `feat: add caching for guidance files (#31)`. Detect whether the repo already uses conventional titles (scan recent merged PR titles / commit history) and only enforce when it does. The field couples PR title to the commit convention because the title is commonly the squash-merge message; both sources reinforce this. Ours generates a free-form title with no type. |
| P2 | M | Support draft PRs and add post-create CI awareness: accept a `draft` argument (use `gh pr create --draft`), and after creating/updating, run `gh pr checks <n>` to report check status and surface failures. Optionally use `gh pr create --fill-first` as a fast seed before the agent refines the body. Standard gh-CLI consensus (Adam Johnson, gh docs): draft for WIP, `gh pr checks` to confirm CI, --fill to seed from commits. Ours creates a ready PR and never looks at checks. |
| P2 | M | Make tone and identity configurable for shareability: replace the hardcoded profane tone and the hardcoded `jackneil` username / `jack_YYYYMMDD_` branch pattern with a neutral default, reading any tone/identity/branch-convention preference from project config (CLAUDE.md or a settings key) and otherwise matching the repo's existing branch-naming convention from git history. This is a shareable suite; baking one user's personal voice, GitHub handle, and branch scheme into the agent breaks portability and can produce confusing PRs in other people's repos. |
★ Ours leads the field on pre-flight cleanup, lint gating, and decision logic, but stops at PR creation — it lacks the post-create half the field considers core: CI status gating (`gh pr checks`), review-comment/thread handling, mergeability/conflict checks, draft PRs, and PR-template/CODEOWNERS/Conventional-Commits awareness.
Genuinely strong and ahead of most community examples in several areas: (1) PHASE 0 pre-flight verification — stash/worktree/untracked/[gone]-branch triage with explicit SAFE-vs-NEEDS-ATTENTION verdicts and a never-auto-delete safety posture is more rigorous than anything found in the field. (2) Parallel state gathering in PHASE 1. (3) Clear A/B/C/D decision matrix (uncommitted / no-commits / no-PR / existing-PR). (4) PHASE 1.5 lint gating with project-type detection and auto-fix, plus /dc and /docs-sync nudges and guardrails-file compliance — this already covers the 'use linting tools' best practice. (5) Reads ALL commits, not just the latest. (6) Aggressive issue-linking with Fixes #XX auto-close. (7) Memory-file freshness reconciliation against gh pr list. The agent's structure (explicit phases + exact bash) matches the field's key lesson that explicit command sequences beat natural language.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a post-create/post-update CI-status step using `gh pr checks --json name,state,bucket,link` (and offer `--watch`/`--fail-fast` when the user wants to wait). Report a compact pass/fail/pending summary with links to failing checks, and treat exit code 8 as 'still pending'. Surface this in Case D as well so checking an existing PR always reports its current CI health. gh pr checks is the canonical CI-gating primitive (GitHub CLI manual) and CI status is the most-cited post-PR concern across sources. Without it the agent answers 'is there a PR?' but not 'is my PR actually good to merge?' — the real end of the workflow. |
| P1 | M | Extend Case D to triage open review comments: `gh pr view --json reviewDecision,reviews` plus `gh api repos/{owner}/{repo}/pulls/{n}/comments` to list unresolved line-level threads, summarize them, and offer to address+reply (reply with commit hash, optionally resolve the thread via the GraphQL resolveReviewThread mutation, gated behind explicit user confirmation). The most concrete real-world Claude Code PR workflow found (Naka) is built entirely around this loop, and reviewDecision/unresolved threads are what actually block a merge. Ours stops at 'update the description,' missing the dominant reason an existing PR needs attention. |
| P1 | S | Before creating a PR, verify mergeability and branch freshness: confirm the branch is pushed and ahead-of-remote is 0 (push if needed, with confirmation), and after creation check `gh pr view --json mergeable,mergeStateStatus`. Warn on CONFLICTING / BEHIND and suggest rebasing on base, rather than silently producing an un-mergeable PR. Protected-branch + conflict realities are standard PR best practice (Crystallize), and creating a PR from an unpushed or conflicting branch is a routine gh failure mode the agent currently walks straight into. |
| P2 | S | Add draft-PR support: a `--draft` path plus a heuristic to suggest it (WIP-style commit messages, failing local tests/lint, or user signalling 'early feedback'). Pass `gh pr create --draft` when appropriate. Draft PRs for WIP / early feedback are an explicit best-practice recommendation (Crystallize) the agent has no concept of; always creating ready-for-review PRs is wrong for in-progress work. |
| P2 | M | In PHASE 4, detect and honor repo conventions: read `.github/pull_request_template.md` (and `PULL_REQUEST_TEMPLATE/`) to seed the body, parse `CODEOWNERS` to propose `--reviewer`, and detect an existing Conventional-Commits style from recent history to format the PR title accordingly. Keep the existing tone overlay on top of the template's required sections. Respecting PR templates, CODEOWNERS, and Conventional Commits is cross-source team consensus (Crystallize); a best-in-class PR tool should adapt to the repo's existing conventions instead of imposing only its own freeform template. |
★ Make the QA pass actually USE the network and DevTools signals the skill already detects -- add an explicit network-error check (P0) and an opt-in Lighthouse/perf-trace step on the changed page -- turning a visual+console pass into a full driving-AND-debugging QA that matches best-in-class Chrome DevTools MCP workflows.
Our qa is already ahead of most of the field and must not regress on these: (1) Multi-tool detection with a graceful fallback chain (Chrome DevTools MCP -> Playwright MCP -> Claude-in-Chrome -> agent-browser) plus exact remediation copy when a tool is down -- more robust than every external example, which assumes one tool. (2) Accessibility-tree-first detection (snapshot preferred over screenshot) -- matches the token-efficiency consensus Kinney documents. (3) git-diff-driven scoping that targets only changed UI areas, with an explicit cross-page-impact spot-check for shared infra (global CSS, router, store, layouts) -- nobody external does this; most just test a URL blindly. (4) Credential auto-discovery from .env with infra-credential exclusion and a security check. (5) Repo Config override + jacked-setup pre-configuration for fast repeat runs. (6) Accessibility lens integration (contrast, keyboard nav, semantic HTML, labels, focus management). (7) Disciplined read-only detect-then-plan posture (issues -> writing-plans -> /dcr) with severity-ranked structured report. (8) Screenshot lifecycle management (setup + cleanup). (9) A clear /qa vs /ux split by blast radius.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | In Step 6, add an explicit 'Network Errors' check alongside Console Errors: after exercising each UI area, list network requests and flag any non-2xx/3xx responses, failed fetches, CORS errors, or pending/hung requests tied to the changed code. Name the per-tool calls (Chrome DevTools MCP list_network_requests/get_network_request; Playwright browser_network_requests; agent-browser equivalent). For a 4xx/5xx, note the URL, status, and whether it correlates with a visible UI defect (empty state, spinner that never resolves, error toast). Failed API calls are a leading cause of 'UI looks broken' bugs and are invisible to visual+console testing. The skill already lists the network tool in Step 1 but never tells the QA pass to use it -- closing a gap with zero new dependencies. |
| P1 | M | Add a 'Performance & Lighthouse (Chrome DevTools MCP only, opt-in)' subsection to Step 6: when Chrome DevTools MCP is the active tool, run lighthouse_audit (mode=navigation, device per viewport) on the primary changed page for a11y/SEO/best-practices, and optionally a performance_start_trace -> stop -> analyze for LCP/INP/CLS. Direct the heavy report to disk (outputDirPath / a tmp path), never inline the full JSON. Explicitly note Lighthouse EXCLUDES performance -- use the trace for Core Web Vitals. Gate it: only when the change plausibly affects perf (images, fonts, large bundles, render-blocking, new heavy components) or a11y, so quick component checks stay fast. Confirmed by Kinney, Continue.dev, and the shipped Cloudflare web-perf skill as the standard, high-signal capability of the tool we already call 'preferred'. Writing to disk and gating keeps it token-cheap and avoids slowing the common case. |
| P1 | S | Add a short 'State Hygiene' note to Step 6: start each area from a known state (fresh navigation or an isolated/incognito context where the tool supports it -- e.g. Chrome DevTools MCP new_page isolatedContext), avoid mutating real/shared records when a throwaway path exists, and after the pass restore state (log out / clear the form / close created entities). Add an explicit warning that with Chrome DevTools MCP --autoConnect the agent is driving the user's LIVE signed-in browser session -- do not leave it mid-flow or on a destructive screen. Prevents false-positive bugs from cross-test contamination and prevents the QA run from corrupting the user's real browser session -- a concrete failure mode the field calls out (isolatedContext, storage-state) that ours is silent on. |
| P2 | M | Add an OPTIONAL Step (after Report Findings) offering to convert a verified PASS journey or a reproduced bug into a runnable regression test: replay the exact steps via the browser tool, then emit a @playwright/test (or the repo's existing e2e framework) spec covering that journey, run it, and iterate until green. Keep it strictly opt-in and behind the read-only posture (the skill itself still only detects/reports; test generation is an explicit user-approved follow-on), and write the test into the repo's existing tests dir. The dominant Claude-Code QA pattern (nikiforovall) turns a one-shot manual finding into durable protection so the bug can't silently regress -- the one structural capability ours lacks. Gating it opt-in preserves the clean detect-then-plan boundary. |
| P2 | S | In the Console Errors check, add the Chrome DevTools MCP calls (list_console_messages / get_console_message) to the list that currently names only Playwright and Claude-in-Chrome, so the check fires on every tool including the preferred one. Removes an inconsistency that could cause the agent to skip console checking precisely when it's on the tool we recommend first. |
★ When the session is Playwright-driven, also emit a Playwright trace.zip (per-action DOM + console + network, shareable interactively at trace.playwright.dev) — the field's gold-standard 'evidence in motion' that ours currently reconstructs by hand instead of capturing natively.
Ours is already ahead of the public field and should not regress these: (1) Step 0 deterministic path selection (Path P Playwright native video+chapters vs Path F frame-stitch) probed UP FRONT, naming 'no way to produce a video discovered mid-session' as the #1 failure mode; (2) the load-bearing 'inject 0.8-1.5s wait beats between actions or the video is an unreadable blur' trick — MCP actions are near-instant; (3) never-fake-login (record up to the auth wall, narrate the rest is blocked) and announce-env-var-names-not-values; (4) Step 6 artifact verification via ffprobe/identify (duration>0, frame count>1, non-trivial size) before claiming success — no other source does this; (5) explicit avoidance of macOS screencapture -v due to the un-clickable TCC prompt; (6) a video-synced narrated doc that already pairs each leg with expected-vs-observed AND console/network state at that moment; (7) shareability awareness (mp4/gif drag-drop into GitHub/Slack, webm->mp4 convert). The generic Claude Code QA posts in the wild do none of 1-6.
Best-in-class sources found
Gaps
Recommended changes
| P1 | M | In Step 0, add a third preferred path 'Path T — Playwright trace' available whenever the driver is Playwright MCP (i.e. alongside/instead of Path P): start tracing with screenshots+snapshots before navigation, stop to a trace.zip, and treat it as the richest evidence artifact. State the priority as: when on Playwright, emit BOTH the native video (human-watchable) AND the trace.zip (interactive, per-action console/network/DOM) — the trace is the QA-grade evidence, the video is the shareable clip. Cite that the trace bundles the exact console/network-at-that-moment signal the doc already wants. This is the one genuinely better technique the field has that ours lacks. Verified at playwright.dev/docs/trace-viewer: trace.zip gives per-action Before/Action/After DOM snapshots plus console+network logs that filter per action and an Errors timeline — strictly richer than a flat video for regression/bug evidence, and nearly free when already on Playwright MCP. |
| P2 | S | In Step 5/Step 7, add trace sharing: tell the reviewer they can open the trace.zip at https://trace.playwright.dev (drag-drop, or append ?trace=<accessible-url>), note it loads entirely in the browser and transmits no data externally, and link it from the narrated doc header alongside the video path. Verified on the official docs page. Gives a privacy-safe, zero-install, interactive way to hand off evidence — a concrete upgrade to the current 'here is an mp4' delivery, and reassuring for sensitive apps since nothing is uploaded. |
| P2 | S | In Step 3, add an explicit ordering note: start the trace/video recording BEFORE the first navigation (for the trace/context API in particular), or the opening state of the journey is lost. Steve Kinney calls this the classic failure ('beginning a screen recording after the bug already happened'). Cheap one-liner that prevents a silently truncated artifact. |
| P2 | S | In Step 7 (Deliver), add an optional offer: 'If the journey passed and is worth keeping green, I can emit a @playwright/test spec that reproduces it' — the manual-exploration -> automated-test hand-off. This is the one transferable idea from the mainstream Claude Code QA pattern (Nikiforov). Turns one-off 'it worked' evidence into a durable regression test, which is the stated purpose ('regression evidence') taken one step further. Keep it optional so the command stays focused on evidence, not test-authoring. |
★ Make the digest outcome-aware — record each tool action's result (exit code / error + a one-line failing tail) and lead the digest with what actually broke, so the recovered session re-anchors on the failure, not just the last command typed.
Ours is already strong and well-matched to the field for its narrow job. Don't regress: (1) cwd resolution never trusts the lossy projects-dir slug — it verifies by reading the recorded `cwd` field inside transcripts and enumerates dirs as fallback; (2) crash-truncated final JSONL line is tolerated (skipped) and flagged as `truncated`; (3) incomplete-last-turn is detected structurally via an unanswered final tool_use id (open_ids), not heuristics; (4) digest captures TodoWrite state, files touched (including file-history-snapshot backups), plan excerpt, and sub-agent summaries; (5) the renderer is char-budgeted and NAMES every section it clips/drops rather than silently truncating; (6) verbatim `/goal` and `/loop` kickoffs are surfaced as an un-trimmable 'Manual restart required' block because native `--resume` cannot restart them — a genuinely subtle, correct insight; (7) confirm-before-inject UX, live-session exclusion via $CLAUDE_CODE_SESSION_ID, /recover self-leak guard, and explicit outdated-CLI ('No such command recover') remediation. Qdrant-free by design so it works on a bare install the moment after a crash.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | In _extract_actions / _action_label, capture the matching tool_result for each tool_use (we already track open_ids and walk user tool_result blocks) and fold its outcome into the action label: for Bash, append the exit/error signal and, on failure, a one-line tail of stderr/stdout (e.g. 'Bash: pytest -q → FAILED: 3 errors'); for Edit/Write, note errors. Read the result block's is_error flag (and/or the toolUseResult content). Then add a digest section that lists the last few FAILED actions first. This is the highest-leverage upgrade: it tells the recovered session what actually broke. cli-continues' Tool Activity section proves outcome-aware handoffs are the norm; after a crash the failing last action is the most important fact to re-anchor on, and our digest currently can't see it. |
| P1 | S | In build_digest, also capture the last assistant THINKING block (content block type 'thinking', falling back to text if absent) and render it as a short 'In-flight intent' section right after the incomplete-last-turn warning, hard-capped to ~400 chars. The agent's final reasoning explains WHY the next step was next — exactly the context a fresh session lacks. cli-continues carries this as a 💭 note; it is small and high-signal. |
| P2 | S | Add a lightweight verbosity lever to `jacked recover --digest`: a --depth {brief|standard|full} (or --budget N) flag that scales DEFAULT_BUDGET_CHARS plus the _RECENT_USER_ASKS / _MAX_TOOL_ACTIONS / _MAX_FILES caps, defaulting to current behaviour as 'standard'. Document it in SKILL.md step 5 ('use --depth full for heavy multi-file sessions'). cli-continues ships minimal/standard/verbose/full presets; one fixed 12k budget under- or over-serves depending on session size, and the skill currently offers no control. |
| P2 | S | Add one line to SKILL.md Requirements noting the on-disk source of truth so the user can self-verify if `jacked recover` finds nothing: transcripts live at ~/.claude/projects/<encoded-cwd>/<session-id>.jsonl (cwd encoded by replacing '/', '\' and '.' with '-'); a near-empty candidate list usually means wrong folder or a non-default CLAUDE_CONFIG_DIR/CLAUDE_PROJECTS_DIR, not data loss. CC issue #59736 and the claude-code-session-recovery toolkit show the most common 'lost session' is an index/path mismatch, not deletion — naming the real path turns a dead end into a one-step fix and reinforces 'do not invent state'. |
★ OURS already exceeds the entire Claude Code ecosystem on the redo primitive; the only real upgrades come from the classic software-rewrite canon — harvest the old code's hard-won edge cases before scrapping, capture/verify behavior so the redo doesn't silently regress, and add explicit scope-discipline and 'is a redo even right?' gates.
OURS already implements and substantially exceeds the canonical community technique ('The Elegant Redo' one-liner). Genuinely strong: (1) Non-negotiable work preservation via git stash with explicit failure-stop if the stash command errors — most community advice just says 'scrap it'. (2) Branch isolation so the old approach stays accessible and comparable (git diff main..HEAD). (3) A mandatory, well-structured 4-question reflection (original goal / what went wrong / what we know now / what the new approach must account for) that directly captures hindsight — this maps cleanly onto Spolsky's 'developers underestimate the cost of reproducing behavior'. (4) Replan-before-code (plan mode) gating implementation on an approved design. (5) Explicit anti-over-engineering mindset ('simplest thing that works'). (6) Hard safety rails and a refusal to skip reflection even when the user says 'just redo it'. (7) A prerequisite gate that refuses to redo when there's nothing implemented yet. This is already best-in-class relative to anything in the Claude Code ecosystem.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a 'Salvage / harvest the old code' sub-step to the reflection (between Step 3 and Step 4): before scrapping, explicitly inventory what the OLD implementation got RIGHT — the edge cases it handled, bug fixes baked into odd-looking conditionals, validations, and non-obvious behaviors — and fold these into the Step-3 question-4 'what the new approach must account for' list. Frame it with the Spolsky insight: those ugly lines are usually hard-won solutions; reproducing the behavior is the expensive part. This is the #1 consensus rewrite failure mode (Potapov, Caudill, Spolsky/Feathers): clean-slate redos discard institutional knowledge and reintroduce already-solved bugs. OURS reflects on failure but never harvests success, so it is exposed to exactly this. Grounded directly in the scraped sources. |
| P0 | M | Add a lightweight behavior-capture safety net: before (or immediately after starting) the redo, note or pin the key behaviors/tests the old approach satisfied (run the existing test suite if present and record what passes; otherwise jot a short characterization checklist of expected inputs/outputs). At wrap-up, verify the redo reproduces those behaviors before recommending the user merge it. Feathers' characterization-test pattern, surfaced in Potapov, is the canonical safety net for replacing code without silent regression — and Potapov's AI caveat (76% of LLM refactor suggestions hallucinate without validation) makes a verification gate essential for an agent-driven redo. Turns 'is the redo better?' from a vibe into evidence. |
| P1 | S | Add a scope-discipline guardrail to Step 4 (Redesign): state explicitly that the redo targets the SAME goal at feature parity and must NOT absorb a deferred wish-list — call out the Second-System Effect by name ('while we're at it' is how redos balloon and stall). New scope goes on a separate list for after the redo lands. Brooks' Second-System Effect (via Potapov) is the specific mechanism that turns a redo into a stalled rewrite. OURS' generic 'simplest thing that works' doesn't name the parity-floor-plus-wishlist trap, which is the part agents and users actually fall into. |
| P1 | S | Add a brief 'is a full redo the right move?' gate in the prerequisite/early phase: a quick check of whether the failure is localized to one module (favor a targeted fix or redo only that slice) versus pervasive across the approach (favor full scrap). If localized, offer the narrower option before scrapping everything. The rewrite literature's clearest refactor-over-rewrite signal is localized problems; defaulting to scrap-everything when a targeted fix would do is precisely the mistake the canon warns against. A cheap gate makes OURS recommend the right scope of redo, not just always the biggest one. |
★ Ours already nails the hard, modern parts (OIDC Trusted Publishers, correct hatch version source, CI/dirty-tree/duplicate-version gates, SBOM) but ships a release before any local build/test gate and creates the immutable GitHub Release BEFORE the PyPI upload — so the top fixes are: build+twine-check-before-tag, run tests locally since publish.yml never does, and verify the artifact is actually installable post-publish.
Ours is already aligned with field best-practice on the hard parts. (1) PyPI Trusted Publishers via OIDC — no long-lived tokens — which the PyPA guide and Simon Willison's TIL both treat as the gold standard. (2) Correct version source of truth: ours edits jacked/__init__.py, which IS the hatch version path (`[tool.hatch.version] path = "jacked/__init__.py"`), with pyproject `dynamic = ["version"]` — so the bump is internally consistent. (3) Strong gating: refuses to release on a dirty tree, refuses to release with failing CI, refuses duplicate versions (PyPI rejects them), never force-pushes without approval, and monitors the publish workflow to completion before declaring done. (4) Clear pre-flight (branch check, gh auth check) and explicit failure handling with `--log-failed`. (5) Branch assumption ('master') and the publish.yml trigger (`on: release: published`) match this repo's actual config. The supply-chain hardening (pinned actions, harden-runner, SBOM, OIDC) already exceeds most community release commands.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Reorder so the tag/release cannot outlive a failed publish: run a local build + (test) gate and a `twine check dist/*` BEFORE `git tag`/`git push --tags`/`gh release create`. Concretely add a step between BUMP and PUSH: `python -m build && twine check dist/*` (and run the test suite if one exists). Only tag after it passes. Pradyun's PyPA design doc: 'The git tag and GitHub release must not be published if the release cannot be uploaded to PyPI.' Ours creates the immutable, `--latest` GitHub Release first (it's the publish trigger), so a build/upload failure strands a public release pointing at a version absent from PyPI. Building locally first catches the most common failure before anything irreversible happens. |
| P0 | S | Make step 6 honest about this repo: state explicitly that publish.yml does NOT run tests/lint (it only builds+publishes on release), so the release command MUST run the project's own tests/build locally before tagging rather than relying on 'CI will run as part of the release.' Verified: the repo's only workflow is publish.yml triggered `on: release: published`; there is no push/PR CI. Ours currently says to proceed when no push CI exists, which means versions ship with zero automated validation. The field universally gates a release on a green test run. |
| P1 | S | Add a post-publish install-verification step: after the publish workflow succeeds, confirm the version is resolvable from the index (e.g. poll `https://pypi.org/pypi/claude-jacked/json` for the new version, or do a clean `uv tool install claude-jacked@X.Y.Z` in a throwaway env) before printing 'Release complete'. PyPA/Trusted-Publishers guides emphasize verifying the artifact is actually live and installable; ours declares success on workflow exit code alone and prints an install command it never validates. PyPI CDN/propagation or a metadata issue can make 'workflow green' but 'not yet installable'. |
| P1 | M | Suggest the semver bump from Conventional Commits since the last tag: before asking, compute `git log <last-tag>..HEAD` and surface whether commits contain feat:/fix:/BREAKING CHANGE, and recommend the bump (still let the user override). python-semantic-release, standard-version, and the SEI guide all derive the bump from commit conventions; ours asks blind. Even a non-binding suggestion reduces mis-bumps and grounds the version choice in actual changes. |
| P2 | M | Add a `.github/release.yml` categorization note + optional CHANGELOG: have the command pass categorized notes (group PRs by label) and, if a CHANGELOG.md exists, prepend the new section before tagging so the released tree contains its own changelog. GitHub Docs document `.github/release.yml` to organize `--generate-notes` output by label; Pradyun and standard-version treat a per-release changelog (delta since last tag, committed before the tag) as part of cutting a release. Repo currently has no CHANGELOG and bare generate-notes. |
| P2 | S | Derive the project name and URLs instead of hardcoding 'claude-jacked' in the success message; note the pyproject-name vs publish-env (`pypi.org/p/jacked`) discrepancy as a thing to confirm rather than assume. pyproject `name = claude-jacked` while publish.yml's environment url says `pypi.org/p/jacked`; a release command that hardcodes the project/PyPI URL can print a wrong link. Reading the name from pyproject avoids drift and surfaces the real naming question. |
★ Ours is a strong session-and-velocity retro but stops at descriptive vanity metrics; adding DORA delivery metrics, churn×complexity hotspots, bus-factor/ownership, and an anti-gaming guardrail would make it best-in-class for git-driven engineering retrospectives.
Ours is already solid and arguably ahead of generic git-stats tools on a few axes: (1) explicit compare-mode that computes period-over-period deltas for velocity/LOC/test-ratio/fix-ratio; (2) coding-session detection via <2h commit-gap clustering with avg/longest session and peak-hours-of-day — git-quick-stats only does raw hour histograms; (3) per-contributor narrative with both a specific praise and a specific growth opportunity (humane, not a leaderboard); (4) test-health signal (test-file ratio, high-churn-no-test-change flag) and fix-ratio as a quality proxy; (5) strictly read-only with cross-platform date handling and a gh-CLI-optional PR section. Structure (gather → sessions → compare → report) is clean and the report template is concrete.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a 'Delivery Metrics (DORA)' section that computes the four keys from git+gh: change lead time = median(mergedAt − first commit time) per PR; merge/deploy frequency = merged PRs (or tags) per period; change failure rate = (reverts + hotfix/`fix!`/`revert` commits or PRs labeled incident) ÷ total merges; recovery time = median time from a revert/hotfix back to green. Show each with the compare-mode delta. Cite dora.dev definitions in the Notes. DORA's five keys are the canonical engineering-retrospective metrics and predict org performance; all are derivable from the git+gh data ours already gathers. This is the single biggest substance gap versus the field. |
| P0 | M | Upgrade the hotspots step from raw change-count to churn × size (proxy for complexity): rank files by (revisions in window) × (lines changed or current LOC) and only flag the high-churn AND high-size quadrant as refactor priorities; explicitly de-prioritize high-churn/low-complexity files (config, lockfiles). Add the paste-ready 12-month churn one-liner and exclude vendor/generated paths. understandlegacycode's hotspots technique is the consensus way to avoid false-positive 'risks' — a high-churn config file is not a hotspot. Ours currently can't distinguish those. |
| P1 | M | Add a 'Knowledge & Bus Factor' subsection: per important file, count distinct authors and the main-dev's ownership %; flag single-author files in hotspots (bus-factor risk) and files with 4+ authors (defect-predictor / coordination risk). Use `git log --format=%an -- <file>` aggregation. Code Maat establishes author-count-per-module and main-dev ownership as defect predictors and the standard way to surface knowledge silos — the inverse risk ours misses entirely. |
| P1 | S | Add a short anti-gaming guardrail to the report header and Notes: state that commit/LOC counts and the per-contributor section are heuristics for conversation, NOT performance evaluation or a leaderboard, and warn against Goodhart-style targets. Frame per-contributor output as team-health observations, not rankings. Both DORA ('common pitfalls') and Code Maat explicitly warn these metrics are gameable and harmful when used to rank/compare people. A retro that ranks by LOC without this caveat actively invites the documented failure mode. |
| P2 | S | Add an optional 'Suggested reviewers' output: for the files changed this period (or for a given branch/PR), list the top historical authors of those files as recommended reviewers, plus a CODEOWNERS cross-check if present. git-quick-stats ships this as a headline feature; it turns the retro from descriptive into actionable for the next change and reuses ownership data already computed for bus-factor. |
| P2 | S | Tighten data hygiene defaults: exclude merge commits (`--no-merges`) consistently, filter known bot authors (e.g. dependabot, *[bot]) and vendored/generated paths via pathspec `:(exclude)`, and support a `.mailmap`-style author normalization note so one human with two emails isn't double-counted. Code Maat and git-quick-stats both treat merge/bot/vendor exclusion and author de-duplication as prerequisites for trustworthy numbers; ours applies --no-merges unevenly and never de-dupes authors, which skews per-contributor and churn output. |
★ Already nails the field's core (verify-before-resume, never-redo-succeeded-work, resume-don't-restart); add a side-effect-duplication safety check, differentiate error classes per official Claude Code recovery (529→/model, context→/compact, usage-limit≠retryable), and swap blind retry for backoff-aware wording.
Already matches the field's central principle: verify what actually landed before resuming, and never redo work that already succeeded (the buildmvpfast 'crash-only / skip already-executed' and the Medium 'know which steps committed' patterns). Correctly frames the error as not-your-fault to stop the model from second-guessing a sound plan. Covers the three real resume scenarios cleanly: (1) work finished, just waiting → confirm, don't redo; (2) inline edit/command/tool interrupted → verify what applied, resume from the precise fall-off point; (3) dead subagents/workflow → resume a workflow via resumeFromRunId and re-dispatch ONLY the agents that failed (this resume-don't-restart granularity is exactly best practice). Explicitly forbids changing the goal/plan or starting new work — a meaningful guardrail. Tight, low-token, single-purpose.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a one-line side-effect safety check before resuming: 'Before re-running any step that has an external side effect (git commit/push, opening or commenting on a PR, sending a message, writing to an API or DB, a destructive command), first confirm it did NOT already land — duplicated side effects are the real damage, not lost compute. For pure reads/edits in the working dir, just redo if unsure.' Directly from the idempotency and saga sources: the costly retry failure mode is duplicated irreversible actions. Ours only says 'don't repeat steps' generically; making the side-effecting step the thing to verify hardest is the single highest-value add and is concrete/grounded. |
| P1 | S | Differentiate the error class and act accordingly: if it's a 529/overloaded/capacity error, note you can switch model with /model to keep working; if it's a context/compaction error, /compact; if it's an account usage/session/weekly limit, recognize it's NOT a transient blip and immediate retry won't clear it (wait for reset / switch plan-model). Keep it to ~3 short bullets. The official Claude Code error reference prescribes materially different recovery per class. Ours treats all as identical 'cloud blips,' which gives wrong advice for usage limits (retrying does nothing) and misses the per-model capacity workaround. |
| P1 | S | Replace 'keep retrying until it goes through' with backoff-aware wording: 'wait a few seconds and back off between attempts (Claude Code already auto-retried with exponential backoff before surfacing this), rather than re-firing instantly; for capacity errors prefer switching model over hammering the same one.' Official docs state CC already exhausted ~10 backed-off retries before the error appeared; blind immediate retry is counterproductive and can worsen 529/429. Grounded correction of an existing line. |
| P2 | S | Add a short reconciliation nudge in the 'verify' step: 'Trust the on-disk / tool-result reality over your own last sentence — a task that was mid-multi-step can look finished but be in a partial state; reconcile what you claimed was done against what actually committed before declaring resume complete.' Straight from the 'stopped halfway / failures that look like successes' source. Sharpens ours from 'verify what landed' to 'actively reconcile claimed-vs-actual,' catching the partial-completion trap. |
★ Give each Phase-1 agent a non-overlapping scope plus an explicit effort budget ('start wide, then narrow, ~N tool calls') — Anthropic's two hardest-won lessons — so divergence becomes broad coverage instead of redundant re-investigation, the single biggest quality and token-efficiency upgrade.
Ours is already strong and structurally ahead of most community swarm/brainstorm commands. Specifically, do not regress: (1) Complexity calibration table that auto-scales agent count 2-5 by signal — matches Anthropic's 'scale effort to query complexity' principle. (2) Three-axis differentiation (persona x constraint x method) with uniqueness enforcement — this is the diversity driver the debate literature says matters most, and it's more principled than the 'N clones with different temperatures' approach common in the field. (3) Explicit two-phase design: divergent research → synthesis (convergence/divergence/unique-insight analysis) → verify + devil's-advocate → bounded iterate loop. The dedicated Devil's Advocate with a 'find NEW risks, don't restate' rule and a 'if you can't break it, that's a strong signal' framing is genuinely best-in-class. (4) Pre-spawn CODEBASE_CONTEXT discovery condensed (not raw files) into agent prompts — avoids prompt bloat. (5) Hard rules: all-parallel spawning, READ-ONLY agents, no auto-transition to implementation, 3-round safety cap, 'no convergence → stop and ask user.' (6) Persists output as an HTML artifact (with Mermaid for convergence maps) — matches the cross-tool consensus that plans should be visible, inspectable artifacts. (7) Synthesis decision logic (clear winner / combination / no convergence) is explicit and refuses to force a false winner.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a per-agent effort budget + 'start wide, then narrow' line to the Phase-1 agent prompt template. Concretely, in the INSTRUCTIONS block add: 'Effort budget: aim for ~[5-15] tool calls scaled to complexity — start with broad exploration, then narrow to specifics. Stop when you have enough to write a complete brief; do not over-search.' Wire the number to the existing complexity calibration tier. Anthropic's #1 and #6 prompting lessons ('scale effort to query complexity', 'start wide then narrow'). Their top early failure modes were agents over-searching and issuing overly-narrow queries that returned nothing. Ours scales agent COUNT but gives each agent zero effort guidance. |
| P0 | M | In DIFFERENTIATION ASSIGNMENT, additionally assign each agent a non-overlapping SCOPE/territory of the problem (e.g. 'data model', 'API surface', 'migration path', 'failure handling') in addition to persona/constraint/method, and add a prompt line: 'Your assigned scope is X — go deep there; assume peers cover the rest.' Anthropic's 'teach the orchestrator how to delegate': different framing alone caused subagents to duplicate the SAME investigation. Distinct angle + distinct territory is what actually prevents redundant work and maximizes coverage per token. |
| P1 | S | Add a CLARIFY GATE between CONTEXT DETECTION and COMPLEXITY CALIBRATION: if the problem is ambiguous or under-scoped (unclear constraints, success criteria, or which approaches are in/out of bounds), ask 2-3 targeted clarifying questions before spawning. Cap at one round; proceed on answer. Windsurf megaplan and Devin both reduce uncertainty with clarifying questions before committing to a plan. Spawning 4-5 agents (~15x token cost) against a misunderstood problem is the most expensive failure this skill can make. |
| P1 | S | Add an economic-viability / right-sizing check to COMPLEXITY CALIBRATION: if the problem is genuinely trivial (single obvious approach, no real trade-offs), say so and recommend a single planning pass instead of a swarm — i.e. allow N=1 / 'swarm not warranted' as a valid calibration outcome. Anthropic: multi-agent uses ~15x the tokens of a single chat and is only worth it for high-value, parallelizable problems; 'most coding tasks involve fewer truly parallelizable tasks than research.' A best-in-class version knows when NOT to swarm. |
| P2 | M | Add an optional CROSS-POLLINATION sub-step inside SYNTHESIS (or as a lightweight Phase 1.5) for the 'no convergence' / high-divergence case: surface each agent's brief to a single reconciliation pass (or one short rebuttal round) where the strongest disagreements are debated before the orchestrator decides — rather than the orchestrator silently picking. Du et al. and the MAD literature show debate/cross-examination beats independent parallel generation AND single-agent reflection; 'Debate or Vote' shows the gain is real but bounded, so keep it to one round (consistent with ours' existing cap philosophy). Ours currently captures divergence but never lets agents refine against each other. |
| P2 | S | Add an anti-fabrication rule to the Phase-1 agent RULES block: 'Do not invent APIs, benchmarks, or library behavior. If you assert an external fact, you must have verified it via WebSearch/file read or explicitly label it ASSUMPTION.' Verify+devil's-advocate only pressure-tests the synthesized plan; a fabricated fact in a Phase-1 brief can bias synthesis upstream of any check. Anthropic's human-eval specifically caught hallucinated answers and source-quality bias as recurring multi-agent failure modes. |
★ Already best-in-class on detection discipline (linters-first, file:line, lens integration, honesty) — the one real gap vs the field is churn-based hotspot prioritization, so add a git-churn step that ranks complex-AND-frequently-changed files to the top and tells users to leave stable-but-ugly code alone.
Already best-in-class on the fundamentals and ahead of every community equivalent found: (1) Real-linters-first (ruff/eslint) with explicit 'if not configured, skip — don't install', which the davila7 agent lacks. (2) Hard file:line discipline on every finding. (3) Three-bucket categorization (Bugs/Risk, Maintenance, Cleanup). (4) Specialist-lens anti-pattern integration — a differentiator no external source has. (5) Output cap (~20 findings) with 'actionable over exhaustive' and a suggested-next deeper scan. (6) Explicit epistemic humility: 'Don't pretend to be a static analyzer', 'say possible not definite'. This honesty already beats tools that over-claim.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a 'Step 3.5: Hotspot Prioritization (git churn)' that runs the churn one-liner in a git repo: git log --format=format: --name-only --since=12.month | egrep -v '^$' | sort | uniq -c | sort -nr | head -50. Cross-reference the high-churn files against the oversized-files and lint-density findings. Any file that is BOTH high-churn AND large/complex is a hotspot — surface those at the top of the Maintenance bucket and label them 'Hotspot (changed N times / 12mo, X lines)'. Explicitly note that large-but-low-churn files are NOT priorities (quadrant 2: 'if the code never changes it isn't costing us money'). This is the single most-cited, most consensus technique across both authoritative sources (CodeScene/Tornhill and understandlegacycode/Carlo) and it's the one thing our command structurally lacks. It's a tool-free git one-liner that fits perfectly with our 'real tools first, you're not a static analyzer' stance, and it directly fixes the 'when everything is urgent, nothing is' failure mode of pure static scans. |
| P1 | S | Add a lightweight per-finding signal in the report. For each finding, append a churn/impact tag, and for Maintenance/refactor items rank by churn rather than listing alphabetically. Borrow davila7's compact scale: tag refactor candidates with Impact and Effort (e.g. 'Impact: high · Effort: M') so the user can triage within a bucket. Grounded in the davila7 Claude Code agent's Ease/Impact/Risk (1-5) scoring. Turns a flat 20-item list into a triage-ready list. Keep it light (a tag, not a verbose table) to preserve our 'actionable over exhaustive' principle. |
| P2 | S | Add a 'trend' note to the hotspot output: compare churn in a recent window (e.g. --since=3.month) against the 12-month window and flag files whose change rate is ACCELERATING as 'degrading / actively getting worse'. CodeScene's central finding is that the direction a hotspot is evolving matters more than its absolute size — 'the warning signs of a growing hotspot were all there' years before it became a problem. Git already has this data; surfacing acceleration catches debt before it's entrenched. |
| P2 | M | Add a de-dup pass: before reporting, grep the repo for an existing debt ledger (e.g. an issues export, TECHDEBT.md, or `gh issue list` if gh is available) and skip / mark findings already tracked or explicitly accepted. The davila7 agent searches existing issues before filing. Because /techdebt is explicitly meant to be re-run 'periodically during long sessions', without de-dup it re-surfaces the same accepted debt every run, training users to ignore the output. |
★ Strong, Python-deep pytest agent that already nails the pyramid, determinism, isolation, and anti-patterns — but it gates on coverage % alone and trusts its own output; add a mutation-score effectiveness gate, ban weak/tautological assertions, and make it run-and-verify its tests before declaring done.
Genuinely strong and Python-deep. Already covers, at or above field level: the full test pyramid (unit>integration>e2e) plus property-based (Hypothesis) and performance (pytest-benchmark); determinism done concretely (freezegun, seeded RNG, no flaky tests); isolation (no shared state, fixture cleanup); a speed budget (<10s unit suite); parametrization and fixture/mock-with-spec patterns with real code; a coverage workflow (pytest-cov, branch/edge identification, HTML reports, sensible exclusions); and an explicit anti-pattern catalog (testing implementation not behavior, over-mocking, order-dependent/brittle tests, hitting production resources, missing error coverage) plus a test-maintenance and quality-gate section. The behavior-over-implementation stance and 'mocks properly spec'd' line already pre-empt two of the field's named AI-test failure modes.
Best-in-class sources found
Gaps
Recommended changes
| P0 | M | Add a 'Mutation Testing & Effectiveness Gate' section: after coverage passes, run a mutation tool (mutmut/cosmic-ray for Python; note Stryker for JS/other) on critical modules and treat surviving mutants as the real gap signal. State plainly that line coverage proves code is exercised, not that tests would catch a bug, and that a covered-but-low-mutation-score module needs stronger assertions, not more lines. Biggest substantive gap. Codecov's piece makes this the central thesis and gives the canonical add()/toBeDefined() example; ours relies solely on coverage % as the quality bar, which the field explicitly rejects as a correctness measure. |
| P0 | S | Add 'weak/tautological assertions' to the anti-pattern list with a one-line rule: every test must assert specific expected values/effects (exact value, raised exception type+message, state change), never just is-not-None / toBeTruthy / does-not-throw. qaskills.sh names this the #1 AI-generated-test flaw and Codecov demonstrates it; it is cheap to add and directly hardens the agent's own output against the most common failure mode for LLM-written tests. |
| P1 | S | Reframe coverage thresholds as a floor + diagnostic rather than a target: keep the 90% critical-path number but add that coverage identifies what is untested (use it to find gaps), pair it with branch coverage and mutation score, and never write tests purely to move the number. Cross-source consensus (Codecov, Fowler, r/ExperiencedDevs) that chasing a coverage number is a known failure mode; ours currently presents the numbers as the gate with no caveat, which can drive the agent to game them. |
| P1 | S | Add a mandatory self-verification loop to the workflow: after writing/updating tests, actually run the suite and coverage, parse failures, fix, and re-run until green and gates pass — then report real (not illustrative) results. Frame as run->analyze->fix->re-run. Both the peer VoltAgent test-automator agent and the r/ClaudeAI coverage-writer+analyzer pattern bake in an execute-and-verify loop; ours only says to 'include example test runs', leaving open the failure mode of shipping unrun, non-compiling tests. |
| P2 | S | Add a flaky-test handling sub-section: how to detect (rerun-to-confirm, failure-rate-by-branch), then quarantine-and-fix rather than retry-blindly; never paper over flakiness with blanket retries. Codecov and qaskills both treat flakiness as a measured, managed metric with a workflow; ours only states 'avoid flaky tests' with no detection or remediation path. |
| P2 | S | Add two suite-health signals: track per-test execution time (surface the slowest tests, not just enforce a global <10s budget) and use cyclomatic complexity to prioritize WHERE to add tests (high-complexity functions first). Codecov's effectiveness framework calls out both; they make the agent's gap-finding targeted instead of uniform, and the complexity signal answers 'what to test next' which ours leaves implicit. |
| P2 | S | Add a short 'language-agnostic core' note: the principles (pyramid, mutation, assertion strength, isolation, self-verify) apply across stacks; if the repo is not Python, map pytest->the project's runner (Jest/Vitest/JUnit/go test) and mutmut->Stryker/equivalent, keeping the Python examples as the reference implementation. Peer agents are stack-agnostic and this agent ships in a shareable suite; a one-paragraph mapping preserves the Python depth while removing the hard lock that limits reuse. |
★ Already at parity with the best comparable (gstack); the only grounded upgrades are cosmetic-to-small: surface the 're-freeze with /freeze' hint, collapse read+remove into one atomic block, and note the freeze state is machine-global.
Ours is a clean, idempotent state-file removal that already matches or beats the field. It guards the empty/missing case ('No freeze is currently active' and stop), records the previous path for the confirmation, removes ~/.claude/jacked-freeze-dir.txt, and confirms with the previously-frozen path. The companion /freeze and the security_gatekeeper.py PreToolUse hook (lines 1772-1810) fail-open if the freeze file is corrupted and only restrict Edit/Write/NotebookEdit (Read/Grep/Glob stay free), so simply deleting the file is the correct and complete reversal — no hook teardown needed. This is functionally equivalent to gstack's unfreeze (116k stars), the strongest comparable in the ecosystem, and the external 'domain technique' art is genuinely thin (mostly clones of this same pattern).
Best-in-class sources found
Gaps
Recommended changes
| P1 | S | Add a re-freeze hint to the confirmation output, e.g. append: 'The freeze hook stays installed for the session — run /freeze <path> again anytime to re-restrict.' This mirrors gstack's unfreeze, which explicitly closes the loop back to /freeze. gstack (the 116k-star canonical comparable) and the Awesome Skills listing both surface this; it removes the user's uncertainty about whether protection can be re-enabled without restarting the session. |
| P2 | S | Collapse Steps 1-4 into a single atomic conditional: if the freeze file exists, capture its contents as $PREVIOUS_PATH, rm it, and report 'Freeze removed (was: $PREVIOUS_PATH)'; else report 'No freeze is currently active.' — matching gstack's single-block read/remove/report. Eliminates the read-then-remove gap and makes the no-op branch and success branch a single deterministic flow; gstack implements exactly this shape. |
| P2 | S | Add one line to the success message noting the freeze file is global, so unfreezing also lifts the boundary for any other Claude Code sessions running on this machine. Grounded directly in the implementation: the path is ~/.claude/jacked-freeze-dir.txt (global, not per-session), confirmed in both freeze.md and security_gatekeeper.py. No comparable warns about this, so it is a genuine, non-invented edge to surface. |
★ Add the field's #1 practice — 'describe the problem and its user impact, never prescribe the fix' plus a positive-acknowledgment opening — and a first-class Robustness/edge-case+empty-state aspect; our orchestration (parallel diff-scoped fan-out, cross-page tiering, discoverability) already exceeds every external source.
Ours is materially more advanced than the popular external art on orchestration and scoping, and we must not regress these: (1) True parallel fan-out across an (aspect × page) matrix with 2-4 isolated agents — OneRedOak's celebrated workflow is single-agent and sequential; (2) git-diff-driven auto-scoping of which pages/aspects to test, including staged+untracked new-file detection — no external source auto-scopes from the diff (Bowser requires hand-authored YAML stories); (3) cross-page impact tiering (Tier 1/2/3 with path+filename signals and explicit exclusions like index.*/*.module.css) to expand coverage on shared-infra changes; (4) new-feature Discoverability detection with entry-point inference, navigation-depth counting, first-use/empty-state and return-navigation checks — absent from all external sources; (5) persona inference from project signals + mobile-context weighted-signal detection; (6) credential auto-discovery from .env with infra-var exclusion; (7) robust multi-browser-tool detection/fallback (Chrome DevTools MCP → Playwright → Claude-in-Chrome → agent-browser) with per-tool tool lists and tab isolation; (8) severity triage, dedup, read-only contract that hands findings to /dcr + writing-plans. This is a complete, dev-loop-native execution stack.
Best-in-class sources found
Gaps
Recommended changes
| P0 | S | Add a 'Communication Principles' block to the Agent Prompt Template (Step 9) and the report rules: (1) Problems Over Prescriptions — describe the problem + its user impact, do NOT prescribe the fix (e.g. 'spacing feels inconsistent with adjacent elements, creating clutter' not 'change margin to 16px'); (2) lead every report with a one-line positive acknowledgment of what works before listing issues. This is the most-cited best practice in the canonical OneRedOak design-review agent and keeps /ux a clean diagnostic that feeds /dcr+writing-plans rather than half-prescribing fixes it isn't allowed to make (matches our read-only contract). |
| P0 | M | Add a 7th aspect 'Robustness & States' (or fold into Interactions as required checks): stress-test content overflow (very long strings, many list rows, narrow columns), exercise every form with invalid input to confirm validation messaging, and explicitly verify loading / empty / error states as distinct screens. Add to the aspect table, the per-aspect checklist, and the Build Test Matrix weighting (JS/form changes → weight Robustness). OneRedOak makes this a dedicated phase; it catches the highest-frequency real-world UI breakage (overflow, missing empty states) that our current Interactions checklist only grazes. Empty-state coverage already exists for Discoverability but not for modified pages. |
| P1 | S | Tighten the Accessibility checklist to WCAG 2.1 AA specifics: require contrast ratio >= 4.5:1 for body text (3:1 for large text), verify Enter/Space activates focused controls, and confirm visible focus state on every interactive element. Replace the vague 'text should be legible' line. Concrete, checkable thresholds raise signal and match the field standard (OneRedOak, WCAG). Agents can verify 4.5:1 via DevTools/contrast computation; 'legible' is unverifiable. |
| P1 | S | Add a compact 'Heuristic Lens' note to the Interactions and Discoverability checklists framing the relevant Nielsen heuristics as questions: Is system status visible during async actions? Is there an escape hatch / undo for destructive or multi-step actions? Are labels jargon-free (match real world)? Is the UI consistent with the rest of the app? Can the user recover from this error? Turns mechanical 'does it click' checks into usability evaluation. Grounded in NN/g's 10 heuristics — the domain's reference framework — and complements (does not replace) the existing aspect checklists. |
| P2 | S | Add a 'Content & Microcopy' line to the Console & Network aspect (or pair them as 'Content & Console'): check visible text for grammar, clarity, and jargon, and flag mislabeled or ambiguous buttons/headings. OneRedOak pairs content review with console as the final phase; microcopy bugs are cheap to catch in a browser pass and high-impact for novice users (ties to the Novice persona we already select). |
| P2 | M | In the persona section of the Agent Prompt Template, instruct each agent to pick one realistic goal per selected persona and actually complete that task end-to-end on the assigned page ('think-aloud': note each point of friction or hesitation), in addition to static aspect scoring. UXAgent and usability practice show task-completion traversal surfaces friction that static checks miss; our personas currently only bias scoring rather than driving a flow. Low cost since agents are already in the browser. |
★ Add an explicit time-criticality / cost-of-delay principle to the Step 6 decision so a genuinely dated, urgent item can outrank the biggest cross-cutting lever — the one prioritization input ours is currently blind to.
Ours is materially more sophisticated than anything found in the field. Specifically worth preserving: (1) Commits to ONE high-leverage initiative instead of returning a menu — the explicit anti-pattern that the only real competitor (claude-toolkit /next, which returns 5 items) falls into. (2) The coverage-matrix strategic lens (personas x contexts, capability vs walked-experience, hunt for cross-cutting levers that lift many cells) — no competitor has any strategic read; they all just scrape signals. (3) Confidence calibration with an explicit honesty bar (inline read is capability-only, never claims near-10/10 on experience, downgrades confidence on a 7-vs-9 call, offers /coverage-matrix before betting weeks) — directly counters the field's #1 failure mode of treating a thin read as truth. (4) The /goal brief forging with the HARD 4000-char size gate measured mechanically via wc -c, convergence-sizing/phasing for XL work, file-backed self-bootstrapping pointer-goals, and a turn backstop — far beyond any competitor and structurally matches Cursor's validated bounded-loop pattern. (5) Thorough DATA-only prompt-injection hardening carried into the brief (cite by identifier + neutral paraphrase, [text omitted] for directive-like titles). (6) Graceful degradation (not-enough-context stop, GH-not-auth handling, lifecycle inference table). (7) Resume-first checkpoint awareness and repo-scoped config override.
Best-in-class sources found
Gaps
Recommended changes
| P1 | S | Add a time-criticality / cost-of-delay decision principle to Step 6. Insert a bullet such as: '**Honor deadlines (cost of delay).** Scan signals for genuinely dated pressure — a regulatory/compliance cutoff, a launch or market window, a seasonal peak, a contractual SLA, an externally-blocked dependency expiring. A dated, high-cost-of-delay item can rightfully outrank a bigger cross-cutting lever; when one exists, name it and weigh delay cost against leverage explicitly rather than defaulting to the biggest move.' Tie it to concrete signals (issue labels like time-sensitive/deadline, dates in plans/issues, milestone due dates) so it stays evidence-grounded, not invented. WSJF's central, well-established insight (Cost of Delay / Job Duration) is the one prioritization input ours structurally omits — it weighs impact, effort, and confidence but is blind to urgency. Without it, a real dated blocker can lose to a larger-but-not-urgent lever. Grounding it in detectable date/label signals keeps it honest. |
| P2 | M | Add lightweight argument handling for a wrap-up mode. Near the top of the command (after the intro / before Step 1), add: 'If invoked with an argument like `done`, `finished`, or `wrap up`, switch to wrap-up mode instead of the full analysis: summarize what changed this session (git diff/log since the branch point), offer to open a PR (via /pr or gh), and offer to record follow-ups surfaced this session into the repo's planning doc (TODO/BACKLOG/ROADMAP) or as GitHub issues — then end by offering a fresh /whats-next for the next direction.' Keep it short and reuse existing gatekeeper-safe commands. The only real competitor (claude-toolkit /next) ships exactly this dual-mode behavior and it is a natural fit: 'what's next' is most often asked right after finishing something. It closes the loop (capture-then-decide) cheaply and reuses tools the suite already has (/pr, gh). |
| P2 | S | In the Step 6 'Also weighed' template, require a one-token source tag per runner-up, e.g. '- [runner-up lever] — [Effort] — [source: issue #42 / ROADMAP §3 / TODO src/api.py:88] — [one line].' Mirror the identifier-only, DATA-only citation rule already used for Evidence. claude-toolkit tags every suggested action with provenance, which makes a quick-glance list trustable; ours does this for the headline call but the runners-up are terse. A single bracketed source token per runner-up adds trust at near-zero cost and stays consistent with the existing security/citation discipline. |