Every finding from the autonomous quality loop lands here — what shipped, what's mid-flight, and what's queued — scored by complexity, impact, and a little bit of wow.
Shipped in #65 (this card lingered as "next" after the fact).
core/lockfile.py now writes through util.atomic_write_text
(temp file + os.replace()) and read() tells "empty"
apart from "corrupt": an unparseable lock is preserved as <lock>.corrupt
and surfaced loudly rather than silently overwritten, so an interrupted or
concurrent write can no longer drop every prior install record.
Fixed by #60: the bogus mutants/None gitlink (a stray mutmut
artifact with no .gitmodules entry) that made Pages' submodule
checkout abort with fatal: No url found for submodule path 'mutants/None'
is gone. Verified: the Pages "build and deployment" now completes successfully and
the site serves — the boards live at
jonnyeclectic.github.io/boost/docs/roadmap.html (200), since Pages
publishes from the repo root.
core/store.pyCut mutmut survivors 54 → 29 and uncovered a latent
timestamp-preservation bug the old tests masked (second-precision
now_iso() collisions hid installed_at/tags
preservation mutants).
core/gitutil.py35 → 1 survivor via argv-assertion tests that record the exact
git command line — sidestepping the macOS case-insensitive filesystem
that let HEAD/head, .git/.GIT and
git/GIT mutants survive against a real repo.
Shipped in #60. The generated mutants/**,
__pycache__/*.pyc and .coverage files were removed from
the index (verified: git ls-files now tracks zero of them), so they
no longer churn every diff or force path-scoped git adds.
browse crashes when you pick a rule or workflowcmd_browse lists every catalog entry — skills, rules
and workflows — then called store.install(picked)
unconditionally, so selecting a rule or workflow raised
… is a workflow, which boost indexes but cannot install yet and the
TUI exited with a fatal Error: after the user had already navigated
and chosen (reported from a real boost browse session on
AGENT-playbook-to-automated-agent-workflow). Fixed in #118:
the install call now catches BoostError and renders the message +
hint as a friendly non-fatal notice (exit 0), so a non-skill pick — or an
already-installed / pinned skill — no longer crashes the browser. Covered by a
regression test that picks a workflow and asserts no fatal exit.
The "is this skill outdated" decision — semver_gt → commit compare → sha256_dir vs lock — was reimplemented nearly verbatim in cmd_update, cmd_outdated and _drift_status — three copies of core business logic that would drift apart. It now lives in one place: core/staleness.py exposes two pure (I/O-free) decisions — upstream_reason() (the version→commit→content ladder rendered by cmd_update/cmd_outdated) and drift_state() (the store/local-edits/upstream ladder rendered by _drift_status). The commands became thin renderers; behavior is byte-for-byte preserved and the new module is fully unit-tested (every branch pinned, all mutants killed).
core/frontmatter.py34 → 11 survivors with exact-equality tests, replacing
substring in checks a mutated string literal still
satisfied (e.g. dump()'s XX%s:XX still
contains tags:).
core/dense.pyThe dense-vector RAG backend's whole test module is
skipif-gated on the [rag] sqlite-vec C
extension, so on the default zero-dependency install
every degradation and ranking path went untested
(17.6% → 89% coverage). New tests force
_load() → None and drive the SQL helpers and
retrieve()'s cosine reducer through in-memory/fake
connections — killing mutants on every machine, extension or not.
core/logs.pyThe black-box diagnostic logger swallows every filesystem/handler
failure so a broken log can never break the CLI — but those
except arms (the parts that matter most during a
crash) were the least exercised. New tests force each failure —
handler close() raising, a read-only log dir, an
unresolvable version, a wedged logger, unlink denied —
taking the module to 100% coverage.
configuration.py~200 lines implementing a full HTTP catalog server and a JSON-RPC 2.0 MCP
server lived inside the "configuration" command module — nearly untestable in
place. The JSON-RPC protocol now lives in core/mcp.py as a pure
handle_request(req, *, version, registry) plus a
serve_stdio() loop with injectable stdin/stdout; the HTTP server
moved to a new core/serve.py with a pure route()
function, serve_page()/skill_text(), and
serve_http(). cmd_serve and cmd_mcp --stdio
became thin wrappers. Behavior is preserved exactly (the full-server and
stdio-protocol functional tests pass unchanged) and the extracted core is pinned
by new unit tests covering every branch.
The loop now runs commit → PR → green CI → squash-merge → release in a
dedicated ~/boost-loop git worktree (own venv for mutmut),
and modern git 2.55 replaced a 2017 build that
fatally choked on the global zdiff3 config.
self-update version detection (dead branch)boost self-update greped __init__.py for a
__version__ = "…" literal that setuptools-scm never
writes, so the regex never matched and it always reported
"already up to date" — even after a pull brought a newer tag.
The success path was unreachable. It now derives the pulled
version straight from git (describe --tags, mirroring
_detect_version()) and treats a moved HEAD
as the "an update landed" signal, so the version-bump path finally
fires. Regression tests cover both the up-to-date and update-landed
branches.
-- guard on git commandsclone_shallow passed the clone URL as a positional with no
-- separator, so a value beginning with -
could be read as a git flag, and git's remote-helper transports
(ext::sh -c …, file::, fd::)
can execute arbitrary commands straight from a URL. Now it refuses
those transports outright and passes -- before the URL
so option parsing can't be hijacked — argument-injection
defense-in-depth beside registry.parse_spec. Kept scoped
to the one primitive that takes a user-influenced URL (the refs in
pull/reset are constants, and --
there would wrongly mean a pathspec). Six tests, mutation-verified.
A native abort (e.g. a macOS Obj-C fork-safety SIGABRT) kills the
process outright, so it never reaches boost's Python crash recorder —
leaving no boost-side trace to confirm or clear boost when an OS crash
report appears. log_invocation now records
pid, ppid and the interpreter path on every
run, so any OS crash report can be cross-referenced by PID: if the
crashing PID never appears in ~/.boost/logs/boost.log,
boost is definitively ruled out.
Shipped in #126. The byte-identical _user() getpass helper —
copy-pasted three times across core/journal.py,
commands/configuration.py and commands/team.py — is now a
single public core/util.user() the three call sites delegate to
(covered incl. the getpass-raises fallback). cmd_simulate also parsed
the same frontmatter twice (_, body = … then
meta, _ = …); collapsed to one
meta, body = frontmatter.parse(text).
cmd_migrate's two-agent validation is intentionally left as-is —
its first-fail wording differs from _check_agents' collect-all
message, so routing it through the helper would change user-facing errors.
docs/index.html inlines its own amber/orange palette
instead of the canonical Aurora cyan→violet→pink system in
style/boost.css. Move it onto the shared stylesheet so
one token edit recolours the guide, roadmap, and demo together.
core/Audited the surviving mutants across all ten listed modules — catalog, util,
ai, registry, lockfile, policy, config, output, agents, paths — and
killed every one that reflected a real behavioral coverage gap: the
search desc-bonus accumulation and classify_workflow
signature logic; score_skill signal detection and the
rel_time bucket boundaries; registry.remove keeping
other taps; a 3-level config.unset slicing bug; the untested
lockfile.remove_rule/remove_workflow returns and
corrupt-history skip; the entirely-untested policy.check_capabilities;
and confirm()'s safe default. The residual survivors are
equivalent mutants — encoding defaults, macOS case-folding (killed
on CI's Linux runner), += vs = on a zero-init score,
// vs / under %d, dead defensive defaults
carried by DEFAULTS, and test-double-stubbed plumbing kwargs —
none of which represents a missing assertion. The required mutation gate
stays green with margin.
commands/ under mutation testingThe ~8,100-line command layer has zero mutation coverage: mutmut is
scoped to core/ only. A blocking 80% floor was attempted and the
baseline has now actually been measured — it does not clear the bar, and
two of the three constraints recorded here earlier were wrong. What a sample run
of commands/taps.py (the best-covered module in the package,
95.8% lines) against tests/unit/ + tests/functional/
reports through this repo's own gate:
570/793 killed — 71.9%, under the 80% floor, with all 223
survivors spread across every one of the module's 7 functions rather than
concentrated in one testable gap. Mutant density is 4.2 per
statement in taps.py but 2.46 across
core/ (9,909 mutants over 4,033 statements), so
commands/'s 5,314 statements imply somewhere between
13,000 and 22,300 mutants; at the measured
1.42 mutations/sec — roughly 6.5× slower per mutant than the
core/ job, which matches the functional-vs-unit suite cost — that is
2.5 to 4.5 hours on one runner, against an ~18-minute job today.
The job sets no timeout-minutes, so it inherits GitHub's 360-minute
cap rather than failing fast.
Two blockers sit underneath that number. Selecting only tests/unit/,
the way core/ does, leaves commands/ at
17.9% line coverage versus 91.5% with functional included — and
"no tests" mutants count against the score, so that route floors out near 18%.
Selecting tests/functional/ instead crashes the run: mutmut's
record_trampoline_hit calls p.resolve(strict=True) on the
relative source path, so any test that chdirs into a temp
project dies with FileNotFoundError: <tmp>/boost_cli — confirmed
on test_verify_sees_a_project_skill. Corrections to the earlier note:
pytest_add_cli_args_test_selection is not single-valued, it
is a list that configparser splits on newlines (a space-separated line is what
errors), and a sibling pytest_add_cli_args takes extra pytest flags.
One prerequisite is already fixed: --no-mcp leaked through
os.environ and made the suite order-dependent, which mutmut exposes
because it runs whichever test subset covers each mutant.
Drive docs/index.html in a real browser across breakpoints
to confirm the glass cards, aurora, and terminal windows render
cleanly — and to catch anything the theme reconciliation shifts.
The front page had drifted from the product: the README claimed 72 commands
in one place and 78 in another (78 is correct), and ~2,600 mutants against a
real ~9,900. A number a reader can check in ten seconds costs more credibility
when wrong than the feature it describes earned. All three corrected — and pinned,
so they cannot rot again: a unit test derives the command and group counts from
cli.COMMANDS and fails if the README, the landing page, or the two
disagree with each other. The README hero now leads with the one-line pitch and a
four-command block that shows the whole flow before any prose. And
demo.gif — a generated artifact whose regeneration needed
brew install vhs on someone's laptop, so in practice it drifted (a
merged PR had to fix it for showing ./boost long after that changed)
— is now re-recorded by CI whenever the surfaces it demonstrates change, proposed
as a PR because a GIF re-encode always differs byte-for-byte and its correctness
is visual, not comparable.
config.load() in-processEvery config.get() re-reads config.json and runs a recursive deepcopy of DEFAULTS (core/config.py:80–102). It's called all over hot paths — ai.enabled, per-skill enabled_agents in sync loops, log-level and policy checks — so one command triggers dozens of full reads + deep-copies. Cache the load, invalidate on save.
Already shipped: core/store.py's _copy_skill stages the
full copy in a temp dir on the same filesystem, then swaps it in with
two atomic os.replace renames and rolls back to the original
on any failure — the old rmtree-then-copytree window is gone. Covered by
test_copytree_failure_preserves_existing and the flaky
os.replace test in tests/unit/test_store.py.
journal._maybe_rotate is a racy read-modify-write that concurrent appends can truncate mid-line; rag._save and config.save repeat the non-atomic pattern (journal.py:63 · rag.py:252 · config.py:93). Factor a single atomic_write into core and route all four through it.
_tilde() — two copies have a boundary bugEight command modules each define _tilde; the quality and intelligence versions use startswith(home) with no separator check (quality.py:78), so /Users/bob-backup wrongly contracts to ~-backup. Collapse to one helper in core/paths.
rag.retrieve() calls all_entries(), which reads & json.loads every tap cache on every search to build the live map (rag.py:323 · catalog.py:166–181) — the BM25 index is mtime-cached, the entry set is not. Memoize the entry set on cache mtime the same way.
catalog.search computes json.dumps(meta).lower() for every entry on every query just to substring-match (catalog.py:222). Precompute a lowercased search blob at index time, or match structured fields directly.
scan_dir walkTwo full rglob passes descend into .git/node_modules and only filter afterward, and the skill-dir membership test is O(files × skill_dirs) (catalog.py:106,119–123). Switch to an os.walk that prunes ignored dirs in place; index skill dirs in a set.
Shipped in #140. The canonical detect_stack prober — a
~75-line pure function that lived in the command module
commands/discovery.py yet was imported across the command layer by
both intelligence and quality — now lives in one place:
a new pure core module core/stackprobe.py (with its private
_SKIP_DIRS/_EXT_LANGS/_read_text helpers,
used nowhere else). Every consumer imports the single core prober;
discovery re-exports it for compatibility, and the now-dead helpers
+ unused os import were removed from it. Covered by 16 new
mutation-gated unit tests. The consumer-local enrichment fallbacks
(_local_stack, _STACK_MARKERS) are intentionally kept —
they add coarse filesystem signal the language-only prober doesn't.
Three separate regexes scan SKILL.md bodies for "Always / Never / Must / Do not" lines across cmd_explain, simulate and conflict (info.py:363 · intelligence.py:254 · quality.py:822) — same concept, three implementations. Extract one shared core extractor.
quality.py (15 cmds / 1,264 lines) mixes audit regexes, conflict NLP, decay scoring, fingerprinting, a health dashboard and tap-provenance; intelligence.py and configuration.py are similarly overloaded. Split along cohesive seams to shrink blast radius. First seam (this PR): lift the installed-skill safety & integrity commands — audit, verify, attest, quarantine — into commands/safety.py, with the two shared helpers in commands/_common.py; quality.py drops from 1,264 to ~955 lines. Tap-side trust and the health/diagnostics set stay put.
tag argument parsingBecause -tag looks like an option, cmd_tag calls parse_known_args then re-walks raw argv to reorder tokens (info.py:575–584) — fragile enough that the comment calls it "defensive". Replace with a positional-only sub-parser or an explicit --add/--remove design.
_make_docs stored piece[:200] — the head of the best
chunk — and retrieve surfaced it verbatim, yet the docstrings
promised the "best-matching passage": when the matched terms sat past the
chunk's first 200 chars, the shown snippet (and the rerank context fed to the
LLM) missed them entirely. Now the index stores a larger head of the matched
chunk (SNIP_STORE) and retrieve windows a
SNIP_WIDTH-char passage centered on the first query term,
with ellipsis markers on trimmed edges (_passage). Ranking is
untouched — the eval harness confirms recall@10 holds at
0.919 — and the index grows a bounded ~34% (an
INDEX_VERSION bump forces the one-time reindex); storing whole
chunks instead cost ~70%.
Already shipped: core/frontmatter._scalar coerces only the YAML 1.2
core keywords (true/false/null/~);
the 1.1 aliases yes/no/on/off/none stay strings, so a skill
named none or tagged on keeps the right type.
Guarded by test_scalar_leaves_yaml11_aliases_as_strings.
Today store.install refuses every non-skill kind
("rules and workflows show up in boost search/boost taps
for now"), so rules are indexed for discovery but land nowhere —
there is no boost install path for a rule. Add one that
writes each rule into the form the target agent actually reads, since
there is no single cross-agent "rules folder": Cursor/Windsurf/Cline
consume a rules directory (.cursorrules /
.windsurfrules / .clinerules / .mdc),
but Claude Code has no rules folder — its standing
rules are CLAUDE.md. So installing a rule for Claude means
merging it into CLAUDE.md (a managed, idempotent block),
while for the others it means dropping the file into their rules dir.
Needs a store/lock model for rules (uninstall must cleanly remove the
merged block), mirroring how skills symlink into enabled_agents().
After rules landed (#141), store.install still refused
kind == "workflow", so slash commands and subagents were
indexed for discovery but installed nowhere — browse could
surface a workflow but not install it. Add the install path. Unlike
rules (no cross-agent rules folder, so Claude needs a
CLAUDE.md merge), workflows are a clean file drop: a
command markdown lands in the agent's commands/ dir and a
subagent in its agents/ dir, with the slot derived from the
source path (commands//workflows/ →
commands, agents//subagents/ →
agents). Mirrors the rule store/lock model so uninstall
removes exactly what install wrote.
boost install --local into the projectboost was user-global by construction: one store at
~/.agents/skills, symlinked into every home-level agent
dir. Right for the skills you use everywhere, wrong for the
ones a team agrees on. Shipped the npm
--save half: boost install <skill> --local
(= --scope project) writes into the repo's own
.claude/skills/, .cursor/skills/, … and
records a committable per-repo lock at
.boost/skill-lock.json. Real directories, never
symlinks — a link into the author's ~/.agents/skills
arrives dangling on a teammate's machine, which is the exact problem
committing skills is meant to solve. Two separate locks, because
~40 call sites resolve a locked skill to
~/.agents/skills/<name>, which a project skill does
not have. Scope resolution walks up for the nearest project root, so
installing from src/deep/nested lands in the repo instead
of scattering a .claude/ three levels down;
list --local, uninstall --local and
sync all understand both scopes, and sync
re-materializes what a fresh clone is missing while never deleting a
directory boost did not write.
boost list shows installed rules and workflowsRule (#141) and workflow (#150) install landed, but
boost list still read only the lock file's
skills section — so a rule or workflow installed from
browse showed up nowhere, and worse, the empty-state
("no skills installed") fired whenever no skill was
present even if rules/workflows were, hiding them entirely. Extend
list to render an installed rules and
installed workflows table (agents drawn from each item's
recorded materializations, workflows also showing their slot), gate
the empty state on all three kinds being empty, and move
--json to a {skills, rules, workflows} shape.
--tag stays skill-only.
Workspace scope shipped in #212
— install --local, list --local,
uninstall --local, info and
sync all understand the per-repo lock. The other ~70
commands still read the user lock alone, and an adversarial review
of that PR named the consequences: boost update and
outdated can't see a vendored skill (the workaround is
install --local --force), and the governance
commands — audit, verify,
drift, doctor, health,
fingerprint, attest — report a clean bill
of health while N third-party skills sit in the repo being loaded by
every agent on the team. That last one is the real prize: vendored
skills are exactly the ones a security review should be looking at,
because they arrive by PR and run on everyone's machine. Shipped the governance slice — the real prize — via a shared
integrity.project_skills() / project_status() pair: verify and
doctor now check project-scoped skills' committed digests the same
way they do user-scope ones, so a drifted vendored skill is flagged
instead of silently trusted.
boost update refreshes installed rules and workflowsAfter rule (#141) and workflow (#150) install landed,
boost update still upgraded only skills — a rule or
workflow stayed frozen at its install-time content even after its tap
moved. Extend the update pass: for each installed rule/workflow from a
refreshed tap, re-materialize (force reinstall) when the source
version bumped or its file content sha changed, mirroring the skill
upgrade loop. Rules/workflows carry no pin/quarantine flags and their
source is a single file, so there is no risky-diff gate — re-applying
a file drop or a CLAUDE.md managed block is cheap and the
refresh is reported per item.
sync rules/workflows like skillsTwo skill-only gaps remained after rule (#141) / workflow
(#150) install: the install-time injection + secret scan
read res.dest/SKILL.md, which doesn't exist for a
rule/workflow (a single file / merged block), so their executable
Markdown went unscanned; and boost sync only reconciled
skills, so a deleted rule/workflow materialization couldn't be
repaired. Fix both: carry the raw source on the install result
(scan_text) so injectscan/secretscan see exactly what was
installed, and extend sync_plan/sync_apply
with a missing_materializations pass that re-materializes
a rule/workflow from its tap when a drop file or CLAUDE.md block is
gone — mirroring the missing-store-dir repair for skills.
boost install --scope user|project for rules/workflowsRule and workflow install always materialized into user-global agent config
(~/.claude/CLAUDE.md, ~/.cursor/rules, …), so a
rule meant for one repo leaked into every session. Add
--scope project: materialize into the current repo instead —
<repo>/.cursor/rules/, <repo>/.claude/commands|agents/,
and, since Claude reads per-repo memory from the root and has no rules
folder, <repo>/CLAUDE.local.md (the personal, git-ignored
file). The chosen scope + base dir are recorded in the lock so
update/sync re-materialize back into the same
repo rather than wherever they happen to run. Default stays
--scope user (unchanged behavior). Uninstall already reverses
by recorded path, so it works for either scope.
registry.get(name) matches on full name, safe-name, or trailing path segment and returns
the first hit with no ambiguity check — unlike catalog.resolve_one, which
explicitly errors on a multi-tap ambiguous match. Three of the five default taps end in
/skills, so after boost tap --defaults, boost untap skills or
boost update skills silently resolves to whichever tap sorts first and the others
become unreachable by short name. Raise on an ambiguous short-name match, mirroring
resolve_one.
Shipped. registry.get() is now tiered — exact owner/repo, then safe_name, then the bare repo tail — and refuses a short name that matches more than one tap instead of taking the first. Two corrections while fixing it: there is no sort, the winner was whichever came first in config.json; and the fix was half a fix as originally scoped. catalog.find() carried an independent, untiered tap in (e["tap"], tail) membership test, so with both taps configured angular/skills:brainstorming also matched microsoft/skills, and boost bundle apply took matches[0] — silently installing from the wrong tap, verbatim the defect this item exists to fix. Fixing only registry.get would have left the CLI self-contradictory: boost untap skills erroring while a Boostfile silently guessed. Both resolvers are tiered now and bundle apply refuses an ambiguous entry rather than picking one.
sync --apply deletes any broken symlink, not just boost's ownsync_plan flags every broken symlink under an enabled agent dir as stale regardless of
whether it points into boost's store — the points_into_store check only runs on the
live-link branch — and sync_apply unconditionally unlinks everything in
stale_links. A user's own unrelated broken symlink sitting in
~/.claude/skills gets deleted by boost sync --apply with no ownership
check. Only treat a broken symlink as boost-managed if its raw readlink() target
resolves under the store dir. Shipped as store.points_into_store(), applied to
broken and live links alike. Two things turned up while fixing it. boost clean
carried the identical overreach — it unlinked every broken symlink under an agent dir with no
ownership test at all — so it now shares the same helper; fixing only sync would
have left boost contradicting itself. And the original check was a substring test
(str(store_dir) in str(target)), which also matches siblings like
~/.agents/skills-backup; the comparison is component-wise now, and a relative
readlink() target is resolved against the link's own directory before it is judged.
Comparing properly is what exposed a Windows bug the sloppy check had been hiding: since 3.8
os.readlink() there returns the reparse point's substitution path, which carries the
\\?\ extended-length prefix, so boost's own link reads back as a
different drive and stopped matching the store at all. Every Windows leg of the matrix
went red on it. The prefix is stripped before comparison, and the behaviour is pinned on every
platform via ntpath, which imports fine on POSIX.
_retrieve_any treats any non-None dense result as final, but
dense.retrieve() returns [] (not None) whenever every KNN
neighbor gets filtered by kind mismatch or staleness — silently short-circuiting the documented
"everything degrades to BM25" contract. Compounding it, dense.build()'s incremental
path only prunes chunks for taps still present in the current entry set, so a tap removed via
boost tap remove leaves ghost vectors in rag_vectors.sqlite forever,
crowding the KNN pool on every future query. Distinguish "dense unavailable" from "dense had zero
live hits," and prune removed taps on every build, not just changed ones.
_maybe_rotate reads the whole pulse file and atomically replaces it with a truncated
snapshot — but two concurrent boost processes (explicitly expected per this repo's parallel-loop
model) can both read, then both write their own snapshot, and whichever writes last silently
discards any event the other appended in between. rotation_healthy() also opens the
file with a bare p.open() and never closes it, relying on GC instead of a
with block like every other open in the module. Rotate under a lock (or an
append-only rename scheme), and close the handle explicitly.
Shipped with util.try_lock(), a portable O_CREAT | O_EXCL advisory
lock that needs neither fcntl (absent on Windows) nor msvcrt (absent
everywhere else). It yields False rather than waiting: a process that cannot take it
just returns, because another one is already trimming and blocking would trade a rare lost
update for a common stall. A lock older than five minutes is stolen, so a process killed
mid-rotation cannot wedge the feed forever. Inside the lock the file is re-read (the count
that got us there was taken outside it) and anything appended since is carried into the new
file instead of dropped. log() stays lock-free — O_APPEND writes of
one short record do not tear, and making every command contend on a lock to protect an
advisory feed would be the wrong trade.
Writing the test for a torn append turned up a third defect neither the card nor the code
had noticed: a single invalid byte in the feed raised UnicodeDecodeError, which
is a ValueError and sails straight past the except OSError guard —
so one bad byte crashed every command that logs, permanently. Both reads decode with
errors="replace" now.
boost uninstall has no confirmation promptboost uninstall deletes a skill's store directory and lock entry straight through
shutil.rmtree() with zero confirmation — every other destructive command
(snapshot restore, cohort delete, profile delete,
untap, replay rollback, bmad uninstall) gates on
out.confirm() first. Shipped: a single prompt naming every skill about to go,
before anything is touched, plus -y/--yes and the existing
BOOST_ASSUME_YES bypass.
The prompt is gated on sys.stdin.isatty(), not just on the flag.
out.confirm() returns its default — False — when stdin is
not a terminal, so the obvious if not out.confirm(...) would have turned
boost uninstall x in every CI step, Makefile and Dockerfile into a silent no-op
exiting 1. Guarding a destructive command must not break the callers that cannot see the
guard; a regression test drives the command with a non-TTY stdin and no
BOOST_ASSUME_YES to hold that line.
update/reinstall silently widen a skill's agent scopeA skill installed with --agent narrowing (e.g. boost install foo --agent
claude-code) records that subset in the lock, but boost update/
boost reinstall force-reinstall via store.install(entry, force=True)
without passing only_agents — so link_agents relinks into every currently
enabled agent and silently overwrites the lock's narrower agent list. Pass
only_agents=lk.get("agents") on both force-reinstall paths, matching what the
rule/workflow update path already does for scope.
lint --tap mis-scores rule/workflow entries as brokenboost lint --tap builds its target list without filtering by kind, so rule/workflow
catalog entries (which have no SKILL.md) always report "missing SKILL.md," and
repo-root items get scored against the entire tap directory instead of their actual file. Running
it against any tap that mixes rules/workflows with skills makes every rule/workflow entry show a
bogus error and a garbage score. Skip or special-case non-skill kinds, mirroring the kind-branching
already used elsewhere in pkg.py.
sync relinks a narrowed skill into every agentstore.preserved_agent_scope (PR #288) stopped update and
reinstall widening a skill installed with --agent narrowing, but
sync_plan() was a second, independent path to the same leak: it walked
agents.enabled_agents() for every locked skill and reported a
missing_link for any agent the skill was not linked into — the lock's recorded
scope was never consulted. sync_apply then dutifully linked them, so a
single boost sync undid the narrowing that install --agent asked for and
rewrote the lock to match.
It was left out of #288 because the fix is a real trade-off, not an oversight:
boost sync is also how a newly enabled agent picks up skills installed
before it existed, and honouring the lock's agents list would silently break that —
agents records what is linked right now, so reading it as a scope would
freeze every existing skill out of any agent enabled later.
Resolved by separating the two questions. A new only_agents field records what the
user asked for, written only by an explicit --agent and carried forward across
update/reinstall; agents keeps meaning what is actually linked.
sync_plan filters through scoped_agents(), which fails open — an entry
with no declaration, which is every entry written before this field existed, still fans out to
every enabled agent. So narrowing is honoured and the newly-enabled-agent path is untouched.
PR #289 added a third Dependabot entry, package-ecosystem: pip with
directory: /, to give pyproject.toml's optional extras the proactive bump
PRs they had never had. The first scheduled run after it merged produced eight PRs, and two
pairs of them are duplicates: #301/#302 (twine) and
#300/#303 (hypothesis) change byte-identical file sets — one PR from
the / entry and one from the pre-existing /requirements entry, for the same
bump. Merging either makes the other redundant, and Dependabot closes the twin automatically, so the
cost is noise rather than breakage. It will recur every week.
Worse, the entry cannot deliver what it was added for. Every constraint under
[project.optional-dependencies] is either an open lower bound
(sqlite-vec>=0.1.6) or explicitly listed in that entry's own ignore
block (the pinned langchain 0.3 stack held back for ragas). Dependabot does not raise a
version PR when the declared range already admits the newest release, so the set of
pyproject.toml version updates this entry can ever produce is empty — which is
why none of the eight touched pyproject.toml.
Do not simply revert it. Dependabot security updates ignore the versioning strategy and fire
on a vulnerable version inside a permitted range, so the entry does give the extras security
coverage they previously lacked — the item it closed was right that
[rag]/[bdd]/[perf] had only reactive pip-audit flags. The
entry should be narrowed, not removed.
Two candidate fixes, both needing verification against a real scheduled run rather than reasoning —
Dependabot's file discovery is what got this wrong in the first place. First, scope the
/ entry so it stops matching requirements/**, leaving
/requirements as the single owner of the hash-pinned lock. Second, if proactive
extras bumps are genuinely wanted, that needs versioning-strategy: increase so open
lower bounds are raised at all; otherwise accept that this entry is security-only and say so in the
comment, which is the honest smaller change.
boost onboard silently overwrites existing generated filesboost onboard writes .boost/telemetry.json, a GitHub Actions workflow, and
.skill-lock.json via a bare write_text() with no existence check,
confirmation, or diff — unlike the sibling "write a generated file" helper elsewhere in the
codebase, which always confirms before overwriting. Re-running it on a repo with its own tracked
lock file silently clobbers it and still reports "created," even though it overwrote. Check
dest.exists() and route through out.confirm() first.
Every Dependabot PR that touches requirements/*.txt is unmergeable, and the
failure is not in the bumped package. Dependabot re-resolves the lock on Linux and writes that
resolution back, which silently drops entries whose environment marker excludes them on the
resolving platform — but the lock is installed with --require-hashes on Windows and
macOS too, so the dropped pins become install failures on the platforms that need them.
Two of two observed in one scheduled run. The hypothesis bump (#300/#303)
deleted colorama==0.4.6 ; sys_platform == 'win32' — pytest's Windows terminal
dependency — from all three of test-tools, coverage-tools and
mutation-tools, so tests (windows-latest, 3.12) and
(windows-latest, 3.14) both died in the install test deps step with the
suite never running. The twine bump (#301/#302) deleted
colorama and pywin32-ctypes (keyring's Windows backend, and keyring is
twine's own dependency) plus pip and setuptools from
release-tools.txt, failing the metadata job in Install build +
validation tools. The GitHub-Actions bumps in the same run were all fine: they are pure SHA
pins in workflow YAML and involve no resolution.
scripts/lock_toolchain.py is the source of truth precisely because it resolves for
every supported platform and keeps the conditional pins. Dependabot has no equivalent. The same run
also rewrote the provenance comments from -r requirements/test-tools.in to
-r test-tools.in, i.e. it compiled from a different working directory — the
machine-independence that test_header_is_machine_independent already guards.
The cheapest honest fix is to stop asking Dependabot to do this: drop the version-update side of the
pip entries (or set open-pull-requests-limit: 0, which leaves security updates
firing, since those are what the entries are actually worth) and regenerate the locks on a schedule
with lock_toolchain.py instead. Whatever is chosen, it needs a guard: a lock that
loses a platform-markered pin should fail its own gate rather than fail three Windows jobs
two steps later, because the current failure names the install step and never mentions the missing
package. Related: [[dependabot-root-pip-entry-duplicates-requirements]].
-n silently inverts log/pulse outputboost log/boost pulse accept -n/--limit as a bare
type=int with no positivity check, and the slicing does
out[:n] if n else out — a negative n becomes a Python negative slice, so
-n -1 silently returns everything except the most recent event instead of erroring.
discovery.py's --limit flags already share a _positive_int
validator that rejects values below 1; reuse it here.
boost log --crashes listing branch has no non-empty test_show_crashes has an empty-state branch and a listing branch that reads each
crash-*.log, extracts its summary line, and swallows OSError on an
unreadable report — only the empty-state message is exercised by any test. A regression in the glob
sort, the summary-extraction regex, or the OSError fallback would ship undetected. Add
a test that seeds one or more crash logs and asserts on the rendered listing.
boost serve's own path-traversal guards are untestedserve.py's defenses against a malicious catalog skill_md path
(_is_within, _safe_join_within, the ".." in rel.parts check)
and its HTTP-handler failure modes are the least-covered lines in core/ — 84.5% file
coverage, the lowest of any module — and no test actually feeds a ..-containing path
through skill_text() to prove the guard fires. Add adversarial tests for the traversal
guards and for _CatalogHandler's error paths.
core/logs.py uses stdlib logging with rotation and crash reports, but its
file formatter only ever emits fixed plain-text lines — unlike core/journal.py, which
already writes one JSON object per line for the pulse feed. A BOOST_LOG_FORMAT=json
option emitting the same fields as structured records would let
~/.boost/logs/boost.log feed straight into jq or a log aggregator instead
of needing regex parsing.
core/ai.py's CLI and API call paths both catch their failure modes (timeout, OS error,
URL error, bad JSON) and return None — but neither ever calls into the logger, unlike
the rest of boost. A user whose expired key or flaky network silently degrades every AI-assisted
command to its heuristic fallback has no trail to diagnose why, even with --debug. Add
a one-line logger.debug(...) in each except branch.
bandit via ruff S rulesTurn on the flake8-bandit (S) rule family already
bundled in ruff — one line in pyproject.toml, zero new
tools. It catches the Python SAST smells the linter is blind to today:
subprocess(shell=True), assert as a runtime
check, tempfile.mktemp, unsafe yaml.load,
hardcoded secrets and weak hashing. Semgrep Community rules are the
heavier alternative when a finding needs real dataflow.
pip-auditPyPA's pip-audit fails CI when a resolved dependency
matches a known advisory in the OSV / PyPI Advisory database.
Dependabot opens update PRs but never blocks a merge on a live
CVE; a scheduled + PR pip-audit job closes that
gap and covers the dev/test toolchain that ships nothing but can still
run malicious code.
Boost's unit tests prove the retrieval math is arithmetically correct,
but nothing graded whether search returns the right skill for a real
question. This adds a golden-set harness — tests/eval/golden.jsonl,
43 query → relevant skill(s) judgments across
skills, rules and workflows — scored with rank-aware IR metrics
(recall@k, hit@1, MRR,
nDCG@k). Deterministic and offline over the tapped catalog, so it
runs as a local pre-release gate (make eval,
--fail-under 0.85) alongside make check, with
a pinned baseline (tests/eval/baseline.json) that flags
regressions. The
first run earned its keep immediately, exposing a duplicate-name double-count
(recall 2.7) fixed by deduping the ranked list. A three-way engine compare
proved BM25 full-content (0.919 recall) decisively beats the frontmatter
heuristic (0.756), justifying the RAG stack. An opt-in, key-gated
make eval-ai arm reuses the same labels to measure the LLM
rerank lift — no judge needed — and confirmed rerank promotes the right skill to
#1 in 16% more cases (hit@1 0.605 → 0.767) at unchanged recall. A
Tier 2b arm (eval_recommend.py, make eval-rec)
grades the boost recommend AI pick stage over golden project
stacks with a hard grounding gate: it found the AI picks statistically tied with
the heuristic on precision but perfectly grounded (0 hallucinated /
off-shortlist picks) — the honest verdict that this stage earns its place through
explanations and safety, not better ranking.
The Scorecard GitHub Action audits ~18 supply-chain signals weekly — branch protection, token permissions, SHA-pinned actions, dangerous workflows, signed releases — and publishes results to the Security tab plus a README badge. Turns "are we set up safely?" into a tracked, trend-able number for a repo that owns a Trusted-Publisher release path.
The golden set (43 queries / 8 stacks / 7 skills) is small enough that
engine comparisons don't yet reach significance — the paired t-test lands at
p≈0.11–0.26 on the pinned corpus, so BM25's numeric lead over the
heuristic can't be called statistically real. Expand and diversify it: many more
labelled queries with balanced per-kind coverage (skill / rule / workflow),
harder near-miss distractors, and judgments for the still-unevaluated surfaces
(search --smart, detect_stack). Tighter confidence
intervals mean comparisons that actually reach significance and a gate that
catches subtler regressions. The single highest-leverage extension of the eval
harness.
gitleaks + push protectionScan the full history and every PR diff for leaked tokens, keys and
PyPI credentials with the free gitleaks action, paired
with GitHub push-protection (free on public repos) so a secret is
blocked before it lands. Cheap insurance for a project whose
release automation trades on trusted identity.
xenon · vultureGate CI on a maintainability grade with xenon (built on
radon) and flag unreachable branches, unused arguments and
orphan helpers with vulture. First target: the
~5,000-line commands/ layer that has no complexity signal
today. Surfaces the structural smells mutation testing can't see.
Tier 1b (the ranx paired t-test, --stats) only runs on
demand today. Wire it into a scheduled CI monitor — like the
eval-explain workflow — that tracks whether BM25's lead over the
heuristic, and any new engine's lead over BM25, is statistically
significant rather than merely numerically higher, and flags when a change
erases a previously-significant win. Stays out of the required stdlib gate (ranx
is an opt-in [eval] dependency); it's a non-blocking monitor, so a
noisy p-value can never block a merge. Turns "recall went up 0.02" into "the
improvement is real (or isn't)."
One external surface over everything the offline gates measure separately: bugs, code
smells, security hotspots, duplication and a coverage overlay, decorating each PR
with a quality gate. Free for public repos. Settings live in
sonar-project.properties — boost_cli as sources with
tests declared separately (so the suite's asserts do not score as
duplication), the generated version stub and boards excluded, coverage read from
the Cobertura report CI already emits rather than re-running the suite.
Non-blocking and inert until onboarded: without SONAR_TOKEN the job
skips and writes what to do into the run summary, so a skipped run explains
itself instead of looking broken. Not added to required checks — boost's own
gates stay authoritative because they run offline and a contributor can
reproduce them.
hypothesis on the parsersGenerate adversarial inputs against core/frontmatter and
core/catalog.scan_dir to surface crashes and round-trip
failures the example-based unit tests never think to try — a
dump→parse must round-trip; a scan must never
raise on arbitrary bytes. Complements the gate: mutmut proves the tests
are strict, Hypothesis proves the inputs are wide.
atheris / OSS-FuzzGoogle's atheris instruments the bytecode and evolves inputs toward
uncovered branches in boost's two hand-rolled parsers — the SKILL.md frontmatter
reader (a stdlib-only YAML subset, so no upstream project's fuzzing covers it)
and the tap-spec parser. It has already paid for itself: the frontmatter harness
found that numeric coercion was lossy. version: 1.10 is ten
patch releases past 1.1, but float("1.10") is 1.1 — so
boost read a skill published at 1.10 as 1.1, compared it as older than 1.9, and
never offered the update, with boost outdated reporting "everything
up to date" while the tap was nine releases ahead. Note that "it doesn't crash"
would have missed it entirely: the invariant has to compare the parsed value
against the source text. The seeds through every invariant run in the required
suite so a harness cannot rot; the coverage-guided run is a weekly non-blocking
job, because a timed search is not reproducible enough to gate a merge. Targets
follow the OSS-Fuzz Python contract, ready for continuous runs once boost
qualifies.
ProxyHandlercore/ai.py and core/embed.py call
urllib.request.urlopen with the default opener, which on
macOS runs getproxies_macosx_sysconf() →
_scproxy → SystemConfiguration/CoreFoundation — an
Obj-C path that is not fork-safe and aborts on the
child side of a fork(). Harmless in today's single-process
CLI, but fragile if boost ever runs post-fork (a future multiprocessing
worker, or an embedding host). Build the opener with an explicit
ProxyHandler honoring HTTP(S)_PROXY/NO_PROXY
so network calls never touch the Obj-C proxy machinery — deterministic
and fork-safe.
Zcore/logs.py builds the logging.Formatter with
datefmt="%Y-%m-%dT%H:%M:%SZ" but never sets
converter = time.gmtime, so %(asctime)s uses
local time while stamping a literal Z (UTC)
suffix. Every line in ~/.boost/logs/boost.log is therefore
off by the machine's UTC offset — which nearly caused a 5-hour
mis-correlation when matching the log against an OS crash report. Set
the formatter's converter to time.gmtime (or
drop the Z) so the trail is honestly UTC, making the
pid/ppid crash-correlation breadcrumbs trustworthy.
[tool.mypy] in pyproject.toml sets only python_version/
files, leaving mypy in its default permissive mode — a function with any untyped
parameter has its body skipped entirely. All 76 cmd_* functions across
boost_cli/commands/*.py take a bare argv, and a third have no return-type
annotation, so the required "zero mypy errors" lint gate isn't actually checking most of the CLI's
dispatch logic. Scoping disallow_untyped_defs (or at least
check_untyped_defs) to boost_cli.commands will surface a real backlog —
exactly what the gate is supposed to catch.
Dependabot #236 bumps ruff 0.15.22 → 0.16.0 and the ruff step
fails with 83 errors. None of them are new code — 0.16 widened the default
selection, and because pyproject.toml uses extend-select
(which adds to the defaults rather than replacing them), every newly-defaulted family
switched on at once. The project only ever opted into
S, B, SIM, C4, PERF, RUF, UP; what fires now is
I001 (33), PLW1510 (19), BLE001 (9),
ISC004 (4), plus TRY, PLR, PIE,
DTZ, RET, PYI, FLY and — awkwardly —
FURB167, a family the config explicitly delegates to refurb via
external = ["FURB"].
So this is a policy decision, not a lint fix, and there are two honest answers.
Pin the policy: switch extend-select to a full select = [...] so
the rule set is stated outright and a future ruff release can never widen it again — the
bump then lands with zero code changes. Or adopt some of them: I001 (import
sorting) and the 43 auto-fixable errors are cheap and arguably worth having, while
PLW1510 (subprocess.run without check=) is 19 real
call sites that each need a judgement about whether a non-zero exit should raise.
DTZ005/006 (naive datetime.now()) is worth a look on its own
merits given the clock-racing test.
Recommended: pin select first so the bump is unblocked and the gate stops
depending on an upstream default, then adopt families deliberately in follow-ups.
Shipped that way in #260: pyproject.toml now states the rule set outright with
select = ["E4", "E7", "E9", "F", "S", "B", "SIM", "C4", "PERF", "RUF", "UP", "I"],
so a future ruff release can only change how the chosen rules behave, never which rules run.
I001 was adopted (34 sites, entirely mechanical); ISC was
considered and declined — all 8 hits were deliberate line wrapping. external = ["FURB"]
is preserved, so FURB167 stays refurb's call rather than ruff's.
requirements/lint-tools.txt carries ruff==0.16.0 and Dependabot #236
was closed as superseded. PLW1510 and DTZ005/006 remain unadopted and
are the deliberate follow-ups.
name[extra]==version, so a valid pin fails the gatetests/unit/test_toolchain_lock.py (added with the lock-stability fix, #248)
asserts every requirement is pinned with ==. Its regex is
(?P<name>[A-Za-z0-9][A-Za-z0-9._-]*)(?P<spec>==[^\s;\\]+)? — and the
name class has no [ or ]. So against
coverage[toml]==7.15.2 the name matches coverage, the spec then
fails to match because the next character is [, and spec comes
back None. The test reports "coverage is not pinned with == in
test-tools" about a line that is pinned exactly.
Latent today — nothing in requirements/*.txt currently carries an extra — but
uv/pip-tools emit name[extra]==version routinely, so the first
legitimate extras-bearing pin will block the gate with a message that points at the wrong
thing. Fix by allowing an optional extras group in the name, e.g.
[A-Za-z0-9._-]*(?:\[[^\]]+\])?, and add both forms to the test's own cases.
Found the honest way: Dependabot #253 (hypothesis) regenerated the lock and tripped this
assertion. That PR is genuinely bad for an unrelated reason — it collapses
coverage==7.10.7 ; python<'3.10' and
coverage==7.15.2 ; python>='3.10' into a single
coverage[toml]==7.15.2, and 7.15.2 requires Python ≥3.10, so it silently
drops the 3.9 leg this project promises. The gate correctly refused it. But it
refused for the wrong reason, and that is the part worth fixing: the same message will one
day reject a lock that is perfectly fine. See
the pytest card — collapsing a
marker-split pin is the recurring shape here, and is worth its own assertion.
boost doctor checks installed rules and workflowsAfter rule (#141) and workflow (#150) install landed,
boost doctor's health loop still only walked the lock
file's skills — so a rule or workflow whose materialized
file a user deleted (or whose CLAUDE.md managed block was
hand-removed) went undetected. Extend the doctor loop to verify every
recorded rule/workflow materialization is still on disk: a file drop
must exist, and a Claude rule's CLAUDE.md must still carry
its managed block. A missing materialization is flagged with a
boost reinstall hint and flips the verdict, matching how
skill-drift is surfaced.
rel_time tests race the wall clock and flake on a loaded runnerTestRelTime builds its input with
iso_ago(n) = (now() - n).strftime("%Y-%m-%dT%H:%M:%SZ"), which
truncates sub-second precision, then asserts on
util.rel_time() calling now() a second time. The elapsed
delta is therefore n + frac(first_now) + runtime, and
rel_time floors it — so whenever
frac(first_now) + runtime ≥ 1.0 the bucket is one higher than the
test expects. Observed failing on tests (ubuntu-latest, 3.14):
assert '31s ago' == '30s ago'.
The off-by-one cases are the mild ones. iso_ago(59) and
iso_ago(59 * 60) sit directly on a bucket boundary, so the same
race flips "59s ago" to "1m ago" and
"59m ago" to "1h ago" — a different unit, not a
neighbouring number. Every assertion in the class shares the window; it is
narrow on a quiet machine and widens with runner load, which is why it reads as
random redness rather than a broken test.
Fix by giving the test a fixed clock — monkeypatch
util.datetime (or inject a now seam into
rel_time) so both reads come from one frozen instant, and the
boundary cases become exact rather than probabilistic. Freezing is preferable to
widening the assertions: the boundaries are precisely the behaviour worth pinning.
Shipped. It came true exactly as described: the mutation gate on main went red with assert '1m ago' == '59s ago' — the iso_ago(59) boundary named above — which skipped the release. Fixed by freezing the clock rather than widening the assertions: a frozen_clock fixture pins util.datetime to one instant and iso_ago measures back from that same instant, so both reads can no longer drift apart. The two absolute-date cases were rebased onto the frozen instant too — they read the real clock and would otherwise never have agreed with it.
zizmorboost's four workflows embed github-script JavaScript and
shell and drive a Trusted-Publisher release — a rich attack surface.
zizmor statically flags template injection, unpinned
actions, over-broad GITHUB_TOKEN permissions and dangerous
triggers. The one tool that audits the automation that ships every
other fix.
actionlintCatches GitHub Actions YAML bugs before they fail a live release:
malformed ${{ }} expressions, deprecated syntax, and
shellcheck run over every run: block. Cheap insurance for a
repo whose release is fully automated — a broken workflow is a broken
publish.
actions/attest-build-provenance emits a cryptographically
signed, verifiable record of which workflow built which wheel from
which commit, layered on top of the existing PyPI Trusted
Publishing. Consumers can gh attestation verify the
artifact they install — provenance without a paid signing service.
Generate a CycloneDX SBOM with the free anchore/sbom-action
and attach it to each GitHub Release, so downstreams can inventory and
scan exactly what boost is built from. Modest today (boost is close to
stdlib-only) but it grows in value as the optional [rag]
extras pull in real dependencies.
osv-scannerGoogle's osv-scanner cross-checks a lockfile or the SBOM
above against the OSV database, with broader ecosystem coverage than a
single-language audit. The SBOM-driven complement to pip-audit:
one scans the resolved env, the other scans the published manifest.
pyrightRun Microsoft's pyright alongside mypy. Its independent
inference catches None-flow and narrowing bugs the current
mypy config lets slide, and it's the same engine most editors use — so
CI enforces what contributors already see. Two type checkers rarely
agree on nothing.
B·SIM·C4·PERF·RUFBeyond the S security family (round 1), enable
flake8-bugbear (B), simplify (SIM),
comprehensions (C4), perflint (PERF) and
Ruff-native (RUF) rules — bug, readability and performance
smells caught at zero new-tool cost since ruff already runs in the lint
gate.
diff-coverEnforce coverage on the changed lines of each PR — no external
service needed, it reads the same coverage.xml the suite
already emits. Pairs with the 80% project gate so new code can't quietly
ride in under-tested behind an already-high overall number.
publish.yml ignores the pip-audit / metadata gatespublish.yml triggers on workflow_run: [ci] and gates only on
github.event.workflow_run.conclusion == 'success' — it never checks the
independently-triggered pip-audit.yml or package-metadata.yml workflows
that run on the same push. A merge that fails the live-CVE gate or the twine/metadata check still
auto-publishes to PyPI as long as ci itself is green.
Shipped as the second option: scripts/release_preflight.py runs as the release job's
first step — before release-drafter creates the tag, so a blocked release leaves no
published Release behind — and waits for each required workflow to conclude for
git rev-parse HEAD, the commit actually checked out and about to be built.
Folding them into ci.yml was rejected: pip-audit also runs weekly on a
cron, which inside ci.yml would drag the whole 9-cell test matrix along with it.
The gate fails closed — red, cancelled, skipped, never started, still
running at the deadline and an unreadable API reply are all refusals. Silence is the dangerous
case: a gate that never ran leaves nothing red to see, so it must never read as consent.
timeout-minutes on any CI jobA hung step — a stalled tap clone, a wedged smoke-test subprocess — runs until GitHub's default
job timeout instead of failing fast, costliest across the 3 OS × 3 Python tests
matrix. (The card said 8 workflow files; there are 24 now, of which only fuzz.yml
and sonarcloud.yml had a timeout.) Shipped: 24 jobs across 22 files, with the
numbers taken from measured durations over recent runs rather than guessed —
mutation's slowest observed run was 24.8m so it gets 60, the tests
matrix peaked at 8.3m so it gets 30, publish's release job gets 45 because the
new release preflight can legitimately wait on a sibling gate, and everything else ran under
2m so it gets 15. A timeout that trips on a normal run is worse than no timeout.
osv-scanner.yml's scan-pr is the one job left without one: it
delegates to a reusable workflow via a job-level uses:, and GitHub rejects
timeout-minutes there outright — the bound has to live in the called workflow.
A unit test holds the line so a job added next month cannot quietly arrive without one, and
it refuses to pass vacuously if its own parser stops finding jobs.
pip entry misses pyproject.toml's extras.github/dependabot.yml declares pip for
directory: /requirements, so the hash-pinned dev/CI toolchain does get weekly
bump PRs. What it does not cover is pyproject.toml at the repo root:
Dependabot only scans manifests under the declared directory, so the optional
extras — [rag] (sqlite-vec), [eval] (ranx, ragas and the pinned
langchain 0.3 stack), [bdd] (behave), [perf] (pytest-benchmark) —
never get proactive version-bump PRs, only reactive pip-audit CVE flags.
Low impact by design: [project].dependencies is empty, so none of this reaches
anyone who installs boost-skill-cli — it is contributor-facing only. The fix is
a second pip entry with directory: /. Weigh it against the noise:
the [eval] stack is deliberately held at langchain 0.3 because ragas
0.2.x breaks against langchain ≥1.0, so that one will raise PRs that must be closed
unmerged until ragas catches up.
Nothing in CI checks SPDX license compatibility of resolved dependencies — pip-audit
gates known CVEs, not license terms, and the [eval]/[rag] extras pull in a
nontrivial transitive closure (the langchain stack, ragas, sqlite-vec) whose licenses are never
verified against boost's own. Add a pip-licenses --fail-on (or reuse lint)
step to package-metadata.yml to catch an incompatible transitive dependency before an
extra ships it.
pytest tmpdir CVE has no fix at our Python floorCVE-2025-71176 — pytest through 9.0.2 relies on the predictable
/tmp/pytest-of-{user} path, letting a local user cause a denial of
service or possibly gain privileges. Dependabot alert #1 (medium), raised against
requirements/test-tools.txt.
It cannot be closed by a version bump. The lock carries two markered pins, and only
one of them is exposed: pytest==9.1.1 ; python_full_version ≥ '3.10' is
already past the 9.0.3 patch, while
pytest==8.4.2 ; python_full_version < '3.10' is not — and cannot be, because
pytest 9.x declares requires-python ≥3.10 while this project declares
requires-python = ">=3.9". The advisory marks every 8.x release
vulnerable, so there is no backport to move to either. The fix is gated on dropping the 3.9
floor, which is a deliberate compatibility promise, not an oversight.
Residual exposure is contributor-facing only: test-tools.txt is the dev/CI
toolchain and [project].dependencies is empty, so nobody installing
boost-skill-cli ever receives pytest, and CI runners are single-tenant
ephemeral VMs where the local-attacker precondition does not hold. Dismissed as
tolerable_risk rather than left open to rot the alert list.
Revisit when either the 3.9 floor is dropped or a patched 8.x lands. If the floor must stay
and the alert must clear, the real mitigation is pointing pytest at a private base
temp dir (--basetemp / PYTEST_DEBUG_TEMPROOT) so the predictable
path is never used — that fixes the behaviour without satisfying Dependabot's version check.
Scorecard files into the same code-scanning inbox as CodeQL, so its findings read like
code defects and get ignored alongside the false positives. They are not defects — they
are posture metrics, and they must not be dismissed as "false positive" the way the
serve.py traversal alerts legitimately were, because that misrepresents the
repo's security posture. Triaged 2026-07-25:
Fixed — TokenPermissions #26/#27/#28: ci,
codeql and adapter-conformance had no top-level
permissions: block at all, scoring 0. Added contents: read as the
least-privilege default.
Intentional, leave — TokenPermissions #39/#37/#25/#24:
publish.yml needs contents: write to create the release and tag,
osv-scanner needs security-events: write to upload SARIF, and
ci.yml's tests job needs contents: write for the
coverage-badge push. "Fixing" these breaks the release pipeline. They should be
dismissed as used-in-tests/acceptable-risk with a reason, not left open to rot the list.
Not code-fixable — MaintainedID is repository age; CodeReviewID
counts approved changesets and is structurally 0 while loops self-merge;
CIIBestPracticesID is an opt-in badge programme; the four
PinnedDependencies hits are npx and pip invocations
inside steps, not actions, so SHA-pinning does not apply to them as written.
BranchProtectionID is tracked separately in
its own card — it is a workflow decision, not a
config oversight.
main has no branch protection, so the release rules are honour-systemGET /branches/main/protection returns "Branch not protected". Scorecard
flags it (BranchProtectionID, score 0), but the sharper way to see it is this:
CLAUDE.md says "never merge onto a red release" and "every merge to main cuts a
PyPI release" — and nothing enforces either. A merge with a red gate succeeds
silently, and the release workflow then ships that commit to PyPI. The rule exists only in
the head of whoever is merging.
This is a decision, not a bug, which is why it is filed rather than fixed. Requiring
status checks would have stopped the exact situation that occurred on
2026-07-25, when three PRs sat red on a lock_toolchain --check failure that
belonged to none of them. But protection cuts both ways here: the repo's whole working model
is parallel loop/* branches opening and merging their own PRs, so
"require pull request reviews" would deadlock every loop (there is no second
reviewer), and that same absence is what keeps CodeReviewID pinned at 0/30.
The shape that fits: require status checks to pass (at minimum lint
and the tests matrix) and require branches to be up to date, but do not
require reviews. That enforces the release rule that actually matters while leaving
self-merge intact. Decide before adding more concurrent loops, not after.
Update — the blocker is gone. This prescription was not implementable as written: lint named three different jobs (ci, markdownlint, theme-lint), and GitHub matches required checks by name alone. Those collisions are now renamed, the required list is checked-in at .github/required-checks.txt and gated against drift, and python3 scripts/check_required_checks.py --print-api emits the exact payload — with required_pull_request_reviews: null, so self-merging loops keep working. Still a decision, but now a one-command one.
Shipped. The decision was taken as prescribed — status checks yes, reviews no.
GET /branches/main/protection now returns strict: true with
required_pull_request_reviews absent, so a red gate blocks the merge that would
have cut the release, and loops still self-merge. One correction to the prescription above:
requiring "at minimum lint and the tests matrix" is right, but the
list must contain only checks that run on every PR — the first version also
required four path-filtered docs checks, which report on some PRs and not others and would
have hung any PR that touched no matching file. See
required-checks-can-declare-a-check-that-deadlocks-prs.
adapter-conformance's LangGraph leg never passed — a quoted matrix valueThe install step ran
pip -q install "${{ matrix.pip }}". Quoting is correct for the two legs
whose matrix value is a single token (crewai[anthropic],
openai-agents[litellm]) — the brackets would otherwise be shell globs.
But the LangGraph leg's value is a requirement list,
langgraph langchain-anthropic, so the quotes handed pip one bogus
requirement and it refused before installing anything:
ERROR: Invalid requirement: 'langgraph langchain-anthropic'.
The leg landed 2026-07-22 and the workflow was schedule/workflow_dispatch
only, so nothing exercised it on its own pull request — the first Monday cron was
its first execution ever, five days later, and it has a 0% pass rate. It read as
upstream framework drift; it was never that. Reproduced locally, and with the
step fixed the whole leg passes end to end against live PyPI: langgraph 1.2.9 +
langchain-anthropic 1.5.2 resolve clean, boost adapt --to langgraph
renders, and the factory builds a real
langgraph.graph.state.CompiledStateGraph.
Fix: pass the spec through env: (rather than inlining a
${{ }} expansion into run:, the shape zizmor flags) and
let it word-split under set -f, which keeps the [extra]
brackets literal — without it, crewai[anthropic] glob-expands against
any matching filename. Also added a narrow pull_request trigger on
adapters.py and this workflow, so an edit to the thing under test is
tested on the PR that makes it. That trigger is path-filtered, so it must stay out
of the required-check list: a required context that does not report on a PR leaves
it pending forever.
paths:, so it green-lit a list that deadlocks every PR.github/required-checks.txt declared 21 contexts and
scripts/check_required_checks.py reported
"21 required, 22 PR check names, no ambiguity" and exited 0. Four of those 21 —
validate, markdown-lint, theme-lint and
vale — are produced by path-filtered workflows.
When a pull request touches none of a workflow's paths:, GitHub does not
create a pending check run that later resolves; it creates no check run at all.
Branch protection then waits forever for a status that is never coming, and the pull
request can never merge. Applying that list would have bricked the repository — and
--print-api would have emitted the payload to do it.
The cause is one regex. The gate decided "runs on pull_request" with
re.search(r"^\s{2}pull_request:", header), which is true for any workflow
that mentions the trigger, whether or not paths:,
paths-ignore: or types: [labeled] narrows it — precisely the
distinction that decides whether a required check deadlocks.
Demonstrated live rather than argued: the PR that fixed the
adapter-conformance install touches no style/** file, and its 30
check runs contain validate, markdown-lint, vale,
check and sweep — but no theme-lint. Requiring
theme-lint would have hung that PR permanently.
Fix: classify the trigger as none/always/filtered and refuse any required name that
resolves only to a filtered workflow, with an error that names the culprit file. The four
docs checks move to a commented block explaining why they are useful but not requireable.
The list is 17 contexts, all verified to report on every PR. Ten new tests cover the rule
and all ten fail against the previous script.
ci.yml's job summary could exit 1 on its own, under the always() it was givenThe job summary step is marked if: always() so a run still reports
when something upstream died. Its last line was
[ -n "$SMOKE" ] && echo "- **Smoke suite:** ${SMOKE}", which is the final command
of the { ... } >> "$GITHUB_STEP_SUMMARY" group, and that group is the final
command of the step. GitHub runs shell: bash as
bash --noprofile --norc -eo pipefail, so with $SMOKE empty the
AND-list's status of 1 becomes the group's status and then the step's exit code.
$SMOKE is empty precisely when smoke.sh never printed its
== results: line — an earlier step died first, or smoke never ran. That is the
exact scenario always() exists for, so the reporting step failed in the one case
it was written to survive. Reproduced directly: exit 1 with no match, exit 0 with one.
It never masked a real failure (the job is already red when this happens), but it planted a
second misleading red step, and would have silently reddened the job if that step were ever
moved into a green context. Replaced with an if block.
post-deploy.yml's always() destroyed the second signal it existed to preserveThe console-health step carries if: always() with the comment
"an HTTP failure above must not hide a console failure here — they are different breakages
and both belong in one report". But the two steps it depends on —
actions/setup-node and the npm install that fetches puppeteer — had
no if: at all. A failure in the HTTP smoke skips both, so the console
check then runs with no node_modules and produces an environment error rather
than a console verdict.
The run history separates the two cleanly: an environment error exits 2, a genuine
console failure exits 1. Run 30214061945 has
HTTP smoke = failure, setup-node = skipped, npm = skipped, console = failure (exit 2),
while the four runs where the npm install actually happened all exit 1. So in exactly the
case the always() was written for, the second signal was destroyed instead of
preserved — and the report said "console check failed" when it meant "the console check
never ran".
Fixed by putting always() on the setup steps too, so the dependency chain the
final step needs survives the failure it is meant to outlive.
.markdownlint-cli2.jsonc globs **/*.md and its ignores
list carved out .claude/** and docs/roadmap/items/** but not
tests/fuzz/corpus/**. So 12 of the 21 files being prose-linted were
deliberately-malformed YAML-frontmatter seeds for the atheris fuzzer —
07-unterminated.md, 09-unbalanced-quote.md, 12-bom.md
and friends. Malformed is their entire purpose, and they passed the gate only by luck.
That luck was going to run out by design. fuzz.yml exists to mine new
malformed inputs and ship them as reproducers
(-artifact_prefix=artifacts/, uploaded as fuzz-crash-<target>),
and one such crash artifact already exists from a real run. The moment anyone committed it into
tests/fuzz/corpus/frontmatter/ — exactly what the workflow is for — an unrelated
prose-style gate would have gone red for a reason with nothing to do with prose, on a file
whose whole point is to be invalid.
Fixed by adding tests/fuzz/corpus/** to ignores. Verified with the
pinned markdownlint-cli2@0.18.1: 21 → 9 files linted, 0 errors, and a planted
crash reproducer no longer reddens the gate.
Only ci.yml declares merge_group among its triggers. The other
workflows producing required contexts — codeql.yml,
pip-audit.yml, package-metadata.yml, osv-scanner.yml —
run on pull_request only.
GitHub's merge queue evaluates the required checks against the temporary
merge_group ref, not against the pull request. So the moment a merge queue is
turned on, codeql-analyze, pip-audit, metadata and
scan-pr / osv-scan would never report on the queued ref and every enqueued PR
would sit forever — the same never-reports deadlock as
required-checks-can-declare-a-check-that-deadlocks-prs, arriving through a
different door.
Nothing is broken today; no merge queue is configured. Filed because the trap is invisible
until the switch is flipped, and because the pressure to flip it is real: with
strict: true and several concurrent loops, every merge forces every other open
PR to rebase and re-run its full matrix, which is exactly the problem merge queues solve.
Fix before enabling, not after: add merge_group: to each workflow that produces
a required context, and extend scripts/check_required_checks.py to fail when a
required name comes from a workflow lacking that trigger — the same gate that now catches
paths: filters.
Measured across twelve consecutive runs, the mutation job took
20.6–39.2 minutes (median ~27) while the next-longest job in the same run,
tests (windows-latest, 3.12), took 8.6. One job was the critical path,
and it was charged twice per change: once blocking the pull request, then again on
main, because publish.yml fires on workflow_run after
CI completes — so the PyPI release waited on it too. PR #278 merged at 20:52 and the
release landed at ~21:20, essentially all of it this gate.
The cost is ~10,502 mutants over 45 files in boost_cli/core, run 4-way parallel
(max_children defaults to os.cpu_count(); ubuntu-latest
is 4 vCPU). Generating them takes 5.3 seconds — the entire cost is executing the unit
suite ten thousand times. A measured local baseline: 1600s wall, 6.70 mutations/second,
8850/10502 killed (84.3%).
Two independent wins, because they apply to different pull requests.
Skip what cannot have changed. Sampling the last 40 merged PRs, 38% touch neither
boost_cli/ nor tests/ nor the mutation configuration — docs,
roadmap cards, other workflows. Their score is identical to the base commit's by construction,
and they were each paying 26 minutes to prove it. The relevance rule is deliberately wider than
boost_cli/core/: setup.cfg's also_copy ships the whole
package into mutants/ and the tests import it, so a change in
commands/ can flip a core mutant. It also fails safe — no resolvable
base commit means run the full gate.
Shard the rest. mutmut run accepts fnmatch patterns over mutant names and
each source file owns a disjoint mutants/<path>.meta, so splitting by file
yields results that merge without any per-mutant reconciliation. Verified live:
mutmut run 'boost_cli.core.nethttp.*' ran 16/16 of that file's mutants and left
store.py at 0/1931. Six shards reach 5.44x — and no further, because
store.py alone is 18% of all mutants and cannot be split. That ceiling is printed
by plan --explain rather than left to be rediscovered.
The dead end, recorded so nobody repeats it. Caching mutants/ between runs
is the obvious first idea and it does not work in mutmut 3.6 — it actively destroys the
data it would need. copy_src_dir skips targets that already exist, so on a warm
tree create_mutants_for_file takes its source_mtime < mutant_mtime
branch and resets every exit code to None. Demonstrated rather than
inferred: a file with 235/235 recorded results came back with 185 wiped and the run
re-testing from zero. It would have looked like it was working while saving 5.3 seconds.
What adversarial review caught, which local testing did not. The first working
version sharded by glob("*.py") — non-recursive — while mutmut walks
source_paths with os.walk. A subpackage
(core/rag/bm25.py) would therefore be assigned to no shard; and because
export-cicd-stats skips a path with no .meta rather than
counting it unkilled, those mutants would drop out of total entirely and the
required check would report PASS over a subset. Fail-open — precisely inverting
the property the file's own docstring claimed. Two further finds: no fnmatch pattern can
address __init__.py, because mutmut rewrites .__init__. out of mutant
names (boost_cli.core.__init__.* matches nothing, and
boost_cli.core.* would swallow the whole package); and
git diff --name-only reports only a rename's destination, so moving
core/x.py to docs/x.py looked like a docs-only change and skipped the
gate. All three are now covered by tests that fail against the pre-fix code, and merge asserts
that every mutatable file was actually accounted for — so future layout drift reddens the
gate instead of quietly narrowing it.
The trap this had to avoid. mutation is a required status check
(.github/required-checks.txt). Path-filtering the job — the natural way to
implement "skip" — means GitHub creates no check run at all, and branch protection waits
for a status that never comes. This repository has already deadlocked that way twice. So the
required job always runs (if: always()) and inspects what the upstream jobs
actually did; always() on its own would have reported a failed shard as a green
required check. Merging likewise fails closed: mutmut counts an unrun mutant as "not checked"
inside total, and the gate divides by it, so a partial merge would quietly depress
the score instead of erroring.
What actually landed, measured on main. The eleven CI runs before this
merged took 20.7–39.4 minutes (median ~28.4); the merge commit's own run took
12.1. This pull request touches tests/, so it ran the full six-shard
gate — the worst case, not the docs-only path.
It also missed its own estimate, in an instructive way. The prediction was ~6 minutes for a code
change; the slowest shard took 9.9. Packing by mutant count is near-perfect — the six
shards carry 1703–1931 mutants each, within 1.10x of ideal — but measured
throughput ran 3.39 to 14.19 mutants/second, a 4.2x spread. Mutant count is simply a
poor proxy for time: a store.py mutant re-runs a far larger covering test set than an
ed25519.py one, and survivors run their tests to completion where kills exit early.
Re-weighting the planner by seconds instead of count would even out the billed minutes but
not the wall clock, because shard 0 is store.py by itself and its 9.5 minutes
is the floor. Going below it means splitting a single file — feasible, since mutant names are
addressable per function (boost_cli.core.store.install_*), which would bring the floor
toward the 5.1-minute even split of the 30.8 minutes of total work.
store.pySharding took the mutation gate from ~28 minutes to ~12 (see
mutation-gate-was-the-whole-critical-path), but it cannot go lower without
splitting one file. Measured across four consecutive six-shard runs:
638bffeb ran 1.9, 3.9, 5.7, 6.7, 7.2, 11.7 minutes;
02c18bd6 ran 2.7, 5.0, 5.2, 5.5, 5.9, 11.7; fc427839 ran 3.1,
4.8, 5.3, 6.4, 7.3, 9.7; and f42e617b ran 2.9, 4.9, 5.5, 6.6, 7.3,
9.3.
mutation-shard (0) was the slowest leg in all four. Total work is
36–37 job-minutes, so a perfectly even split would be 6.1; the actual critical
path is 9.3–11.7, or 1.5–1.9× that. The gap is not bad packing —
the planner balances mutant count to within 1.10× of ideal. It is that count is
a poor proxy for time: measured throughput spans 3.39 to 14.19 mutants/second, because a
store.py mutant re-runs a far larger covering test set than an
ed25519.py one, and survivors run their tests to completion where kills exit early.
Re-weighting the planner by seconds would even out the billed minutes but not the wall
clock, because store.py is 1931 mutants (18.4% of the repo) in one file and
longest-processing-time packing cannot place a single file across two bins. The floor is that
file.
The way under it: mutant names are addressable per function, so
boost_cli.core.store.install_* can be its own shard. That would bring the critical
path toward the 6.1-minute even split. Two things must survive the change —
pattern_for() still has to emit patterns that match nothing else, and
cmd_merge's completeness assertion has to keep failing closed, since mutmut counts
an unrun mutant inside total and a partial merge would silently depress the score
rather than error.
With the mutation gate no longer dominating, the largest remaining term in CI wall clock is
queueing. Measured over six consecutive successful ci runs: 104
job-minutes queued against 236 job-minutes executing — 31% of all job
time spent waiting for a runner.
The obvious metric lies. A run's run_started_at minus
created_at is 0.0 for every run, which reads as "no queueing anywhere".
Queueing happens per job, not per run, so it is only visible as
job.started_at - job.created_at. Anything measuring this at run level will
conclude, wrongly, that there is nothing to fix.
The per-run spread shows what drives it — median job queue by run:
c36c53a7 0.1 min, c15ed9da 0.1, 903b704f 0.2,
638bffeb 0.7, ad71ba99 2.5, 9e190537 2.9.
That is a 29× swing driven not by the change under test but by how many sibling
loop/* branches and Dependabot PRs happen to be live at the same moment. It is why
one merge took 41 minutes wall clock with a max shard of only 11.4.
Sharding the mutation gate made this worse on purpose: it traded one runner slot for six. That
is the right trade for latency in isolation, and the wrong one to keep making blindly while a
dozen branches are in flight, because every concurrent PR now multiplies its footprint. Options,
roughly in increasing order of intrusiveness: a concurrency group that cancels
superseded PR runs; capping the shard matrix with max-parallel; or making the shard
count adaptive. Worth measuring before choosing — this item is the measurement, not the
fix.
lint job pins zizmor==1.27.0 — a yanked releaseci.yml runs the workflow SAST step as
pipx run zizmor==1.27.0 .github/workflows. PyPI marks 1.27.0 as yanked, with
yanked_reason: GHSA-f42p-wjw5-97qh. It is the only yanked release out of 66,
and 1.28.0 is the successor.
Exact-pinning the security scanner is correct — an unpinned linter is how
ruff-defaults-broke-boost-lint happened. Pinning to a version its maintainer has
withdrawn for a security advisory is the failure mode on the other side, and it is
quieter: a yank does not stop an exact pin from resolving. pip installs
1.27.0 and emits a warning nobody reads in CI logs, so the required lint
gate goes on passing while running a withdrawn build of the tool that audits the workflows
driving a Trusted-Publisher release.
Two things worth doing together. Bump to 1.28.0 and confirm the four documented
dangerous-triggers exemptions in .github/zizmor.yml still suppress
cleanly — the line-anchored ignore entries (publish.yml:19,
sbom.yml:29, …) are exactly the kind of thing a version bump can invalidate.
And sweep the other exact pins for yanks: nothing currently checks for this. pip-audit
catches CVEs in the resolved dependency closure, but a yanked pinned tool is neither a
CVE in the closure nor a manifest diff, so osv-scanner does not see it either. This
was found by accident, while installing zizmor locally to verify an unrelated change.
publish.yml fires on workflow_run when ci completes, gates
on that run's conclusion == 'success' (plus event == 'push' and
head_repository == this repo), and then checks out ref: main. Those are
not the same commit. Observed 2026-07-28: ci for ecfe38bc
completed at 20:19:05Z; the release run was created at 20:19:07Z; it tagged and shipped
c1727bac as v1.0.271 — while c1727bac's own
ci, started 20:18:40Z, was still in_progress. One commit's green
verdict authorised a different commit's release.
Most of this is already mitigated, and the card should not be read as "untested code ships".
A strict "require branches to be up to date" policy means the shipped commit was green as a pull
request before it could merge, so its tree was gated. And
release_preflight.py is already pointed at the right commit —
--sha "$(git rev-parse HEAD)", with an explicit comment that this is "not the trigger
sha". It polls each required workflow for that exact sha and fails closed on red, cancelled,
skipped, never-ran or still-running-at-deadline. The machinery is correct.
Its coverage is the gap. .github/required-checks.txt lists 18 required
contexts. publish.yml passes preflight exactly two of them —
--require pip-audit.yml and --require package-metadata.yml. The other
16 are never re-checked against the commit being shipped, including all 14 that come from
ci.yml and codeql-analyze. The clearest evidence this is an oversight
rather than a decision is in-tree: release_preflight.py's own docstring says it waits
for every required workflow for the exact commit being released, and the caller hands it
two.
A required context that can never report on main at all. While checking the
other 16: osv-scanner.yml declares only pull_request and
merge_group triggers — no push. So scan-pr / osv-scan
is green as a PR gate and produces nothing for any commit on main, race or
no race. Adding it to preflight without first giving that workflow a
push: branches: [main] trigger would fail the release closed every time.
The fix is not to change the checkout ref. Pinning to
github.event.workflow_run.head_sha would build the older commit while release-drafter
still tags the default branch's head — artifact and tag would disagree. The lower-risk
change is to widen preflight's --require set to the contexts that actually run on
main, and optionally fail the job when git rev-parse HEAD differs from
the triggering run's head_sha. This is the third defect found in the same few lines
— see publish-trigger-was-reachable-from-a-fork (who can fire it) and
publish-gate-ignores-pip-audit-and-metadata (which gates are consulted); this one is
which commit they were consulted about.
demo.yml still fails on every push — and the fix is not in the workflowdemo-gif-workflow-has-never-succeeded (shipped, PR 294) fixed the recorder:
charmbracelet/vhs-action's broken ffmpeg installer was replaced with apt plus
ttyd/vhs pinned by release-asset id. That worked. The recording step now succeeds on both event
paths. The workflow still fails on every push to main — observed
2026-07-28 at 20:56, 20:04 and 09:27, and it is the only failing workflow in the last 60 runs.
The failure moved one step later, to open a PR if the recording changed. The run
annotation gives it verbatim:
GitHub Actions is not permitted to create or approve pull requests.
Nothing in demo.yml is wrong, so nothing in demo.yml will fix it.
The job already declares contents: write and pull-requests: write,
which is exactly what peter-evans/create-pull-request documents as required. The
workflow uses no secrets. persist-credentials: false on the checkout is correct and
is the repo-wide convention across all 31 checkout steps — the action supplies its own git
auth and explicitly unsets any persisted credential. Every one of those is a plausible-looking
false lead. The actual blocker is a repository setting: Actions → General
→ Workflow permissions → "Allow GitHub Actions to create and
approve pull requests", which the API reports as
can_approve_pull_request_reviews: false. This is the same shape as
sbom.yml: a workflow that is correct in the file and inert in reality.
Deciding it is a real decision, not a formality. Turning that toggle on grants every
workflow in the repo the ability to open pull requests, which is a genuine widening of what a
compromised action can do. The alternative is to accept that this step cannot work and change
what the push path does — upload the GIF as an artifact on both paths, as the pull-request
path already does, and drop the PR-opening step.
A second, independent defect in the same file. Its concurrency group is
group: demo, not ref-scoped. Every other per-ref workflow scopes it —
sonarcloud-${{ github.ref }}, fuzz-${{ github.ref }},
eval-stats-${{ github.ref }}, eval-explain-${{ github.ref }}; only the
deliberately-singleton release and post-deploy are global. With
cancel-in-progress: true, a pull-request run and a main run therefore
cancel each other, which is where demo's cancelled runs come from. Fix is
group: demo-${{ github.ref }} regardless of what is decided about the toggle.
Worth reconciling while here: the tree disagrees with itself on the history.
demo-gif-workflow-has-never-succeeded says "3 runs, 3 failures", while
demo.yml and ci-failure-issue.yml both say six of six.
loop/* branches nobody deletesThe repository carries 77 branches, 70 of them loop/*. Classifying each by the
state of its pull request: 68 have a merged PR and are safe to delete, 1 has an open
PR (loop/dependabot-findings), and 1 has no PR on record. The default branch list is
almost entirely finished work.
The obvious measurement is wrong here, which is worth recording. Testing "is it merged?"
with compare(main...branch).ahead_by == 0 reports 0 of 70 as merged —
because this repo squash-merges, and a squash leaves the branch's original commits outside
main's history forever. Every merged branch looks permanently ahead. The state has to
come from the pull request, not from commit reachability.
The cause is a single repository setting: delete_branch_on_merge is false, so
nothing removes a head branch when its PR merges. No workflow prunes them either — none of
the 25 workflow files triggers on create/delete or touches
loop/*, and no scripts/ entry or Makefile target does branch cleanup.
The one branch-deletion automation in the tree is demo.yml's
delete-branch: true, which is create-pull-request tidying its own
bot/demo-gif branch and has nothing to do with loop/*. No documentation
asks anyone to clean up either: CONTRIBUTING.md, CLAUDE.md and
.claude/commands/roadmap-loop.md all describe branching and squash-merging, and none
of the three mentions deleting the branch afterwards.
Not a claim that branches are never deleted. The item files name 144 distinct
owner: loop/<topic> values against 70 surviving branches, so roughly 98 have
gone at some point — by hand, or by topic reuse. The accurate statement is narrower:
nothing in the repo causes cleanup, so merged branches accumulate until someone does it
manually.
Low impact and near-zero risk to fix — flip delete_branch_on_merge to true and
the backlog can be pruned by listing merged PR head refs. The reason it is worth doing at all is
that CLAUDE.md tells every agent to run git worktree list and inspect branch state
before touching a tree, and 68 stale entries make that check noisier for every concurrent loop.
Canonical style/boost.css ships tokens + primitives but
no nav, logo, or footer, so every page (the guide and
both roadmaps) redefines that chrome inline — and the Design Roadmap
currently renders its nav unstyled because of the gap.
Add .site-nav/.logo/footer
primitives to the one sheet so the chrome is defined once and can't drift.
pre-commit + pre-commit.ciA .pre-commit-config.yaml runs ruff, mypy, codespell and the
whitespace fixers before code ever reaches CI, and the free-for-OSS
pre-commit.ci auto-fixes and auto-updates hooks right on the
PR. The foundation the rest of this section plugs into — one config,
many checks, faster feedback.
import-linterEnforce the architecture boost already assumes: core/ must
never import commands/, and the CLI depends inward only.
import-linter turns that contract into a CI check, catching
the cross-layer imports that slowly erode a clean engine — a structural
smell no line-level linter can see.
codespellCheap, high-signal for a project this documentation-heavy: scan the
README, docs/*.html, docstrings and every user-facing CLI
string for misspellings. A typo in boost --help is a bug
users see on their first run.
interrogateGate CI below a docstring-coverage threshold so the engine stays
self-documenting as it grows. core/ is already well
commented; interrogate keeps it that way and flags the
new public function that shipped without a word of explanation.
pytest-benchmarkBenchmark the hot paths — a catalog scan over a large skills tree, a registry load — and fail the build when a change regresses them past a threshold. Catches the performance bugs a correctness suite waves through: the O(n²) that only bites at scale.
refurb + pyupgraderefurb's unique FURB checks flag dated idioms
the ruff families miss, and pyupgrade rewrites to the
cleanest form the >=3.9 floor allows. Keeps the codebase
reading like modern Python instead of accreting legacy patterns.
The self-contained diff-cover gate enforces patch coverage in CI; this is
its hosted counterpart — the same coverage.xml the gate already
reads, turned into PR diff-coverage comments, a file sunburst and a trend line
over time. Wired into the existing patch-coverage job (no second
test run) and deliberately non-blocking: codecov.yml marks
every status informational, because boost already enforces coverage
twice offline and a merge should never hinge on a third party's uptime. Inert
until a CODECOV_TOKEN secret exists — the step's if
reads the token through env (a secret cannot be referenced directly
in an if), so with no secret it skips rather than fails.
The lint job installed its tools unpinned
(pip install ruff mypy …), so every run pulled the latest
release. When ruff 0.16.0 shipped it began flagging rule
families this repo deliberately does not select
(UP/BLE/PLW — ~1100 findings of
exactly the pyupgrade-style churn the roadmap already declined),
turning the gate red on every open PR and on main with no
code change. A gate whose meaning can shift out from under you on an
upstream release is not a gate. Froze the eight lint tools to their
known-good versions in one requirements/lint-tools.txt
that both CI and the Makefile install from, so upgrades become a
deliberate, reviewable bump. (The broader runtime/build reproducibility
story is the uv.lock item; this is the lint-gate slice
that was actively on fire.)
Free for public repos, Harden-Runner watches (and can block) unexpected
network egress from CI runners — the runtime signal that catches a
compromised action exfiltrating a token. The dynamic complement to
zizmor's static workflow analysis, guarding the release
path from both sides.
Shipped. .github/required-checks.txt is now the source of truth and scripts/check_required_checks.py (in make lint, so in CI) fails when a required name stops matching a job that runs on pull_request. It also caught something the card missed: three check names were ambiguous — lint in ci/markdownlint/theme-lint, audit in lighthouse/pip-audit, analyze in codeql/sonarcloud. GitHub matches required checks by name, so none of those could be required unambiguously, and the colliding jobs are renamed. Original report follows. Branch protection was hand-configured in GitHub Settings and only described in prose in
CONTRIBUTING.md — there is no config-as-code (a repo ruleset export, Terraform, or a CI
check diffing required names against real job names) to catch drift. It has already drifted:
ci.yml's install-smoke job, and the pip-audit/
package-metadata workflows, are never listed as required, so a PR can merge to main
with any of them red.
The repo has SECURITY.md, CONTRIBUTING.md, and
dependabot.yml, but no .github/ISSUE_TEMPLATE/, no
.github/PULL_REQUEST_TEMPLATE.md, and no CODE_OF_CONDUCT.md. For a project
that explicitly expects parallel external contributors (per the worktree-coordination rules in
CONTRIBUTING.md), a bug-report template built around boost doctor/crash-log
output and a PR checklist would cut triage friction more cheaply than most engine work.
The behave suite added under tests/bdd/ (11 features, 47 scenarios) passes cleanly
today, but make bdd is invoked nowhere in .github/workflows/ — only
lint, test, smoke, and mutation run in
ci.yml. Unlike the eval/mutation gates, which degrade cleanly but still execute, this
suite only runs if a human remembers to run it locally, so a CLI change that breaks the step glue
(output-string matching, mocked shutil.which) can rot silently for any number of merges.
boost adapt — render a skill as another framework's agent sourceboost only installs skills as files for editor agents (Claude Code, Cursor, Windsurf); CrewAI / OpenAI-Agents-SDK apps can't consume them — an agent there is a value built in source. boost adapt <skill> --to crewai|agents-sdk renders one SKILL.md into each framework's native Agent(...) as a deterministic, zero-dependency string transform (core/adapters.py). Strings emit via json.dumps so any body (quotes, """, unicode) stays valid Python; 97.6% mutation kill. Docs: adapters.html.
boost adapt --to langgraph — third framework rendererExtend boost adapt (shipped in #146) with a LangGraph target. Unlike CrewAI / Agents-SDK, a LangGraph agent isn't one Agent(...) constructor — the skill body is the system prompt of a node bound into a StateGraph. Add a render_langgraph to core/adapters.py emitting a node factory (prebuilt create_react_agent, or a prompt-carrying node) + its FORMATS row, with byte-exact golden + compile() tests. Then add a langgraph leg to .github/workflows/adapter-conformance.yml (pip install langgraph, import the emitted file). Follows the pattern in adapters.html: one render_* fn + registry row + golden test + one matrix line.
boost adapt — multi-agent skills → crews/graphs, not one AgentToday boost adapt (#146, #163) projects a skill into a single framework Agent carrying its instructions/role/model — but a skill can declare tools and a whole subagent graph (e.g. rust-review = worker + dedup-judge + fp-judge). Adapt the richer structure: emit a CrewAI Crew (agents + tasks) or a LangGraph StateGraph when a skill's frontmatter/plugin dir declares subagents, and surface declared tools as stubs. Scope: detect subagent/tool declarations in SKILL.md/plugins/*/agents/*.md; add a crew/graph renderer path in core/adapters.py (single-Agent stays the default for flat skills); golden + compile() tests + a conformance leg. Turns "one skill → one agent" into "one skill → a runnable multi-agent workflow." See adapters.html.
boost run — search → adapt → a live agent doing the task, in one commandThe allure ceiling on boost adapt (#146, #163): it stops at source you assemble, and the adapted agent has a brain (prompt) but no hands (tools). Close both. boost run <skill> [target] = adapt → wire a default toolset (file read, ripgrep/shell, web) → execute on boost's model and stream the result. Opt-in like the conformance path (installs the framework + needs a key, so the zero-dep default holds). Two prerequisites ship with it: (1) tool-wiring — emit/attach the tools a skill needs (or declares) so the agent can act, not roleplay; (2) reuse the adapt renderer for the agent def. Pairs with [[framework-adapter-multi-agent]] so a skill like rust-review runs as a real crew. This is the screenshot: "one command → an expert agent audits your repo, on Claude, that you never wrote." Working preview: examples/boost-run-prototype.sh already does this by hand (adapt → wire a read_file tool → Runner.run on a file with planted bugs).
The boost MCP server returned no initialize instructions and described its tools by boost's nouns ("Search AI coding skills across the configured tap registries"), so an agent mid-task never mapped "I'll write a code-review workflow" onto "search boost first" — and reinvented the wheel. Add server-level instructions (the field MCP hosts load into the agent's context) framed by the agent's trigger — before you author a reusable skill / rule / subagent, call boost_search FIRST — and rewrite the six tool descriptions to lead with that trigger and a "don't reinvent one that already exists" frame. Follow-up: a self-bootstrapping boost-first meta-skill that boost mcp register / onboard offers to install.
boost install resolves a skill's requires: closureSkills already declare relationships in frontmatter (requires: /
conflicts:), and boost info / boost deps
show them — but boost install X only ever installed
exactly what you named, never what X depends on. That is the one headline
package-manager behavior boost was missing: installing a skill should pull
in the skills it needs, in the right order. Add a pure, mutation-tested
resolver in core/resolve.py — post-order DFS over the
requires: graph, cycle-safe, deduped, already-installed nodes
pruned — and wire it into cmd_install so a named skill installs
its transitive requires: closure (dependencies first), with a
--no-deps escape hatch. Declared conflicts: against
an installed or co-installed skill surface as an advisory warning, and a
requires: naming a skill in no tap is flagged, not fatal.
Surfaced by mining boost's own catalog: the densest skill cluster is
dependency management, which every package manager treats as table stakes.
.mcp.json on installMining boost's catalog surfaces a recurring shape: skills that only work paired
with a Model Context Protocol server (manage-mcp-servers,
mcp-integration, mcp-builder and dozens more). boost
already is an MCP server and registers itself, but a skill could not say
"I need server Y", so it installed cleanly and then failed in the agent for a
reason nothing surfaced. Now a skill declares mcp: github, playwright
in frontmatter and/or bundles a standard .mcp.json; on install boost
states the requirement and offers to register the servers that came with a
runnable spec, the same way boost mcp register wires boost itself.
boost info lists them beside capabilities:, and
--no-mcp opts out. Deliberately flat, not the nested
mcp: block first sketched: boost's frontmatter parser is a stdlib
YAML subset that does not fail loudly on a nested mapping — it hoists the inner
keys to top level and clobbers their siblings — so the declaration meets the
parser where it already works and full specs live in the sidecar that needs no
parser at all. The decision layer is core/mcpdecl.py, pure and
I/O-free like core/resolve.py.
roadmap.html goes stale on every rebase, so a card and a merge race redden the whole matrixThe data-driven roadmap solved the conflict problem for item files: two loops adding
two cards touch two different files and merge cleanly. But docs/roadmap.html is
still a single committed artifact generated from all of them, and
update-branch merges it textually.
So whenever another roadmap-carrying PR merges first, the rebase brings in the new item
.md cleanly and leaves the generated HTML behind. The board no longer matches
docs/roadmap/items/, and both build_roadmap.py --check (in lint) and
tests/unit/test_roadmap_fresh.py fail — on every test leg, for a PR that
changed no Python at all. It reads as a catastrophic failure and is really a stale generated
file.
This is not hypothetical: one PR hit it twice in a single afternoon, each time needing a
manual regenerate-and-push, and the cost scales with the number of concurrent loops — the
exact workload the item-file split was introduced to support. With strict: true
every merge rebases every open PR, so any PR carrying a card is guaranteed to hit it if it
sits behind one other roadmap PR.
Options, roughly in order of appeal: have CI regenerate and push the board on the PR branch
(fixes it silently, needs a bot token and care with the required checks); a custom merge
driver for the generated boards that reruns the generator instead of merging text; or stop
committing the HTML and build the site at deploy time (biggest change, and it removes the
artifact from review). At minimum, make the --check failure message say
"regenerate after rebasing" so the cause is obvious from the log.
make lint reports success when actionlint fails — and says it wasn't installedMakefile:64 guards the workflow linter like this:
@command -v actionlint >/dev/null 2>&1 && actionlint || echo "actionlint not on PATH — skipping (CI enforces it)"
Because || binds to the whole && chain, the fallback fires in
both failure modes — actionlint absent, and actionlint present and reporting
findings. Reproduced with a stub that exits 1:
actionlint: FAKE ERROR
actionlint not on PATH — skipping (CI enforces it)
make exit=0
So a real workflow-lint failure is swallowed, and the message asserts something false — that
the tool is not installed, when it is installed and just failed. make lint and
therefore make check can never fail on actionlint. CLAUDE.md calls
make check "the one gate that matters" and tells every agent to run it before calling
a change done; for this tool it returns green unconditionally, and the developer who reads the
output is told the opposite of what happened.
The local-versus-CI gap is wider than this one line. CI's required lint job runs
three workflow/security tools that make lint has no equivalent for at all:
actionlint (present but unfailable), zizmor and gitleaks (absent from the
Makefile entirely). No CI job invokes make lint, so the two lists are hand-maintained
duplicates with nothing checking they agree — which is how they drifted.
Minimal fix is to stop discarding the exit status:
@if command -v actionlint >/dev/null 2>&1; then actionlint; else echo "actionlint not on PATH — skipping (CI enforces it)"; fi
The stricter option is to install actionlint in make venv and let absence itself
fail, so make check cannot report success while skipping a required gate. The same
masking idiom appears at Makefile:143 (hyperfine) and Makefile:167
(node); neither is reached by check, so they are cosmetic by comparison, but they are
the same bug. Incidental drift found alongside: CLAUDE.md describes the lint recipe
as "18 commands" and it is 19.
The living roadmaps were unreachable unless you knew the URL:
design-roadmap.html was linked nowhere and the README had no
roadmap link at all. The Visual Guide nav now carries both boards (a
Design ↗ entry beside the existing Roadmap ↗),
and the README opens with a Roadmap section linking the code and design
boards plus the Visual Guide — so the two data-driven boards are one click away
and the work stays visible.
treosh/lighthouse-ci-action scores index.html
and roadmap.html on performance, accessibility,
best-practices and SEO on every deploy, and fails the build against
budgets. Turns the marketing surface's quality into four numbers that can't
silently regress as the Aurora theme evolves — four floors (a11y and
SEO 0.95, best-practices 0.90, the noisier performance 0.85) asserted
on the median of three runs, calibrated below the real scores with
margin the same way perf_gate sets its thresholds. And,
as every checker on this repo has, it found real defects on the
storefront: muted text at 3.9:1 contrast (below WCAG AA),
an h2 → h4 heading skip, and colour-only footer links —
all fixed, so both pages now score 100 on accessibility.
pa11y-ci / axe-coreWCAG 2.1 AA over every docs page, in two halves. scripts/a11y_check.py
is the always-on gate: pure stdlib, so it runs in the lint job beside the other
--check scripts with no Node, Chrome or network. It covers what the
markup alone decides — lang, alt, accessible names for
links and buttons, duplicate ids, heading order — plus the 1.4.3 contrast ratios
computed from the Aurora tokens for the ink/ground pairs the CSS actually puts
together (listing the real pairs, not the cross-product, so a pair that never
renders can't cry wolf). The axe-core sweep in tests/visual/ is the
other half: contrast as rendered over glass and gradient, ARIA validity
against the computed tree, landmarks, focusable-but-hidden elements — everything
that needs a live DOM and therefore can't be a cheap always-on gate. The audit
found one real failure, a skipped heading level on the design board, fixed by
promoting the section headings with their class-scoped selectors in lockstep so
the rendering is unchanged. Contrast came back clean: the muted tokens the card
suspected were already fixed (--text-3 had been raised from a 3.9:1
value), and this gate is what keeps them that way.
html-validateThe guide and roadmap are hand-authored HTML whose tags are balanced by
eye today. A CI html-validate pass catches unclosed
elements, duplicate ids and invalid nesting before they
ship a subtly broken layout — the automated version of the manual
tag-count sanity check.
lycheeFast, free link checker over the README and docs/*.html:
verifies external URLs (GitHub, PyPI), asset paths
(../style/boost.css) and in-page anchors
(#next, #health) still resolve. A dead link in
the landing page is the first broken promise a visitor sees. Red on
its first run: eight were already broken — four ../adapters.html
at the wrong depth, two dangling anchors, one link to an item file as
though it were a page, and a README pointing at
docs/overview.html, renamed to index.html
and never updated.
valevale enforces a consistent voice across the README and docs
with a small boost vocabulary — canonical spelling of skill,
tap, registry; no Boost/boost
drift; flagged weasel words. Keeps a docs-heavy project reading like one
author wrote it.
markdownlint-cli2Lint every Markdown surface — README, CLAUDE.md,
DEBUGGING.md, rag-architecture.md — for
heading order, list style and fenced-code hygiene. Cheap, deterministic,
and it keeps the contributor-facing docs as tidy as the code the gates
already enforce.
stylelint + eslintThe shared style/boost.css and boost.js theme
the guide, roadmap and demo together, so a bug there recolours or breaks
everything at once. stylelint guards the CSS tokens and
eslint the reveal/interaction JS — the one place a small
mistake has system-wide blast radius.
Every other docs gate runs against the working tree — html-validate and
the a11y checker parse the files, the visual sweep loads them over
file://. All of them can be green while the deployed site is
broken, because deployment is where the paths change: Pages serves this repo
under /boost/, so a link that resolves on disk can 404 in production.
Nothing was watching that. Now, triggered by the Pages deployment finishing (so it
checks what was actually published rather than guessing at propagation),
scripts/post_deploy_smoke.py asserts every page answers 200 and every
local asset and internal link resolves — pure stdlib over HTTP, no browser — and
console_check.mjs loads each live page in headless Chrome for the
half only a browser sees: uncaught JS, console.error, and runtime
request failures. Off-site links are deliberately excluded: a flaky third party
must never redden boost's deploy, and links.yml already owns those.
boost has no browsable command reference — help lives only in
--help output and scattered README prose, so there's no
versioned, searchable home for what each command does. Build a proper
docs site in the shape of
learning-python.readthedocs.io:
a left-nav tree of commands, per-command pages (synopsis, flags,
examples, exit codes), and full-text search. Generate the command
pages from the CLI's own definitions so the docs can't drift from the
code, and publish alongside the existing Pages site under the Aurora theme.
docs/*.html page from the main pageMaking the roadmaps discoverable
put both boards in the Visual Guide nav, but the same gap remains for
the rest of docs/*.html: mcp-hub.html is
reachable only if you already know the URL — nothing on
index.html links to it. Add a nav entry for every shipped
doc page (today just the MCP Hub, beside Roadmap ↗ /
Design ↗) so no page is an orphan, and treat "a new
docs/*.html means a new nav link" as the standing rule —
ideally checked in CI so a future page can't ship unlinked.
mcp-hub.html)On the Fig 2 RAG-pipeline diagram, the read_body → chunk node
overflows its rounded box: the third caption line
(1000 chars · 150 overlap · ≤40…) runs past the node's
right border and is clipped where the pink embed arrow begins.
Give the node room so all three lines sit inside the bubble — widen its
min-width, wrap or shrink the caption, or trim the label text — and
re-check the other nodes at the same breakpoint so none clip.
Promoting the chrome
into style/boost.css gave the docsite .site-nav and
footer primitives, but nothing adopted them: three pages still had
no nav at all (eval, adapters,
mcp-hub), commands.html had neither nav nor footer,
and design-roadmap.html had no footer. Alongside that,
stale content — the Visual Guide advertised “73 commands”
in three places while cli.py's COMMANDS held 78,
its embedded array missing adapt, run,
trust, hooks and bmad — and
unprofessional footers: four pages signed off with “Styled with the
shared Aurora design system” over a raw ../style/boost.css
link, the roadmap credited “the boost quality loop”, and the MCP Hub
listed three internal PR numbers.
Shipped: every page now carries the same nav and the same footer —
brand line, install line, the page index, GitHub/PyPI/Portfolio and the licence,
and nothing else. The command inventory is regenerated from COMMANDS.
On phones the link row drops to its own full-width line and scrolls with
40 px tap targets (WCAG 2.5.5) and a faded trailing edge, replacing
both roadmap.html's display:none below 560 px
— which deleted the nav outright — and the column stack that turned
fourteen links into six rows of sticky header. Two latent bugs fell out:
roadmap.html's hero was a second bare <header>, so
it silently inherited the shared sheet's sticky chrome, and generated cards had
no id, so no item could link another. Locked by
tests/unit/test_docsite_chrome.py.
README shipped a single demo.gif and the guide described
78 commands entirely in prose — nothing showed what a boost session actually
looks like. Shipped: docs/carousel.html, a keyboard-
and swipe-navigable carousel of 8 terminal recordings, one flagship command
per command group — install, search, explain,
tap, distill, doctor, hooks and
pulse. Every recording is scripted as a
VHS tape checked in beside the
GIF it produces, so a stale frame is one make carousel away from being
regenerated rather than re-recorded by hand. Each GIF carries descriptive alt text
covering what the terminal actually does, so the page is usable without images.
The counts on the page are hand-typed no longer: the per-slide
“N commands in this group” notes and the headline total are asserted
against cli.py's COMMANDS by
tests/unit/test_docsite_chrome.py, the same guard that covers the
guide — the page arrived claiming 76 commands with two group counts already
adrift. Built on the shared chrome from
the docsite audit, so it carries the
same nav and footer as every other page.
demo.yml has failed every run since it landed — vhs-action cannot install ffmpegdemo.yml re-records docs/demo.gif with
charmbracelet/vhs-action and opens a PR when the recording changes. It exists
because "a generated artifact whose regeneration is manual is one nobody regenerates".
All 3 of its 3 runs since 2026-07-26 failed at the recording step with the annotation
Failed to install ffmpeg, so the follow-on "open a PR" step was skipped every
time and it has never produced a single PR.
The net effect is worse than not having it: demo.gif is exactly as stale as
before, but the repo now looks covered. Same class as the LangGraph conformance leg — a leg
that has never passed since the day it was added. A secondary annotation on the same runs
flags that vhs-action v2.1.0 targets Node 20, now deprecated and force-run on Node 24.
Options: install ffmpeg explicitly before the action runs, pin a runner image that carries
it, switch to a maintained recorder, or delete the workflow and regenerate the GIF by hand.
Whichever is chosen, the workflow should fail loudly rather than sit red indefinitely — a
scheduled job nobody reads is how this went unnoticed for its whole lifetime.
install --upgrade — which no-opsA user on the README-recommended pipx install ran
pipx install boost-skill-cli --upgrade and stayed on 1.0.203 while PyPI was at
1.0.226 — 23 releases behind. Nothing was broken. pipx's install --upgrade only
acts when the installed version does not satisfy the supplied spec, and the spec was
the bare name boost-skill-cli, which every version satisfies. It printed
boost-skill-cli 1.0.203 already satisfies boost-skill-cli and did nothing.
Confirmed at the source (pipx 1.16.2): package_spec_satisfied() returns True for
a bare name at any installed version, so the upgrade branch is never reached;
a pinned spec returns False. pipx upgrade boost-skill-cli is the correct verb,
verified end to end (1.0.203 → 1.0.227).
The root cause is documentation, not pipx. Every install surface — README,
docs/index.html, the site root — stopped at the install command; the word
"upgrade" appeared nowhere in any of them. With no correct command to copy, guessing
install --upgrade is the reasonable move.
Fixed by adding the upgrade line inside the code block users already copy from
(docs/index.html's Copy button takes the whole block, so the right command
travels with the paste). The deeper bug — boost self-update hard-failing on
pip/pipx installs and telling a PyPI user to clone the repo — is separate and still open at
self-update-broken-for-pip-pipx-installs.
The test matrix runs macOS and Ubuntu only, so
windows-latest path separators, case-sensitivity and
console-encoding bugs ship uncaught to any Windows user. Adding one
matrix leg closes the biggest untested surface for a tool that promises
to work "everywhere your agent does".
pipxEvery gate runs against an editable pip install -e ., which
hides missing package-data, a broken entry point or an undeclared
dependency. A job that pip installs the built wheel into a
fresh venv (and pipx installs it) then runs
boost --version proves the artifact a user actually gets
works.
uv --resolution lowest-directCI always installs the newest compatible dependencies, so the
lower bounds declared in pyproject.toml are never actually
exercised. Resolving and testing against the minimum versions catches
the "works here, breaks on the floor we advertise" class of bug.
An allow-failure matrix leg on Python pre-releases and the free-threaded (no-GIL) build gives early warning of breakage before users on new interpreters hit it. Cheap insurance for a project that already targets 3.9 → 3.14 and can't afford a surprise on release day.
-X importtimeFor a CLI, cold-start latency is the UX — every command pays it.
A gate on python -X importtime boost catches the accidental
top-level import of a heavy module (the optional [rag] stack
is the obvious trap) and keeps common commands feeling instant.
twine check + friendsBefore every publish, twine check confirms the long
description renders on PyPI, check-wheel-contents catches
stray or missing files, and pyroma scores the metadata
completeness. Stops a broken PyPI page or an empty wheel from reaching
users — the release path currently trusts the build blindly.
requirements/*.txtEvery dev/CI tool was resolved at install time, so two runs of the same gate could
install different bytes — and only requirements/lint-tools.txt pinned
anything at all (added after ruff 0.16.0 reddened the gate with no code change).
Now every tool comes from a generated, hash-pinned
requirements/*.txt: exact versions plus a sha256 for every
artifact in the transitive closure, which pip enforces on install, so a yanked or
tampered dependency fails loudly instead of silently changing a build.
scripts/lock_toolchain.py regenerates from the .in
declarations and --check gates drift in make lint; a new
Dependabot pip entry keeps the locks from rotting. One file per
consumer, because pip enforces hashes across a whole resolution — and
test-tools resolves universally, so one file covers the
3 OS × 3 Python matrix without dragging 3.12/3.14 back to the 3.9 floor's
versions. The opt-in [eval] extra stays out on purpose: locking its
deliberately-old langchain stack into a scannable file would re-expose the pin
osv-scanner.yml is PR-diff-scoped to avoid.
noxA noxfile.py makes the full lint / test / smoke / mutation
gate reproducible across 3.9 → 3.14 in isolated environments,
locally and in CI alike — so "green on my machine" and "green in CI"
finally mean the same thing, and a contributor can run the exact gate
before pushing.
boost mcp launch against macOS Obj-C fork abortsOn macOS, a host that spawns boost mcp --stdio via fork()
can abort on the child side pre-exec when Obj-C is touched
post-fork (CFPreferences / _scproxy proxy lookup) — the
classic "crashed on child side of fork pre-exec"
SIGABRT. Not boost's own code (it aborts before boost
runs), but boost can make its integration robust: when registering the
MCP server (boost mcp register / the emitted client
config), set OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES (and
no_proxy=*) in the launch environment so the host's fork
into boost can't trip the Obj-C fork-safety abort. Document it for
hosts that launch boost directly.
The launch-env fix (#119) only
protects hosts that register boost through boost, which
injects no_proxy=*. A host that fork()s into
boost mcp --stdio from a stale or hand-written config —
or any stray default-opener call inside the process — is still one
getproxies() away from the macOS _scproxy
Obj-C SIGABRT. Close the gap from inside: at the top of
main(), seed no_proxy for the current
process (only when no *_proxy env is already set, so a
real proxy is never clobbered) so the stdlib default
getproxies() short-circuits on
getproxies_environment() and never consults
SystemConfiguration. Belt-and-suspenders atop the
nethttp opener (#115).
self-update is non-functional for pip/pipx installsboost self-update only works when boost runs from a git checkout: paths.repo_root()
resolves inside site-packages for a pip/pipx install, gitutil.is_repo() is
False, and cmd_self_update raises "boost is not running from a git checkout" with a hint
to clone the repo — telling a normal PyPI user to abandon their install method. Detect the install
method and shell out to pip install --upgrade boost-skill-cli /
pipx upgrade boost-skill-cli, falling back to the git-pull path only for source
checkouts. Shipped as core/selfupdate.py: detection reads evidence on disk —
.git, then pipx_metadata.json or uv-receipt.toml in
sys.prefix, then installed package metadata — and a --dry-run flag
prints the exact command without running it. Two details carry the weight: the pip branch
invokes sys.executable -m pip, never a bare pip, because the pip
first on PATH can belong to another interpreter and would upgrade a different copy while
reporting success; and an install nothing has a record of reports that plainly instead of
guessing pip. The new version is read from a freshly spawned boost --version,
since the running process imported its own version before the upgrade — if that probe says
nothing, neither does boost.
boost mcp register shelled out to
claude mcp add --scope user -e OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES -e
no_proxy=* boost -- … — the -e flags before the
server name. But claude's -e is variadic, so it
swallowed boost as a third env var and aborted with
Invalid environment variable format: boost, making the one-command
install path fail outright on macOS. Reordered to
claude mcp add boost --scope user -e … -- … (the
add <name> [options] -- <command> form), so the name is
consumed as the positional before -e can grab it. A regression
test pins name < first -e so a future reorder can't silently
re-break it.
The highest-signal gap: boost installs Markdown an agent then executes,
and nothing inspects that content. Scan skills for injection patterns —
"ignore previous instructions", data-exfiltration prompts,
embedded curl … | sh — with free rule engines
(semgrep custom rules, garak/llm-guard
patterns) at tap and install time.
boost verifyThe lockfile already records a sha256 per install via
util.sha256_dir, but nothing ever re-checks it. A
verify command (and a doctor gate) that
re-hashes installed skills and flags drift turns that stored digest into
actual tamper-detection — cheap, and built on machinery that already
exists.
A tap records its commit today but offers no cryptographic
proof the content came from its claimed publisher, so a hijacked mirror
can serve altered skills undetected. Verify a signature (Sigstore
keyless or minisign) before trusting a tap — free, and the
standard the wider ecosystem is converging on.
boost explainToday the ragas faithfulness check (Tier 2c, scripts/eval_explain.py)
is eval-time only — an offline monitor that flags when generated
explanations drift from their SKILL.md, but never intercepts a live
response. Promote it to a runtime guardrail: score the generated summary
against the source and, when it falls below a faithfulness threshold, fall back
to the deterministic extractive summary cmd_explain already ships
for the no-AI case (description + outline + key rules) — or append a
low-confidence caveat. Turns "a human notices the drift next week" into "the
tool refuses to show an ungrounded explanation," closing the gap between
detecting hallucinations and preventing them.
The classic package-manager attack: a skill named one edit-distance from
a popular one, or a familiar name that quietly resolves to an unexpected
owner/repo. Flag near-duplicate names and owner mismatches
at search and install time so a user can't fat-finger their way into a
malicious skill.
Shipped in #132. boost update no longer overwrites an
installed skill in place unseen. A new pure core module
core/updatediff.py diffs the installed tree against the incoming
source (diff_tree) and flags when the change adds
executable-looking instructions — shell commands, pipe-to-shell,
shebangs (touches_executable). When it does,
cmd_update prints the unified diff and requires confirmation before
applying, so a poisoned update is visible instead of silent; routine
version bumps and prose edits still apply quietly. Fully unit-tested and
mutation-covered, plus functional coverage of the confirm / decline / no-gate
paths.
Workspace scope (#212)
guarded the delete side — uninstall --local re-derives every
recorded path and refuses one that resolves outside the repo — but not
the write side. A project install builds its destination from
agent-dir names committed in the repo
(.claude/skills and its per-agent siblings), and a hostile
clone can ship one of those as a symlink pointing outside the
tree — at ~/.ssh, say. Cloning that repo and
running boost install <skill> --local then writes on the
far side: confirmed by reproduction, a skill directory landed in a
sibling of the repo instead of inside it. The squatter check does not
stop it (a brand-new leaf name like authorized_keys
collides with nothing, and --force waives it anyway).
scopes.ensure_in_base is the write-side mirror of the
delete guard: it re-derives containment through symlinks for every
target up front, so one escaping directory aborts the whole install
before anything is written. Covers all three project-scope kinds —
skill, rule and workflow.
Point the round-2 secret scanners (gitleaks/trufflehog)
at third-party skill content, not just boost's own repo — catching
a skill that ships embedded credentials or, worse, one whose prompt
coaxes the agent into harvesting the user's. Same free tools, a different
and higher-stakes target.
Promoted the recorded sha256 from a note to a rule. The check
moved into core/integrity.py (where the mutation gate
covers it), and every command that serves a skill's content routes
through one resolver that now refuses a tree whose bytes have drifted
from the lock — a tamper tripwire at the point of use, since boost
can't police what the agent loads but can refuse to hand you
a skill that no longer matches what you reviewed. Opt-in
(config security.enforce_digest, default off) so it never
surprises an existing setup; verify reports drift either
way. Commit pinning rides alongside: boost pin <skill>
--commit freezes the exact source commit, and
verify flags it if the recorded commit ever moves off the
pin.
Extend the existing policy.py so a skill declares the
capabilities it expects — network, shell, file scope — and the user's
policy allows or denies them. Turns install-time governance into
least-privilege for the instructions an agent is about to run, the
natural next step for a tool that already blocks installs by policy.
install_from_path bypasses pin & policy checksShipped. Confirmed by running it: install_from_path enforced no gate, so import, create --install, migrate --from-skills-cli and distill/infer/absorb --install each walked past a blocklist, pin_only, max_skills and denied_capabilities. Worse than filed: the lock write hardcoded "pinned": False, so a re-import did not merely skip the pin check — it silently cleared an existing pin. Fixed in core/store.py with a force flag for the legitimate reinstall path; force covers the pin, never policy. Three corrections to this card: boost rename does not exist (misread of the rename= parameter behind import --name); evolve does not route through this function (it writes the store in place — a separate pin bypass, still open); and allowed_taps would not have refused any of these, since policy.py exempts local explicitly. Adopting install()'s "already installed" refusal was rejected — this is the re-import path, so it would break boost reinstall. The multi-item callers (import --all, reinstall --all, migrate) now warn per item and keep going instead of aborting the run on the first refusal, and _install_generated catches the refusal so a paid LLM generation is written to disk rather than deleted with the tempdir.
catalog._make_entry only slugifies a catalog name when it contains a space, so a tap's
rule/workflow frontmatter carrying name: ../../../../.ssh/authorized_keys sails straight
through into rule_target/workflow_target, which build the destination as
root / "rules" / (name + ext) with zero traversal guard — unlike
skill_store_dir()'s [A-Za-z0-9._-]+ regex. A malicious tap can write an
arbitrary file outside .cursor/rules/ or .claude/, and it is worse under
--scope project since base is the victim's own repo. Apply the same
name-validation regex used by skill_store_dir() before any rule/workflow install path
is built.
boost audit --skills — a trust/staleness report for installed skillsboost already computes every individual trust signal, each in its own command:
verify checks lock-file integrity, outdated compares an
installed skill against its tap, trust reports tap-level signing
provenance, deps shows the requires:/conflicts:
graph. What nothing answered is the aggregate question — of the skills I
actually run, which ones should I stop trusting? boost audit --skills
gathers all of it into one report: every installed skill that is unsigned, signed
by an untrusted key, signed but failing verification, sitting on a tap nobody has
synced in a month, behind its tap, or conflicting with another installed skill.
The decision layer is core/trustaudit.py — pure and I/O-free like
core/staleness.py, so every branch is unit-tested and reachable by
the mutation gate. Only a malformed signature is HIGH; an unsigned tap is the norm
for most of the catalog today and stays LOW, so the command never cries wolf on an
ordinary install.
sbom.yml has never run — it waits for an event GITHUB_TOKEN cannot emitsbom.yml triggers on release: types: [published] and promises
"a CycloneDX SBOM for every release, attached to the GitHub Release as an asset".
It has 0 runs. Releases here are created by release-drafter inside
publish.yml authenticated with GITHUB_TOKEN, and events created
with GITHUB_TOKEN do not trigger workflows. publish.yml's own
header states that rule verbatim — "we never rely on the release event to trigger a second
workflow" — and sbom.yml is exactly that pattern.
So the if: github.event_name == 'release' guard on its upload step is dead code,
workflow_dispatch is the only reachable path and has never been used, and every
one of the last ten releases carries assets: []. Zero SBOMs have been produced
since the file landed on 2026-07-22. Same shape as the LangGraph conformance leg: wired up,
looks covered, never executed once.
By the time it was fixed the count was 253 releases, 0 runs, 0 SBOM assets.
Fixed with a third option neither branch of that decision considered:
workflow_run. The GITHUB_TOKEN restriction applies to
events a token creates; workflow_run is documented as exempt and fires
on the upstream run completing regardless of what triggered it — which is
already how ci → release works in this repo. Chaining
release → sbom makes this the third link, inside GitHub's documented
three-level workflow_run limit. So no PAT is needed, and the SBOM logic stays
out of publish.yml.
That separation turned out to matter more than "keeps the logic in its own file":
publish.yml's job holds PyPI Trusted-Publishing OIDC credentials, and generating
the SBOM means installing a third-party build plugin (cyclonedx-bom). Inlining it
would have put that plugin inside the one job that can publish to PyPI. Kept separate, it
holds only contents: write.
Two details the trigger change forced. The workflow now resolves its tag from the release
commit (git tag --points-at) rather than "newest release" — releases here
land minutes apart, so gh release view would race and SBOM the next
version; and two tags on one commit is real (v1.0.248 and v1.0.249
both point at c750651), so it takes the highest. And per the original note, a
final step re-reads the release and fails unless the asset is actually attached — a
silent upload no-op is indistinguishable from success, which is the same class of quiet
nothing that produced this bug.
main ruleset is inert — its ref pattern is refs/heads/"main", quotes includedThe repo has an active ruleset named main (id 19130332) carrying eight
rules: deletion, non_fast_forward, pull_request,
required_status_checks (9 contexts), code_quality,
code_coverage (minimum 80), code_scanning and
required_deployments. Its ref condition is:
"include": ["refs/heads/\"main\""] — with literal quote characters, so it
matches a branch named "main", not main.
GitHub's authoritative endpoint settles it: /rules/branches/main returns
0 rules, while /rules/branches/%22main%22 returns all 8. Everything in
that ruleset is unenforced. Only the legacy branch protection is actually gating
main, and it requires a different, shorter list.
Do not just fix the quotes. The ruleset includes
required_deployments: ["github-pages", "pypi"]. Correcting the ref pattern
activates that rule, and no pull request deploys to the pypi environment —
publish.yml runs post-merge on workflow_run. So the one-character
fix converts an inert ruleset into a hard deadlock on every PR, for a reason that has
nothing to do with status checks. Its check list is also stale relative to
.github/required-checks.txt (missing the three Windows legs,
install-smoke, patch-coverage, codeql-analyze) and it
pins CodeQL from the code-scanning app rather than the workflow job.
The safe move is one edit that does all of it: fix the pattern, drop
required_deployments, and reconcile the contexts with the checked-in list — or
delete the ruleset outright and keep the legacy protection as the single mechanism. Two
overlapping systems is how this stayed invisible.
Shipped — repaired in a single write, so it never existed in the deadlocking state:
pattern corrected to refs/heads/main, required_deployments removed,
and the status-check list replaced with the 17 contexts from
.github/required-checks.txt (dropping the code-scanning CodeQL
entry in favour of the codeql-analyze workflow job the checked-in list names).
Verified after: /rules/branches/main returns 7 rules, and
/rules/branches/%22main%22 returns 0. Confirmed against all three open
pull requests beforehand that every one of the 17 contexts actually reports — including a PR
touching only .github/ — so the tightened list cannot deadlock. One API wrinkle
worth recording: GET returns code_coverage.max_coverage_drop: null
but PUT rejects it ("data matches no possible input"), so round-tripping a
ruleset requires stripping null-valued parameters.
branches: filters head_branch, not the eventpublish.yml fires on workflow_run of ci with
branches: [main]. That filter matches the triggering run's head_branch —
not the event type, and not the repository. ci.yml also runs on
pull_request, so a pull request opened from a branch named main
produced a ci run whose head_branch was main and satisfied the filter. The job
gate checked only workflow_run.conclusion == 'success', so a green run on that
path fired the release job with contents: write and PyPI Trusted-Publishing
OIDC — cutting a tag, a GitHub Release and a PyPI upload. There was no second gate: the
pypi environment has no protection rules and no deployment branch policy.
Never code execution — the checkout pins ref: main, so what ships is always
main's code. The exposure was an unreviewed release triggerable from outside the repo.
Measured rather than assumed: ci-failure-issue.yml draws from the same
workflow_run source with no branches filter and has 679 runs
against ci's 254 pushes, proving workflow_run fires for PR-triggered runs
too; only the head_branch filter kept publish.yml near the push count.
Fixed by requiring the triggering run to be a push whose
head_repository is this repo. No regression: all 254 ci runs with
head_branch=main are pushes from jonnyeclectic/boost, and the gate
still admits push-to-main and workflow_dispatch while rejecting fork PRs,
same-repo PRs and red CI.
code_scanning ruleset rule can go back on — but only scoped to CodeQLRepairing the main ruleset dropped its code_scanning rule in favour of
codeql-analyze as a required status check (see
main-ruleset-ref-pattern-has-literal-quotes). That was the safe call at the time, and
it left an open question: can the rule come back? Yes — and the thing that would break it
is not what it looks like.
Nothing is blocking it. Verified without touching the live ruleset: POST a throwaway
ruleset with enforcement: disabled and a ref pattern matching no branch, carrying
code_scanning for tool CodeQL → HTTP 201, rule stored intact.
Delete it, then diff the live ruleset against a backup taken beforehand → no drift. That
disabled-and-non-matching probe is the general way to test a ruleset change without risking
anyone's in-flight pull request.
The stale analyses are a red herring. 693 analyses still carry the old
codeql.yml:analyze category against 122 on the current codeql-analyze,
which looks like it should matter and does not: the rule keys on tool name, not category.
Deleting them is irreversible, destroys alert history, and fixes nothing. Don't.
The actual hazard is the tool list. Three tools file into the same code-scanning inbox.
CodeQL has 0 open alerts (9 dismissed, 17 fixed). Scorecard has 7 open, every one
severity=error — and they are posture metrics that must not be dismissed,
so they will stay open. Enabling the rule with Scorecard in its tool list therefore blocks
every merge in the repository, immediately. The GitHub UI offers every tool that has
reported analyses, so Scorecard sits right next to CodeQL in the picker. That is almost certainly
why this has been assumed to be blocked.
Scoped to CodeQL alone it is safe and adds real protection: codeql.yml carries no
path filters and runs on push, pull_request and
merge_group, so it always reports and cannot produce the never-reports deadlock this
repo has hit twice. And it catches something the status check does not —
codeql-analyze passes when the job succeeds, including when it succeeds
having found error-severity alerts.