# matrx-ai test-health — RESOLVED 2026-07-07 (record + residual)

> **Status: the ~30 nondeterministic `matrx-ai` failures are FIXED.**
> `uv run pytest packages/matrx-ai -q` → **2581 passed, 12 skipped, 3 xfailed**
> on clean runs. **Zero were product-code bugs** — one test-infrastructure root
> cause (an import-time bootstrap side effect) plus a body of stale-assertion test
> rot. This doc records what was fixed and the one minor residual, so the next
> agent doesn't re-investigate from scratch. **None of this touched the
> Agent-Handoff / Value-Store / wire-swap work** (that suite is green independently).

## Root cause (the one that mattered) — FIXED

`matrx_ai/providers/generic_openai/test_huggingface.py` is a **manual smoke
script**, not a unit test, and it did `from initialize_testing import
get_test_resources` **at module-import time**. `initialize_testing.py` runs the
**entire aidream host bootstrap** (`matrx_ai.configure(...)` + tool-registry
population) as a process-global side effect. pytest collected the file (named
`test_*`, not excluded), so every test collected after it saw polluted singletons
— the vfs `_DURABLE_INSTALLED` flag, the capability `_REGISTRY`, and the
`ToolRegistry` — which is why the failure SET shifted run-to-run (capabilities,
injection-catalog, structured-input, vfs, prompt-caching all took turns failing).

**Fix (applied):** added the three manual scripts to `collect_ignore` in
`packages/matrx-ai/conftest.py` (which already excluded three others for exactly
this reason): `providers/generic_openai/test_huggingface.py`,
`agents/tests/test_categorization.py`, `agents/tests/new_agent_test.py`. These are
run by hand (`python <file>`) against a live env; they are not unit tests (they
also take positional args pytest reads as missing fixtures, and need a live
AppContext/DB/LLM). This one change eliminated ~22 pollution/collection failures.

## Stale-assertion test rot — FIXED (product code was correct; tests lagged behavior)

Each updated to assert the current, intentional, documented behavior:

- `capabilities/tests/test_capabilities.py::test_sandbox_fs_is_registered_after_import`
  — asserted `enabled_tools == ()`; the bundle now statically enables 10
  fs_*/shell_*/git_ingest tools (`built_in.py::_SANDBOX_FS_TOOLS`). Now asserts them.
- `capabilities/tests/test_browser_dom.py::test_capability_metadata` — asserted
  `requires_auth is True`; product is `False` (guest mode, 2026-05-18). Fixed.
- `processing/blocks/tests/test_parsers/test_various_parsers.py` — transcript /
  recipe / table `test_basic_*` supplied COMPLETE blocks but called the parser
  without `is_final=True`; the parsers deliberately withhold the last (possibly
  mid-write) item during streaming. Now pass `is_final=True`.
- `processing/blocks/tests/test_parsers/test_flashcard_parser.py` — referenced the
  removed `partial_card` field; the parser now emits one growing `cards` list
  (in-progress card appended with `back=None`; flushed on `is_final=True`). Rewrote
  the 3 tests against `cards` + `is_complete`, verified against actual parser output.
- `matrx_ai/tests/test_llm_params_drift.py` — `skill_injected_tool_ids` is written
  server-side only (`aidream/services/tooling/skill_merge.py`), part of the tool
  envelope, not a client LLM param. Added to the test's `UNIFIED_CONFIG_INTERNAL_FIELDS`
  exclusion set (same category as `tools` / `supports_tools`).

## Residual (minor — optional follow-up)

- **One rare cross-test flake remains.** In ~1 of 3 full runs, a single test flaked
  (observed: `tests/test_prompt_caching.py::test_system_block_carries_cache_control`);
  it passes in isolation and on repeat full runs. A smaller cross-test state leak
  survives the huggingface fix. To chase: run the full suite several times capturing
  the FAILED name each time, then bisect the polluter for whichever test flakes (grep
  for module-level `configure()` / registry mutation / global assignment in the
  `test_*` files that run before it). A package-scope autouse snapshot/restore fixture
  for the capability `_REGISTRY` + vfs `_DURABLE_INSTALLED` + `ToolRegistry` would be
  the belt-and-suspenders second layer once the specific leaker is contained. (I did
  NOT add a bare capability-only restore fixture — I tried it and it doesn't help,
  because the dominant leak was import-time, now removed.)

## One product observation (NOT a test failure — low priority)

`tools/implementations/browser_discovery.py:~345` calls
`LoadBrowserToolsArgs.model_validate(args)` which RAISES `ValidationError` on a
bad/missing `category` rather than returning `ToolResult(success=False, ...)`. The
`ToolExecutor` wraps it so runtime is safe, but a direct caller gets an exception
instead of the typed tool-failure contract. Worth aligning to the contract if
touched. (Surfaced by the categorization sweep.)
