Phase 1 Test Review - MaybeAI Sheet CLI
LIVE CHECK FOUND 1 ISSUE
Phase 1 semantic command tests pass locally, and a real endpoint probe against https://play-be.omnimcp.ai completed. Live upsert by worksheet name failed; retry by gid passed.
62/62 PASS
python -m unittest discover -s tests -v passed in 1.664s.
REAL API PROBE RUN
Live requests were sent to https://play-be.omnimcp.ai. Verification workbook: MaybeAI Sheet.
Scope Reviewed
Reviewed the current dirty worktree, focusing on the requested files: docs/superpowers/plans/2026-07-01-mbs-agent-friendly-command-tree.md, docs/agent-friendly-command-tree.md, src/maybeai_sheet/commands/*.py, pyproject.toml, README.md, and tests/test_cli.py.
The worktree already had source, test, and documentation changes before this report. This review did not modify source code or tests.
Test Commands Run
| Command | Result | Notes |
|---|---|---|
python -m unittest discover -s tests -v |
PASS | Ran 62 tests in 1.664s. All CLI, formatter, resolver, and update tests passed. |
CliRunner help inspection for range, workbook, worksheet, table, formula |
PASS | Confirmed range help lists only read, write, and clear; no semantic range read-many. |
rg source/test inspection for play-be.omnimcp.ai, httpx.Client, post_json, upload_file |
PASS | Found real HTTP implementation in MaybeAIClient, but tests patch the HTTP boundary or command client methods. |
uv run mbs --base-url https://play-be.omnimcp.ai ... |
15 PASS / 1 FAIL | Real endpoint probe created verification workbook and exercised Phase 1 semantic commands through the installed mbs entry point. |
Live Endpoint Verification
本次补充验证使用真实 MAYBEAI_API_TOKEN 环境变量，但报告未记录 token 或 Authorization header。所有命令显式使用 --base-url https://play-be.omnimcp.ai，并通过 uv run mbs 发起真实请求。
| Run ID | phase1-live-20260701-122533 |
|---|---|
| API base | https://play-be.omnimcp.ai |
| Verification workbook | https://www.maybe.ai/docs/spreadsheets/d/6a4496beef4212b56a15f4a8?gid=0 |
| Document ID | 6a4496beef4212b56a15f4a8 |
| Worksheets | Data gid 0; Model gid 1 |
| Result | PASS=15 / FAIL=1 / SKIP=0 |
| Command | Status | Endpoint | Live evidence |
|---|---|---|---|
workbook create | PASS | /api/v1/excel/write_new_sheet | Created the verification workbook linked above. |
workbook manifest | PASS | /api/v1/excel/workbook_profile | Returned workbook profile for document 6a4496beef4212b56a15f4a8. |
workbook capabilities | PASS | /api/v1/excel/workbook_profile | Returned capability/profile response through the same routed endpoint. |
worksheet list | PASS | /api/v1/excel/list_worksheets | Returned Data gid 0 and Model gid 1. |
worksheet create | PASS | /api/v1/excel/write_new_worksheet | Created worksheet Model. |
range read | PASS | /api/v1/excel/read_sheet | Read bounded range Data!A1:D5. |
range write --verify | PASS | /api/v1/excel/update_range | Wrote Model!A1:E2 and read it back. |
table headers | PASS | /api/v1/excel/read_headers | Read header row from Model. |
table append-rows --verify | PASS | /api/v1/excel/append_rows | Appended row M002 and verified via read-back. |
table upsert-rows --worksheet-name Model | FAIL | /api/v1/excel/update_range_by_lookup | Real backend returned HTTP 400: missing worksheet gid/name for document=6a4496beef4212b56a15f4a8. |
table upsert-rows --gid 1 --verify | PASS | /api/v1/excel/update_range_by_lookup | Retry by gid updated row M002. This proves the endpoint works and isolates the failure to worksheet-name targeting. |
formula set | PASS | /api/v1/excel/formula/set | Set Model!E2 to =C2*D2. |
formula batch-set | PASS | /api/v1/excel/formula/batch_set | Set Model!E3 via batch operation. |
formula recalculate | PASS | /api/v1/excel/recalculate_formulas | Recalculated worksheet Model. |
formula read | PASS | /api/v1/excel/read_sheet | Read formula cells Model!E2:E3 with formula render option. |
range clear | PASS | /api/v1/excel/clear_range | Cleared Model!E3:E3. |
raw post | PASS | /api/v1/excel/list_worksheets | Posted a JSON body through the raw escape hatch and listed worksheets. |
请打开 验证工作簿，确认 Data 和 Model 两个工作表存在；Model!E2 应显示公式计算结果 42，Model!E3 已在清理步骤中被清空。
Coverage By Phase 1 Requirement
| Requirement | Implementation | Automated Test | Evidence |
|---|---|---|---|
mbs executable registration |
YES | PARTIAL | pyproject.toml:37-38 registers mbs = "maybeai_sheet.cli:app". Tests import and invoke the Typer app, but no test asserts console-script metadata or installed mbs execution. |
workbook import alias |
YES | YES | workbook.py:176-202 delegates import to create_from_file. tests/test_cli.py:1078-1094 verifies it uses /api/v1/excel/upload through patched upload_file. |
worksheet list/create |
YES | YES | worksheet.py:15-65 wraps legacy sheet functions. tests/test_cli.py:800-826 verifies /api/v1/excel/list_worksheets and /api/v1/excel/write_new_worksheet. |
range read/write/clear |
YES | YES | range.py:16-108 exposes all three commands. tests/test_cli.py:828-902 verifies /api/v1/excel/read_sheet, /api/v1/excel/update_range, and /api/v1/excel/clear_range. |
table headers/append-rows/upsert-rows |
YES | YES | table.py:16-112 exposes the three commands. tests/test_cli.py:904-974 verifies /api/v1/excel/read_headers, /api/v1/excel/append_rows, and /api/v1/excel/update_range_by_lookup. |
formula read/set/batch-set/recalculate |
YES | YES | formula.py:17-133 exposes all four commands. tests/test_cli.py:976-1076 verifies formula render option, set, batch-set, and recalculate endpoints. |
Legacy sheet compatibility |
YES | YES | Legacy command tests cover read, read-range, read-many, named-range, headers, worksheets, formulas, write-range, clear-range, append, formula-set, formula-batch-set, recalculate, upsert, and create-worksheet in tests/test_cli.py:115-634. |
No range read-many in Phase 1 |
YES | MANUAL ONLY | The plan explicitly excludes it at docs/superpowers/plans/2026-07-01-mbs-agent-friendly-command-tree.md:65-68. Source inspection shows only legacy sheet.py:160 has read-many; runtime help confirms range exposes only read, write, and clear. No automated negative test guards this. |
Online Endpoint Usage Verdict
A supplemental live probe did make real network calls to https://play-be.omnimcp.ai/.
- The CLI default base URL is
https://play-be.omnimcp.aiinsrc/maybeai_sheet/cli.py:45-48andsrc/maybeai_sheet/config.py. - The command tests patch
MaybeAIClient.post_jsonorMaybeAIClient.upload_filebefore invoking commands, so endpoint assertions inspect mocked call arguments. tests/test_cli.py:100-105creates aMaybeAIClientwithbase_url="https://play-be.omnimcp.ai", buttests/test_cli.py:100patchesmaybeai_sheet.client.httpx.Client. The request is captured by aMagicMock, not sent to the online API.- The real network boundary is
src/maybeai_sheet/client.py:47-52for JSON posts andsrc/maybeai_sheet/client.py:61-87for file uploads; these paths were mocked in the tests that exercised commands. - The supplemental live probe intentionally bypassed the mocked test suite and ran
uv run mbs --base-url https://play-be.omnimcp.ai ...commands with a real token from the environment. - Live verification workbook: https://www.maybe.ai/docs/spreadsheets/d/6a4496beef4212b56a15f4a8?gid=0.
Findings And Risks
P2 Console script registration is inspected, not tested
pyproject.toml correctly registers mbs, but the automated suite does not install the package or assert the project script entry. A packaging regression could break the executable while app-level CLI tests still pass.
P2 Negative coverage for range read-many is manual
The current source and help output satisfy the Phase 1 exclusion, but there is no unit test that fails if range read-many is accidentally added later.
P3 Tests prove wrapper-to-endpoint routing, not live backend compatibility
This still applies to the default unit suite. The supplemental live probe now proves most Phase 1 payloads are accepted by the routed API, but it also exposed a real compatibility issue for table upsert-rows --worksheet-name.
P1 Live table upsert-rows --worksheet-name fails
The live endpoint returned HTTP 400 with missing worksheet gid/name for document=6a4496beef4212b56a15f4a8 when running table upsert-rows --worksheet-name Model. Retrying the same operation with --gid 1 passed. The likely implementation issue is that the upsert payload path forces gid-style targeting and does not include worksheet_name in the request body, so worksheet-name addressing is accepted by the CLI but not honored by this endpoint.
P3 Worktree contains unrelated pre-existing dirty changes
git status --short showed modified source/docs/tests and untracked command/report directories before this report was written. Treat this review as a point-in-time audit of the current tree, not a clean commit review.
Concrete Recommendations
- Add a unit test or packaging smoke test that reads built metadata or installs the package in a temporary environment and verifies
mbs --helpresolves tomaybeai_sheet.cli:app. - Add an explicit regression test that
range --helpdoes not containread-many, whilesheet --helpstill does. - Keep the current mocked endpoint-equivalence tests as the Phase 1 safety net. They are fast, deterministic, and avoid depending on
https://play-be.omnimcp.ai/. - Run a separate opt-in integration suite against the online endpoint only when credentials and environment are intentionally provided. Do not mix that with default local unit tests.
- Fix
table upsert-rows --worksheet-namelive targeting or document that this command requires--giduntil the backend contract accepts worksheet names for/api/v1/excel/update_range_by_lookup.
Final Verdict
Phase 1 local test completion is effectively done for the implemented semantic command surface: all required semantic commands have local endpoint-equivalence tests, legacy sheet compatibility is covered, and the suite passes. The supplemental live probe created a MaybeAI Sheet and sent real requests to https://play-be.omnimcp.ai; 15 live command checks passed and 1 failed. The live failure is actionable: table upsert-rows works by --gid but fails by --worksheet-name.