Metadata-Version: 2.4
Name: dsh-cua
Version: 0.3.4
Summary: Windows computer-use MCP server for dsh (DeepSeek Harness) — client-neutral, works with any stdio MCP client. Accessibility-first element actions, skyshot text trees, guarded raw input, and a cross-session arbiter that yields to the human.
Author: Hutusion
License: MIT
Project-URL: Homepage, https://github.com/Hutusion/dsh-cua
Keywords: mcp,computer-use,windows,ui-automation,accessibility,agent
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: mcp<2,>=1.2
Requires-Dist: pywinauto>=0.6.9; sys_platform == "win32"
Requires-Dist: comtypes>=1.4; sys_platform == "win32"
Requires-Dist: pyperclip>=1.9
Requires-Dist: Pillow>=10
Provides-Extra: test
Requires-Dist: pyyaml>=6; extra == "test"
Dynamic: license-file

# dsh-cua

[![ci](https://github.com/Hutusion/dsh-cua/actions/workflows/ci.yml/badge.svg)](https://github.com/Hutusion/dsh-cua/actions/workflows/ci.yml)
[![PyPI](https://img.shields.io/pypi/v/dsh-cua)](https://pypi.org/project/dsh-cua/)
[![Hutusion/dsh-cua MCP server](https://glama.ai/mcp/servers/Hutusion/dsh-cua/badges/score.svg)](https://glama.ai/mcp/servers/Hutusion/dsh-cua)

<!-- mcp-name: io.github.Hutusion/dsh-cua -->

**English** · [中文](https://github.com/Hutusion/dsh-cua/blob/main/README.zh-CN.md)

<!-- Links in this file are absolute on purpose: PyPI renders this same README at
     https://pypi.org/project/dsh-cua/, where a relative target (README.zh-CN.md,
     examples/, skill/...) resolves against pypi.org and 404s. GitHub needs no help with
     either form. Do not "simplify" these back to relative paths. -->

An MCP server + agent skill for computer use on Windows: **accessibility element actions come
first and screenshots are only the fallback**. It ships a cross-session arbiter — when several
agents share one machine it serializes them, and it yields while you are actually using the
computer yourself.

This repository contains only the MCP server and the skill. It stays **neutral toward any stdio
MCP client** (dsh / Claude Code / Codex / Cursor / Cline / ZCode …) — nothing here requires dsh.

**Platform semantics (0.3.1 and later)**: the tools only work on Windows — they drive
user32/kernel32 and UI Automation. But **the package also imports elsewhere and the server
starts there**, answering `tools/list` as usual, so any client or directory crawler can
enumerate all 19 tools with their full schemas; actually calling a tool returns a clear
"requires Windows" error rather than the process failing to start at all. In 0.3.0 the import
itself raised, which made such crawlers unable to see the server at all
(`tests/linux-handshake.py` is the regression test for this property, and CI runs it on
`ubuntu-latest`).

## What it is

A stdio MCP server exposing 19 tools. Every tool name is prefixed `tool_`, exactly as
`tools/list` returns it.

- **Observe** (read-only, callable at any time): `tool_skyshot` (reads a window as a compact,
  diffable text tree — three orders of magnitude smaller than a screenshot), `tool_element_at_point`,
  `tool_read_element`, `tool_find_elements`, `tool_capture_window` (DPI-aware, cropped to the client area),
  `tool_list_windows` / `tool_find_window` / `tool_get_window_rect`, `tool_list_displays`, `tool_cursor_position`,
  `tool_clipboard_read`, `tool_coexistence_status`
- **Element actions** (soft gate: serialized across agents, no physical input injected):
  `tool_element_action` / `tool_element_action_at` — press / set_value / select / toggle / expand /
  collapse / scroll_into_view / focus, delivered straight to the UIA element, so they
  **never steal focus and never care about z-order**
- **Other mutating calls** (soft gate too: the mutex, but **no** human-contention yield):
  `tool_type_text` (targeted PostMessage), `tool_clipboard_write`, `tool_open_application`. They
  synthesize no physical input, so they do **not** wait for you to stop working — and a clipboard
  write still destroys whatever you last copied, so announce it when you do.
- **Physical input** (hard gate: serialized across agents **and** yields to the human): exactly two
  things share your one cursor and one keyboard — `tool_click_at`'s **raw_event path** and
  `tool_send_keys`' global hotkeys. The gate waits for the machine to go input-quiet, then refuses
  with `user-active` rather than fight you for the cursor. `tool_click_at` tries its **element path
  first** (`ax_press`), which injects no physical input and therefore takes the mutex only; the
  receipt's `method` field says which path actually ran.

"Read-only" here means it takes no mutating action and synthesizes no input, so it is safe to
call while someone is using the machine. Two of them have a side effect worth knowing:
`tool_capture_window` writes the screenshot to disk (`save_path`; a temp file when omitted), and
`tool_skyshot` updates the server-side diff baseline it diffs the next shot against.

Every action returns a **receipt** rather than a self-reported success: `action_sent` /
`effect_verified` / `foreground_changed` / `user-active` / `arbiter-busy`. "The call was
accepted" and "the effect happened" are two different things, and the tool separates them for
the agent.

## How it differs

There are already several mature open-source Windows implementations. dsh-cua's differences are
concentrated on one thing: **sharing a machine with a human.**

| | dsh-cua | [cua-driver](https://github.com/trycua/cua) | [ahk-mcp](https://github.com/anomalous3/ahk-mcp) | [lean-computer-use-mcp](https://github.com/Kvxw1105/lean-computer-use-mcp) |
|---|---|---|---|---|
| Element actions delivered as UIA patterns (no focus steal, z-order irrelevant) | ✅ | ✅ (ax mode) | ❌ reads via UIA, acts by coordinate click | via cua-driver |
| **Recent human input → refuse** | ✅ `user-active` | ❌ | ❌ | ❌ |
| Cross-agent serialization (multi-process) | ✅ named mutex | ❌ | ❌ | ❌ |
| Per-action effect assertion | ✅ three-state `effect_verified` | reports a delivery tier | ❌ | ❌ `state_changed` heuristic only |
| Foreground-steal side effect measured | ✅ `foreground_changed` | ❌ | ❌ | ❌ |
| Tool count | 19 | 59 | 15 | 6 |

**The key distinction is two things that are routinely conflated:**

- **"No focus steal" is a mechanism guarantee** — either a UIA pattern or a targeted
  `PostMessage`, so the cursor and keyboard focus are physically never touched. cua-driver has
  it (ax mode). **ahk-mcp does not**, and the distinction is narrower than "no UIA": it reads
  through UIA (`ahk_uia_tree` / `ahk_uia_find` / `ahk_uia_url`), but it has no UIA *pattern
  action* — per its README it acts with coordinate clicks or synthetic keys, so an action does
  move the real cursor.
- **"Yield the moment you move" is a timing guarantee** — it reads the age of the human's last
  input via `GetLastInputInfo`, waits when it sees you using the machine, and on timeout
  **refuses** (`user-active`) instead of barging in. As of 2026-09-25 a pattern search across
  the other three codebases in that table found no equivalent — that is search evidence, not
  proof, and it covers input-age detection only: cua-driver does have human-facing guards of a
  different kind (a consent requirement, and foreground-steal detection with restore).

`effect_verified` is likewise something the alternatives lack: it splits "the call was accepted"
from "the effect happened" and gives three states (`true` changed as expected / `false` accepted
but unchanged, downgraded to a failure / `null` no comparable state, i.e. unconfirmed). The
usual alternative is to re-observe once after the action and leave the judgement to the model.

**What dsh-cua does not do** (stated up front to avoid misunderstanding): no pixel/vision
grounding — interfaces a tree cannot express (canvas, games, remote desktop) are out of reach;
no record-and-replay; no isolation sandbox. There are better-suited tools for those.

## Install

You need **Windows x64 + an interactive desktop session + Python ≥3.10** to actually drive a
desktop. (The package installs and starts on Linux/macOS too, `tools/list` answers normally,
and a tool call then reports "requires Windows" — see "platform semantics" above.)

```bash
# Option 1: uvx, zero install (recommended)
uvx dsh-cua                      # runs the stdio MCP server directly

# Option 2: pip
pip install dsh-cua

# Option 3: from source
pip install git+https://github.com/Hutusion/dsh-cua.git
```

However you install it, **start the server with `python -m dsh_cua`**:

```bash
python -m dsh_cua                # depends on no executable being on PATH
```

> **Why the README does not say `dsh-cua-server`**: pip installs console scripts into the
> interpreter's `Scripts` directory, and **that directory is not necessarily on PATH** —
> measured on a stock python.org 3.12 install, neither the User nor the Machine PATH contained
> it, so `pip install dsh-cua` succeeded while `dsh-cua-server` reported command not found.
> `python -m` needs no PATH entry at all. The console script is still shipped and works when
> PATH does contain it.
>
> Options 1 and 2 both work today: the package is published on PyPI
> (<https://pypi.org/project/dsh-cua/>). If `uvx`/`pip` ever 404s, use option 3 — it always
> works.

## Wiring it up

Any MCP client; name the server **`win32`** (the skill's tool-name convention is
`mcp__win32__*`).

**`python -m` (no PATH dependency, recommended)**:

```json
{ "mcpServers": { "win32": { "command": "python", "args": ["-m", "dsh_cua"] } } }
```

**`uvx`**:

```json
{ "mcpServers": { "win32": { "command": "uvx", "args": ["dsh-cua"] } } }
```

More shapes are in [`examples/`](https://github.com/Hutusion/dsh-cua/tree/main/examples): Claude Code / generic clients / a dsh
`cordis.patch.yml` fragment / the route modality declaration you need if you want the model to
read screenshots (`tr-route-settings.yml`).

## Skill (optional but strongly recommended)

[`skill/computer-use/SKILL.md`](https://github.com/Hutusion/dsh-cua/blob/main/skill/computer-use/SKILL.md) is the companion doctrine for using
these tools: the observe → locate → act → verify loop, receipt semantics, retry safety, and the
discipline of coexisting with a human. The model can use the tools without it, but with it the
model **picks the right path by itself** — the measured difference is large. Copy it into your
skills directory:

```bash
# Claude Code / generic agents
cp -r skill/computer-use ~/.agents/skills/
# dsh
cp -r skill/computer-use ~/.dsh/skills/
```

## Security model

| Tier | Operations | Gate |
|---|---|---|
| Read-only | the 12 observe tools | no gate, callable at any time |
| Soft | element actions, PostMessage typing, clipboard write, launching applications | cross-agent mutex (named mutex, multi-process, automatic serialization) |
| Hard | raw clicks, global hotkeys | mutex + `GetLastInputInfo` yielding: if the user typed recently it waits, and on timeout refuses with `user-active` instead of stealing the cursor |

Honest boundaries: yielding is a cooperation protocol, not a hard guarantee (the tight check
150 ms before injection narrows the window as much as possible); a few applications
self-activate even on `set_value` (the receipt reports `foreground_changed` truthfully); and
two operators on the same window has no technical solution — do not drive the same window the
agent is driving.

## Tests

```bash
python tests/verify-coexistence.py    # 25 checks: zero-input proof / cross-process mutex / synthetic human contention / kill switch
python tests/verify-p0-fixes.py       # the three P0s fixed in 0.2.0: each fails before the fix
```

The tests need no human cooperation — "user input" is synthesized with one real 1-pixel cursor
move, and the cursor is restored afterwards.

**The tests need a real interactive desktop session** (some checks create windows and address
them through UIA), so they **cannot run on a GitHub-hosted runner**. What CI does cover is the
part that needs no desktop: packaging and installation, module import, regressions for the diff
index and tree-line escaping, and the arbiter's decision logic — see
[`.github/workflows/ci.yml`](https://github.com/Hutusion/dsh-cua/blob/main/.github/workflows/ci.yml).

```bash
python tests/ci-desktop-free.py       # the local equivalent of the above, no desktop needed
```

## License

MIT
