Metadata-Version: 2.4
Name: blindpilot-mcp
Version: 0.1.0
Summary: MCP server for desktop UI automation
Requires-Python: >=3.14
Requires-Dist: mcp[cli]>=1.27.2
Requires-Dist: psutil>=7.1.3
Requires-Dist: pyautogui>=0.9.54
Requires-Dist: pyperclip>=1.11.0
Requires-Dist: pywinauto>=0.6.9; sys_platform == 'win32'
Provides-Extra: linux
Requires-Dist: dogtail>=1.0.0; extra == 'linux'
Provides-Extra: macos
Requires-Dist: appium-python-client>=5.2.4; extra == 'macos'
Requires-Dist: pyobjc-framework-quartz>=12.0; extra == 'macos'
Description-Content-Type: text/markdown

# Blindpilot MCP

BlindPilot MCP redefines computer use. Instead of AI wasting time interpreting screenshots, it reads directly on the operating system's UI tree and sends it to the LLM as text, enabling faster, more accurate, and more reliable desktop automation. No screenshot -> AI -> control loop. This project is also compatible with Linux and macOS.

## Installation

### MCP Configuration

If `blindpilot-mcp` is published as a Python package, you only need the MCP
server name, command, and package executable:

```json
{
  "mcpServers": {
    "blindpilot": {
      "type": "stdio",
      "command": "uvx",
      "args": ["blindpilot-mcp"]
    }
  }
}
```

Claude Code:

```bash
claude mcp add --transport stdio blindpilot -- uvx blindpilot-mcp
```

MCP UI fields:

```text
Name: blindpilot
Command: uvx
Arguments: blindpilot-mcp
```

If you publish the package under another name, replace `blindpilot-mcp` with
that package command.

### Local Development

Use this while developing from a local clone.

1. Clone the repository:

```bash
git clone <repo-url>
cd blindpilot-mcp
```

2. Copy the absolute path to the project.

On Windows PowerShell:

```powershell
(Get-Location).Path
```

On macOS or Linux:

```bash
pwd
```

3. No project install is required for local Claude/uvx usage.

Claude can run the server through `uvx`, which creates an isolated environment
from this project automatically.

For local development, you can still install dependencies with uv:

```bash
uv sync
```

If you prefer pip:

```bash
pip install -r requirements.txt
```

4. Install optional platform dependencies when needed.

Linux accessibility support:

```bash
uv sync --extra linux
```

macOS Appium support:

```bash
uv sync --extra macos
```

Install every optional dependency:

```bash
uv sync --all-extras
```

With pip, use:

```bash
pip install ".[linux]"
pip install ".[macos]"
```

Windows does not need an extra; pywinauto is installed automatically on Windows.

5. Add the local MCP server to Claude Code with stdio and uvx.

Replace `<absolute-project-path>` with the path copied in step 2:

```bash
claude mcp add --transport stdio blindpilot -- uvx --from "<absolute-project-path>" blindpilot-mcp
```

Example on this machine:

```powershell
claude mcp add --transport stdio blindpilot -- uvx --from "C:\Users\Noah\Documents\blindpilot-mcp" blindpilot-mcp
```

To share the MCP configuration with the repository:

```bash
claude mcp add --transport stdio --scope project blindpilot -- uvx --from "<absolute-project-path>" blindpilot-mcp
```

6. For Claude Desktop or another MCP client, use this local stdio server configuration:

```json
{
  "mcpServers": {
    "blindpilot": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "--from",
        "<absolute-project-path>",
        "blindpilot-mcp"
      ]
    }
  }
}
```

### GitHub Source

You can also run directly from a GitHub repository without cloning first:

```bash
claude mcp add --transport stdio blindpilot -- uvx --from "git+https://github.com/<owner>/blindpilot-mcp" blindpilot-mcp
```

MCP JSON:

```json
{
  "mcpServers": {
    "blindpilot": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "--from",
        "git+https://github.com/<owner>/blindpilot-mcp",
        "blindpilot-mcp"
      ]
    }
  }
}
```

## Usage

Blindpilot exposes MCP tools for desktop UI automation:

- run multiple tools in order with `run_batch`
- list open windows
- read accessible elements from a window
- click accessible controls
- type text into controls or the active window
- use keyboard, mouse, and clipboard actions
- terminate processes
- handle system dialogs

Interaction guidance:

- Use `run_batch` for ordered action chains.
- Do not add `wait` between normal interactions.
- Use short waits only for loading screens, routine checks, rate limits, or small animation delays.
- Prefer `wait_for_element` when waiting for a specific UI state.
- If `snapshot_window` fails, returns empty data, or looks incomplete, call it a second time before giving up.
- Use `key_down` + `key_press` + `key_up` for shortcuts. The unreliable combined `hotkey` helper is not exposed.

Batch example:

```json
{
  "steps": [
    {"tool": "key_down", "args": {"key": "ctrl"}},
    {"tool": "key_press", "args": {"key": "l"}},
    {"tool": "key_up", "args": {"key": "ctrl"}},
    {"tool": "type_global", "args": {"text": "https://example.com"}},
    {"tool": "key_press", "args": {"key": "enter"}}
  ]
}
```

## Development

Sync dependencies:

```bash
uv sync
```

Run the stdio server manually:

```bash
uv run blindpilot-mcp
```

When launched manually, the process waits for MCP messages on stdin. In normal use, Claude Code, Claude Desktop, or another MCP client starts it automatically.

Compile the modules to validate syntax:

```bash
uv run python -m compileall src
```

## Publishing

This project is configured for PyPI Trusted Publishing through GitHub Actions.

In PyPI, create a trusted publisher with:

```text
Workflow name: publish.yml
Environment name: pypi
```

Then publish by creating a GitHub release, or run the `Publish to PyPI`
workflow manually from the repository's Actions tab.

## Project Structure

```text
src/
  main.py                         Compatibility entrypoint
  blindpilot_mcp/
    __init__.py                   Package marker
    server.py                     MCP server factory and stdio runner
    tools/
      __init__.py                 Tool package export
      registry.py                 Tool registration list
      context.py                  Platform detection and shared automation state
      accessibility.py            Window and accessibility-tree tools
      batch.py                    Ordered batch execution tool
      input.py                    Keyboard and mouse tools
      clipboard.py                Clipboard tools
      processes.py                Process termination tools
      sync.py                     Waiting and polling helpers
      values.py                   UI element value readers
      dialogs.py                  System dialog handling
```

## Requirements

- Python 3.14
- uv
- Windows for UIA automation through pywinauto

Linux requires AT-SPI/dogtail support. macOS requires Appium with the mac2 driver.

## Troubleshooting

If Claude logs an error like:

```text
error: The system cannot find the file specified. (os error 2)
```

check the `uvx` arguments in the MCP configuration.

For the simple published-package setup, use:

```json
"command": "uvx",
"args": ["blindpilot-mcp"]
```

For local development, the `--from` value must point to this repository, not to
a Claude workspace folder.

Correct:

```text
C:\Users\Noah\Documents\blindpilot-mcp
```

Incorrect:

```text
C:\Users\Noah\Documents\Claude
```

The configured directory must contain:

```text
pyproject.toml
src/blindpilot_mcp/server.py
```
