Metadata-Version: 2.4
Name: robotframework-desktop
Version: 0.1.1
Summary: Robot Framework library for Windows desktop automation with hybrid image (pyautogui) and UI Automation (pywinauto) engines.
License: Apache-2.0
Keywords: robotframework,desktop,automation,pyautogui,pywinauto,uia,wpf,winforms
Classifier: Framework :: Robot Framework :: Library
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: robotframework>=6.1
Requires-Dist: robotframework-pythonlibcore>=4.2
Requires-Dist: pyautogui>=0.9.54
Requires-Dist: opencv-python>=4.8
Requires-Dist: pillow>=10.0
Requires-Dist: mss>=9.0
Requires-Dist: pywinauto>=0.6.8
Requires-Dist: pyperclip>=1.8
Requires-Dist: rapidocr>=3.0
Requires-Dist: onnxruntime>=1.17
Provides-Extra: ocr
Requires-Dist: rapidocr>=3.0; extra == "ocr"
Requires-Dist: onnxruntime>=1.17; extra == "ocr"
Provides-Extra: dev
Requires-Dist: pytest>=7; extra == "dev"
Requires-Dist: rapidocr>=3.0; extra == "dev"
Requires-Dist: onnxruntime>=1.17; extra == "dev"
Requires-Dist: PyYAML>=6.0; extra == "dev"
Requires-Dist: blue>=0.9; extra == "dev"

<div align="center">

# 🖥️ robotframework-desktop

**Hybrid Windows desktop automation for [Robot Framework](https://robotframework.org)**
Image recognition · UI Automation · OCR — one keyword set, chosen by the locator.

![Python](https://img.shields.io/badge/python-3.10%2B-3776AB?logo=python&logoColor=white)
![Robot Framework](https://img.shields.io/badge/Robot%20Framework-6.1%2B-000000?logo=robotframework&logoColor=white)
![Platform](https://img.shields.io/badge/platform-Windows-0078D6?logo=windows&logoColor=white)
![License](https://img.shields.io/badge/license-Apache--2.0-green)
![Code style: blue](https://img.shields.io/badge/code%20style-blue-1f6feb)

[Install](#-install) · [Quick start](#-quick-start) · [Locators](#-locators) · [YAML locators](#-locators-in-a-yaml-file) · [OCR](#-ocr) · [Keywords](docs/DesktopLibrary.html)

</div>

---

## ✨ Features

- 🔀 **Hybrid keywords** – `Click`, `Input Text`, `Drag And Drop`, ... work with images, UI elements or regions; the engine is picked from the `element` argument.
- 🖼️ **Image recognition** – pyautogui + OpenCV, per-call similarity, multiple candidate images, offsets and search regions.
- 🌳 **UI Automation** – pywinauto (`uia` and `win32` backends) for WPF, WinForms, UWP and Win32 apps.
- 🔤 **OCR** – [RapidOCR](https://github.com/RapidAI/RapidOCR) on onnxruntime, models bundled, **no external binary**; accent and case insensitive text search.
- ⏳ **Busy cursor aware** – actions wait automatically while the app shows the busy cursor.
- 🧩 **SikuliLibrary compatible** – implements all its actions, plus UIA keywords (`Attach To Window`, `Get Element Text`, `Set Element Value`, `Select Combobox Item`, ...).

## 📦 Install

```bash
pip install robotframework-desktop          # core
pip install robotframework-desktop[ocr]     # + OCR (rapidocr + onnxruntime)
```

## 🚀 Quick start

```robot
*** Settings ***
Library    DesktopLibrary    image_paths=${CURDIR}/images;${CURDIR}/shared    similarity=0.6

*** Test Cases ***
Write In Notepad
    Open Application     notepad.exe
    Attach To Window     name_re:.*Notepad
    Input Text           class:Edit    Hello
    Click                ${{ {"path": ["save.png", "save_dark.png"], "similarity": 0.85} }}
    Wait Until Screen Contain    done.png    10s
    [Teardown]    Close Application    notepad.exe
```

> [!TIP]
> Relative image paths are searched in the `image_paths` folders (in order, extendable with `Add Image Path`) and then in the current working directory. The extension can be omitted.

## 🎯 Locators

| Locator | Engine |
|---|---|
| `save.png` · `save.png=0.85` · `image:C:/refs/save.png` | 🖼️ Image |
| `{"path": [...], "similarity": 0.8, "offset": {x, y}, "region": {x, y, width, height}}` | 🖼️ Image |
| `automation_id:btnSave` · `name:OK AND control_type:Button` | 🌳 UIA |
| `win32:class:Button` · `{"name": "Save", "backend": "win32"}` | 🌳 Win32 |
| `{"automation_id": "btnSave", "offset": {"x": 5, "y": 0}}` | 🌳 UIA |
| `[x, y, w, h]` · `{"region": {...}}` · region/match returned by keywords | 📐 Coordinates |

## 🗂️ Locators in a YAML file

Keep all element locators in one variable file ([full example](tests/atest/elements.yml)):

```yaml
APP:
  BTN_SAVE: "automation_id:btnSave"
  TXT_NAME:
    automation_id: txtName
    control_type: Edit
  BTN_SAVE_WIN32:
    name: Save
    class_name_re: ".*BUTTON.*"
    backend: win32
  IMG_TARGET_OFFSET:
    path: target.png
    offset:
      x: 30
      y: 20
  IMG_IN_REGION:
    path: save.png
    region:
      x: 0
      y: 0
      width: 800
      height: 600
```

```robot
*** Settings ***
Library      DesktopLibrary
Variables    elements.yml          # requires PyYAML

*** Test Cases ***
Save
    Input Text    ${APP.TXT_NAME}    Robot
    Click         ${APP.BTN_SAVE}
    Click         ${APP.IMG_TARGET_OFFSET}
```

## 🔤 OCR

No language setup needed — the default RapidOCR models read Latin text (accented letters may come out unaccented, so matching ignores accents and case).

```robot
${text}=    Get Text    automation_id:lblStatus
${text}=    Read Text From Region    [10, 400, 300, 30]
Click Text    Clear
Set ROI    [0, 0, 600, 400]
Region Click Text    Clear
```

## 🔁 Looping over elements

```robot
@{items}=    Find Elements    control_type:ListItem          # [] when nothing is found
FOR    ${item}    IN    @{items}
    ${text}=    Get Element Text    ${item}
    IF    $text == "Item 3"    Click    ${item}
END
@{icons}=    Find Elements    icon.png    sort_by=column    min_count=3
@{rows}=     Find Child Elements    automation_id:grid    control_type:DataItem    depth=1
```

## ⏳ Busy cursor handling

Action keywords wait while the cursor is busy (`wait` / `appstarting`) until it stays normal for 0.5 s — **before** searching, before acting and **after** the action (watched for 0.2 s). On timeout (default 30 s) the keyword fails, or only warns with `busy_cursor_fail=False` / `ignore_busy_error=True`. Use `show_warnings=False` to log warnings as INFO.

```robot
Library    DesktopLibrary    busy_cursor_timeout=1 min    busy_cursor_after_delay=0.3s

Click    automation_id:btnLoad    ignore_busy_error=True
Set Busy Cursor Wait     False
Set Busy Cursor Types    wait,appstarting,custom
Wait Until Cursor Is Not Busy    2 min
${cursor}=    Get Current Cursor Type
```

## 📚 Documentation

Full keyword reference: **[docs/DesktopLibrary.html](docs/DesktopLibrary.html)** (`just docs` to regenerate).

## 🛠️ Development

Requires [uv](https://docs.astral.sh/uv/) and [just](https://github.com/casey/just) (PowerShell recipes).

| Command | Description |
|---|---|
| `just setup` | Create `.venv` and install the library (editable + dev deps) |
| `just fmt` | Format code with [blue](https://github.com/grantjenks/blue) |
| `just test` | Unit tests (pytest) |
| `just atest` | Acceptance tests (Robot Framework) |
| `just docs` | Regenerate libdoc |
| `just build` | Build both wheels |
| `just build-full` | `dist/full` – OCR deps required |
| `just build-no-ocr` | `dist/no-ocr` – OCR optional (`[ocr]` extra) |
| `just check` | Show dependencies of the built wheels |

### Release

CI ([.github/workflows/ci.yml](.github/workflows/ci.yml)) runs lint, unit and acceptance tests on every push and pull request.
To publish a release, bump `version` in `pyproject.toml`, then:

```powershell
git tag v0.2.0
git push --tags
```

[.github/workflows/release.yml](.github/workflows/release.yml) runs the tests, builds both wheels and creates a GitHub Release with them attached:

- `robotframework-desktop` - OCR optional (`[ocr]` extra)
- `robotframework-desktop-full` - OCR deps required

> [!WARNING]
> The acceptance suite launches a small WinForms app ([tests/app/SampleApp.ps1](tests/app/SampleApp.ps1)) and generates its reference images at runtime. **Don't use the mouse or keyboard while it runs.**

## 📄 License

[Apache-2.0](https://www.apache.org/licenses/LICENSE-2.0)
