Metadata-Version: 2.4
Name: winuia-auto
Version: 0.1.5
Summary: Python UIAutomation for Windows
Home-page: https://github.com/pythonlw/winuia-auto
Author: lw
Author-email: 
License: Apache-2.0
Keywords: windows ui automation winuia inspect
Platform: Windows Only
Classifier: Development Status :: 4 - Beta
Classifier: Environment :: Win32 (MS Windows)
Classifier: Intended Audience :: Developers
Classifier: Operating System :: Microsoft :: Windows
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: comtypes>=1.2.1
Requires-Dist: pywin32>=306; platform_system == "Windows"
Requires-Dist: pynput>=1.7.6
Requires-Dist: lxml>=4.9.0
Provides-Extra: visual
Requires-Dist: Pillow>=10.0.0; extra == "visual"
Requires-Dist: numpy>=1.24.0; extra == "visual"
Provides-Extra: input-fallback
Provides-Extra: ocr
Requires-Dist: rapidocr>=3.9.0; extra == "ocr"
Requires-Dist: rapidfuzz>=3.0.0; extra == "ocr"
Requires-Dist: onnxruntime>=1.28.0; extra == "ocr"
Requires-Dist: opencv-python>=4.8.0; extra == "ocr"
Provides-Extra: dev
Requires-Dist: build>=1.2.0; extra == "dev"
Requires-Dist: rapidfuzz>=3.0.0; extra == "dev"
Requires-Dist: twine>=5.0.0; extra == "dev"
Requires-Dist: wheel>=0.42.0; extra == "dev"
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: keywords
Dynamic: license
Dynamic: license-file
Dynamic: platform
Dynamic: provides-extra
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

# winuia-auto - Refactored Python UIAutomation for Windows

`winuia-auto` is a refactored Windows UI Automation package built on the classic Windows UI Automation API. It keeps the familiar import style and most legacy function names while adding a cleaner internal architecture, modern input helpers, safer clicking, tab-window utilities, visual matching helpers, and PyPI-ready packaging metadata.

This package is designed for automating Windows desktop applications that expose Microsoft UIAutomation providers, including Win32, WinForms, WPF, Modern UI, Qt-based apps, browser windows, and Electron-based apps when accessibility support is enabled.

## Installation

```bash
pip install winuia-auto
```

The default installation includes `comtypes`, `pywin32`, `Pillow`, `numpy`,
`opencv-python`, and `pynput`. Image matching and hotkey helpers work without
installing feature extras. This project does not depend on `pyautogui`.

Development tools remain optional:

```bash
pip install "winuia-auto[dev]"
```

## Compatibility

The public compatibility layer is intentionally preserved:

```python
import winuia as auto

window = auto.GetForegroundControl()
button = window.ButtonControl(Name="OK")
button.Click()
```

Most old names such as `Click`, `SendKeys`, `ControlType`, `PatternId`, `PropertyId`, `WindowControl`, `ButtonControl`, and `EditControl` remain available from the top-level package. New code can also use the added helper APIs described below.

## What Changed in This Refactor

### Modular Architecture

The old project was centered around one very large `winuia.py` file. This refactor starts splitting responsibilities into focused modules:

- `winuia/enums.py`: UIAutomation and Win32 integer enums such as `ControlType`, `PatternId`, `PropertyId`, `Keys`, `MouseEventFlag`, and `KeyboardEventFlag`.
- `winuia/structures.py`: `Rect` and ctypes input structures such as `INPUT`, `MOUSEINPUT`, `KEYBDINPUT`, and `HARDWAREINPUT`.
- `winuia/inputs/core.py`: low-level `SendInput`, `MouseInput`, `KeyboardInput`, and virtual-key scan-code helpers.
- `winuia/inputs/keyboard.py`: `pynput` hotkey helpers.
- `winuia/win32_api.py`: pywin32-first, ctypes-fallback wrappers for selected Win32 APIs.
- `winuia/client.py`: thread-safe singleton helpers and package DLL path handling.

The original `winuia.py` remains as a compatibility aggregation layer while larger areas such as controls, patterns, bitmap handling, and logging are migrated progressively.

### Modernized Constants and Rect

- Classic constant classes have been moved to `enum.IntEnum` while keeping int-compatible behavior.
- `Rect` is now a dataclass-compatible helper in `structures.py`.
- Empty rectangle detection now treats zero or negative width/height as empty.

### Modern Input Simulation

- Mouse clicks, mouse button down/up, mouse wheel, single-key input, and `SendKeys` now use `SendInput` internally.
- Deprecated `mouse_event` and `keybd_event` names are still available as compatibility wrappers.
- `SendInput` batches input events instead of sending them one by one.

### Human-like Input Simulation

The standard input APIs can opt into human-like timing and movement with
`humanLike=True`. This keeps the existing API surface while adding smoother
mouse paths, randomized click timing, randomized typing intervals, and more
natural wheel scrolling when explicitly requested.

```python
import winuia as auto

# Move with a cubic Bezier path by default.
auto.MoveTo(500, 300, humanLike=True, duration=0.8)

# Click with human-like movement, pre-click pause, and mouse-down hold time.
auto.Click(500, 300, humanLike=True, duration=0.8)
control.Click(humanLike=True, duration=0.8, jitter=4)

# Type with variable intervals. duration controls how long the mouse takes to
# reach the control before typing. errorRate is optional and defaults to 0.
control.SendKeys(
    "hello from winuia",
    humanLike=True,
    duration=0.8,
    intervalRange=(0.05, 0.18),
    errorRate=0.01,
)

# Scroll with non-uniform wheel intervals and a controlled mouse-arrival time.
control.WheelDown(wheelTimes=5, humanLike=True, duration=0.8, intervalRange=(0.1, 0.3))
```

`MoveTo()` accepts `curveFunc` for advanced callers that need a custom movement
curve. The function receives `(startX, startY, targetX, targetY, t)` and returns
`(x, y)` for the current progress value:

```python
def linear_curve(start_x, start_y, target_x, target_y, t):
    return (
        start_x + (target_x - start_x) * t,
        start_y + (target_y - start_y) * t,
    )

auto.MoveTo(500, 300, humanLike=True, duration=0.8, curveFunc=linear_curve)
```

Use `duration=seconds` when the mouse should take a fixed amount of time to
reach the target. Use `durationRange=(minSeconds, maxSeconds)` when the arrival
time should be randomized. `duration` takes priority when both are supplied.

`SafeClick(humanLike=True)` skips silent UIA invoke/post-message paths and uses
the physical mouse click path. Use these options only for normal desktop UI
automation where realistic input timing is desired; they are not intended for
bypassing access controls or verification challenges.

### DPI and Multi-Monitor Improvements

New helpers improve high-DPI and multi-monitor coordinate behavior:

```python
import winuia as auto
auto.SetDpiAwareness()
rect = auto.GetVirtualScreenRect()
inside = auto.IsPointInVirtualScreen(x, y)
```

`SetDpiAwareness()` prefers PerMonitorV2 awareness and automatically falls back to older Windows DPI APIs.

### Safer Clicks

Controls now support visibility and clickability checks:

```python
control.IsVisibleOnScreen()
control.IsClickable()
control.SafeClick()
auto.SafeClick(x, y)
```

`control.SafeClick()` now prefers UIA `InvokePattern` for center-click style calls. This is more reliable for buttons whose visual center is covered by another layer or whose physical clickable point is inaccurate. If `InvokePattern` is unavailable or fails, it falls back to the previous mouse-click path with `ScrollIntoView` and clickability/occlusion checks.

`auto.SafeClick(x, y)` is a coordinate-based safe click: it gets the control at `(x, y)`, tries silent UIA actions on the hit control and its ancestors, then tries a Win32 `PostMessage` click, and falls back to `auto.Click(x, y)` only when those silent attempts fail. `PostMessage` can only confirm that mouse messages were posted; some modern or hardware-accelerated apps may ignore them.

To force the physical mouse-click path:

```python
control.SafeClick(preferInvoke=False)
auto.SafeClick(x, y, useInvoke=False, useLegacy=False, usePostMessage=False)
```

### Fluent Relative Search

Controls can be filtered by relative position:

```python
import winuia as auto
window = auto.ControlFromHandle(handle) # handle为窗口句柄
save = window.ButtonControl(Name="Save").RightOf(cancel_button)
save = window.ButtonControl(Name="Save").right_of(cancel_button)
```

Available helpers:

- `LeftOf()` / `left_of()`
- `RightOf()` / `right_of()`
- `Above()` / `above()`
- `Below()` / `below()`
- `AddCompare()`

```python
def relative_position():
    # 先定位一个已知的参考控件（作为“锚点”）
    reference = auto.ButtonControl(Name="智能分流")
    # 然后使用相对位置查找目标 A.RightOf(B) 表示“A 在 B 的右边”
    target = auto.ButtonControl(Name="全局代理").RightOf(reference)
    print(target)
    # 或者写成链式：
    target1 = reference.LeftOf(auto.TextControl(Name="全局代理"))  # 反过来也行？
    # # 注意：通常的写法是：目标控件的条件 .RightOf(参考控件)
    # 目标（智能分流） 在 参照物（全局代理）的左侧
    print(target1)
    #目标（快速连接） 在 参照物（智能分流）的上方
    # target2 = auto.TextControl(RegexName="快速连接").Above(reference)
    # print(target2)
```

### Smart Waiting

Modern wait aliases are available:

```python
control.WaitUntilExists(timeout=10)
control.WaitUntilDisappear(timeout=5)

auto.WaitUntilExists(control, timeout=10)
auto.WaitUntilDisappear(control, timeout=5)
```

### UI Automation Event Listening

`EventHub` can listen to UI Automation events such as focus changes, window opened events, and property changes. Event delivery depends on COM message pumping; `EventHub(..., autoPump=True)` starts a background pump thread. Keep callbacks lightweight.

Typical use cases are places where event-driven waiting is cleaner than polling:

- Wait for a login, save, or export dialog to open before finding its controls.
- Wait for a button, menu item, or text box to become enabled before clicking or typing.
- Record focus changes while debugging or building a simple script recorder.

```python
import time
import winuia as auto
from winuia import winuia as auto_core

hub = auto.EventHub(auto_core._AutomationClient.instance().IUIAutomation)

def on_focus(sender):
    control = auto.Control.CreateControlFromElement(sender)
    print(control.ControlTypeName, control.Name)

with hub.listen_focus_changed(on_focus):
    time.sleep(10)
```

Property waits can avoid polling when the current value is not ready:

```python
value = auto.WaitForProperty(
    button,
    auto.PropertyId.IsEnabledProperty,
    lambda current: bool(current),
    timeout=10,
    hub=hub,
)
```

### Recursive FindAll and Tab Utilities

Windows with tab controls can be automated with `FindAll()` and tab helpers:

```python
tabs = window.FindAll(auto.ControlType.TabItemControl, maxDepth=10)
for tab in tabs:
    url = auto.GetTabUrl(tab, window=window)
    print(tab.Name, url)

current_tab = window.GetCurrentTab()
window.SwitchToTabByName("Settings")
window.SwitchToTabByUrl("example.com")
url = window.GetCurrentUrl()
tab_urls = window.GetAllTabUrls()
```

Module-level helpers are also available:

```python
tabs = auto.FindAll(window, auto.ControlType.TabItemControl)
tabs = auto.find_control_num(window, auto.ControlType.TabItemControl)
tab = auto.get_current_tab(window)
auto.switch_to_tab_by_name(window, "Settings")
auto.SwitchToTabByUrl(window, "example.com")
url = auto.get_current_url(window)
url = auto.GetTabUrl(tab, window=window)
tab_urls = auto.GetAllTabUrls(window)
```

`FindAll()` supports `maxDepth` to avoid walking extremely large UI trees indefinitely. Current-tab detection first uses `SelectionItemPattern.IsSelected`, then falls back to keyboard-focus properties. Tab switching prefers `SelectionItemPattern.Select()` and falls back to `Click()`. `SwitchToTabByUrl()` supports substring URL matching by default. `GetTabUrl()` and `GetAllTabUrls()` switch tabs to read the active address bar and then restore the originally selected tab when possible.

### Chromium Browser Facade

Version 0.0.7 adds `Browser`, a high-level facade for a caller-provided Chromium
window. It reuses the existing tab and URL helpers, locates localized address
bars, and waits for a navigation change followed by stable UIA state.

```python
browser = auto.Chrome()
browser.Navigate("https://example.com")
browser.WaitPageLoad(timeout=30)

def on_focus_changed(sender):
    print("focused:", sender.Name)

handler = auto.CreateFocusChangedEventHandler(on_focus_changed)
auto.AddFocusChangedEventHandler(handler)
try:
    # Browser/UI Automation work...
    pass
finally:
    auto.RemoveFocusChangedEventHandler(handler)
```

`WaitPageLoad()` waits for the URL/title/document UIA state to change and then
remain stable for consecutive samples. It does not guarantee that every
background network request has completed. For automatic cleanup, use the
context-manager form:

```python
with browser.ListenFocusChanged(on_focus_changed):
    # Focus events are delivered while this block is active.
    pass
```

### Safe Top-Level Window Enumeration

Version 0.0.8 adds `GetTopLevelWindowHandles()` and `GetTopLevelControls()`.
They enumerate top-level windows through Win32 `EnumWindows` before optionally
converting each HWND to a UI Automation control. Prefer these helpers when
enumerating desktop windows instead of walking `GetRootControl().GetChildren()`;
individual UIA providers can still fail during `ControlFromHandle()`.

```python
handles = auto.GetTopLevelWindowHandles()
windows = auto.GetTopLevelControls()
```

Three more helpers print or return window inventories at different levels, which
makes them convenient for exploring a desktop before writing the actual script:

```python
# 全量扫描所有窗口：Win32 原生元数据，不依赖 UIA provider。
# 每项是一个 {'hwnd', 'title', 'className', 'processId', 'visible'} 字典。
for info in auto.GetAllTopLevelWindowInfos():
    print(info)

# UIA 控件列表：走 GetRootControl().GetChildren()，所以已经过滤掉了不支持
# UIA 协议的窗口和转换失败的窗口。它一边打印每个窗口的信息，一边返回列表。
windows = auto.GetTopLevelWindowControls()

# 只取隐藏的顶层窗口：Win32 枚举，只保留不可见但带标题的窗口。
for ctrl in auto.GetHiddenTopLevelWindowControls():
    print(ctrl.Name, ctrl.ClassName, ctrl.NativeWindowHandle)
```

`GetAllTopLevelWindowInfos(visibleOnly=True, includeOwned=True)` is the rawest
view: it enumerates HWNDs with Win32 `EnumWindows` and returns dictionaries with
`hwnd`, `title`, `className`, `processId`, and `visible`. Because it never asks a
UI Automation provider for anything, it also lists windows that have no UIA
support at all. Set `includeOwned=False` to drop owned windows such as popups and
modal dialogs.

`GetTopLevelWindowControls()` returns `Control` objects instead of dictionaries
and is the API to reach for when you want only windows that actually support UIA.
It walks `GetRootControl().GetChildren()`, so windows without a UIA provider and
windows whose conversion raised are skipped, and it prints a summary of every
window it keeps as it goes. `GetHiddenTopLevelWindowControls(search_title="",
process_id=None)` is the opposite filter: it collects only *invisible* top-level
windows that still have a title, skipping the usual system noise (`Default IME`,
`MSCTFIME UI`), and optionally narrows the result by a fuzzy title match or a
process id. Related: `GetTopLevelWindowControlByClassAndTitle()` returns the
first window matching a class name and/or title with the same fast Win32
enumeration.

### Hidden Windows and Restoring Them

Some applications (WeChat, QQ, and other tray-style apps) hide their main window
instead of closing it. A hidden window is not reachable from the desktop UIA
tree, so use the hidden-window list plus the native handle to restore it:

```python
import time

import winuia as auto


def restore_window():
    # 恢复隐藏的微信窗口
    # 获取隐藏窗口，只适用于有隐藏窗口的进程
    win_list = auto.GetHiddenTopLevelWindowControls()
    for i in win_list:
        if i.ClassName == 'Qt51514QWindowIcon':
            hwnd = i.NativeWindowHandle
            window: auto.WindowControl = auto.ControlFromHandle(hwnd)
            print(window.IsVisibleOnScreen())
            window.Restore()
            time.sleep(1)
            wx_restore()
            break
```

`GetHiddenTopLevelWindowControls()` returns `WindowControl` objects built with
`ControlFromHandle()`, so `NativeWindowHandle`, `ClassName`, `Name`, `Restore()`,
and `IsVisibleOnScreen()` all work on the results. Because a hidden window's UIA
subtree is often stale or incomplete, a hidden window can also be restored by
falling back to screen coordinates derived from its bounding rectangle:

```python
def wx_restore():
    wx = auto.WindowControl(ClassName='Qt51514QWindowIcon')
    print(wx)
    # 1. 查找最小化按钮
    rect = wx.BoundingRectangle
    # 假设最小化按钮在右上角往左偏移 100 像素的位置
    min_btn_x = rect.right - 100
    min_btn_y = rect.top + 20  # 往下偏移 20 像素
    print(f"预估最小化按钮位置: ({min_btn_x}, {min_btn_y})")

    wx.SetFocus()
    auto.MoveTo(min_btn_x, min_btn_y)
    # wx.DoubleClick(x=rect.right, y=rect.top)
    wx.Click(min_btn_x, min_btn_y)
    wx.Restore()
```

### Non-UIA Windows

`NonUIAWindow` is a separate HWND-based facade for windows that do not expose
reliable UI Automation controls. Use `WindowControl` and other `Control`
classes for UIA-capable applications; use `NonUIAWindow` for window-level
automation, screenshots, template matching, OCR, and screen-coordinate input.

```python
window = auto.NonUIAWindow(
    title_keyword="My App",
    class_name="MyWindowClass",
)

window.activate()
window.click(120, 48)  # relative to the window's top-left corner
window.send_keys("hello")
window.screenshot("window.png")

match = window.find_image("save.png", confidence=0.9)
window.click_image("save.png", timeout=5)
```

Pass `hwnd` when it is already known; it takes priority over keywords. Without
an HWND, `title_keyword` and `class_name` each match by exact value or
substring. A lookup that matches zero or multiple windows raises `LookupError`.

`to_bitmap()` tries `PrintWindow` first and falls back to a Win32/GDI screen
capture when needed. Image matching is automatically limited to the current
window rectangle. `click_image()` validates the latest window bounds before
clicking the matched center.

`close()` sends `WM_CLOSE` by default, so an application can close its window
while keeping background processes alive. Pass `terminateProcess=True` to
force-stop the owning process; add `killProcessTree=True` to stop its child
processes too. Forced termination does not save unsaved application data.

```python
window.close()
window.close(terminateProcess=True, killProcessTree=True)

# Stop a known application root process and all of its descendants.
auto.TerminateProcessTree(root_process_id)
```

If an application destroys its HWND after `hide()` or `set_topmost()`, those
methods return `False` instead of reporting a successful state change.

OCR is optional:

```powershell
pip install "winuia-auto[ocr]"
```

```python
match = window.find_text("OK", minScore=0.8, offscreen=True)
window.click_text("OK", minScore=0.8, offscreen=True)
```

If a window later exposes usable UIA controls, conversion is explicit and
optional:

```python
uia_window = window.to_uia_control()
if uia_window:
    uia_window.ButtonControl(Name="OK").Click()
```

```python
import winuia as auto

window = auto.WindowControl(
    ClassName="Chrome_WidgetWin_1",
    RegexName=".*Google Chrome",
)
browser = auto.Browser(window)

browser.navigate("https://example.com", timeout=30)
print(browser.get_current_url())
print(browser.get_current_tab().Name)

browser.select_tab("Example Domain")
browser.select_tab_by_url("example.com")
for tab in browser.get_all_tab_urls():
    print(tab["index"], tab["name"], tab["url"])
```

`Browser` currently targets Chromium windows such as Chrome and Edge. Pass the
window explicitly to avoid attaching to an unintended browser instance. Page
loading is inferred from URL, title, and document state; it is not a guarantee
that a web application has completed every background request. Renderer content
still requires the browser accessibility configuration described below.

P1 also provides factories that filter the matching Chromium window by process
name, plus browser-standard history commands:

```python
chrome = auto.Chrome(title="Example Domain")
edge = auto.Edge(title="Example Domain")

chrome.back()
chrome.forward()
chrome.reload()
document = chrome.get_current_document()
```

The factories raise `LookupError` when the requested executable is not running
or no window title matches. `get_current_document()` prefers a visible
`DocumentControl`; renderer content requires `--force-renderer-accessibility`.
When no browser history entry exists, `back()` and `forward()` return `False`
after the default one-second `no_change_timeout` rather than waiting for the
full navigation timeout. `reload()` waits for stable UIA state even when the
URL does not change; use `wait=False` when only issuing the shortcut matters.

The real browser integration test starts an isolated temporary profile and is
disabled by default:

```powershell
$env:UIAUTOMATION_RUN_BROWSER_INTEGRATION = "1"
$env:UIAUTOMATION_BROWSER = "chrome" # or "edge" or "firefox"
python tests\test_browser_integration.py
```

### Firefox Profile and Page Extraction

Firefox can use the same facade after locating its top-level window and passing
`FIREFOX_PROFILE`. Its address-bar profile supports the standard labels and
the variable search-engine label used by Firefox.

```python
firefox_window = auto.WindowControl(
    ClassName="MozillaWindowClass",
    SubName="Example Domain",
)
firefox = auto.Browser(firefox_window, profile=auto.FIREFOX_PROFILE)
```

Extract UIA content from the active document:

```python
links = browser.get_all_links()
for link in links:
    print(link.Name)

text = browser.get_all_text()
print(text)
```

`get_all_text()` prefers `TextPattern` and falls back to visible `TextControl`
names. Chromium may expose those controls through the window tree rather than
as direct document children; the facade filters that fallback to controls with
a `DocumentControl` ancestor. UIA text is not a DOM serialization and may omit
content that the browser accessibility provider does not expose.

### Browser Tabs and JavaScript Modes

Use browser-standard shortcuts to create or close tabs. `new_tab(url)` opens a
URL in the newly active tab. `close_tab()` closes the active tab by default,
closes one zero-based index, or closes every tab matching title and URL filters.

```python
browser.new_tab()
browser.new_tab("https://example.com")
browser.close_tab()
browser.close_tab(2)
browser.close_tab(
    "Example Domain",
    url="https://example.com/orders/42",
    exact=True,
)
```

For `close_tab()`, an integer `tab` is always treated as the tab index and
ignores `url`, so it closes only that tab. A non-empty string `tab` closes all
title matches; add a non-empty `url` to require both values. Pass `tab=""`
with a URL to close all URL matches. Matching tabs are closed in descending
index order so index changes after each close do not affect later matches.
Passing both `tab=""` and `url=""` raises `ValueError`.

For explicit page-side automation, the default JavaScript mode enters a
`javascript:` URL through the address bar:

```python
browser.execute_javascript("document.title = 'Ready'")
```

Use `method="console"` when the script must be issued through DevTools. It
focuses the Console prompt and types the script directly, avoiding DevTools
paste protection. If this call opens DevTools, it closes it after execution by
default; set `close_console=False` to leave it open. A Console that was already
open before the call is never toggled or closed.

```python
browser.execute_javascript(
    "document.title = 'Ready'",
    method="console",
    close_console=True,
)
```

Set `paste=True` to use the text clipboard instead. The facade restores the
prior text clipboard value before returning. Some DevTools sessions block
pasting until the user enters `allow pasting`; pass `allow_pasting=True` to
attempt that input first:

```python
browser.execute_javascript(
    "document.title = 'Ready'",
    method="console",
    paste=True,
    allow_pasting=True,
)
```

Chrome may intentionally reject an automated `allow pasting` entry as part of
its self-XSS protection. In that case, type the phrase once in the DevTools
console manually, then call the `paste=True` mode. Neither mode retrieves a
JavaScript return value or bypasses browser security policy. The caller is
responsible for the script's effects.

### pynput Hotkey Helper

```python
auto.Press_Hotkey("ctrl", "a")
auto.Press_Hotkey("backspace")
auto.Press_Hotkey("ctrl", "shift", "a")
auto.Press_Hotkey("shift", "f10")

edit.Press_Hotkey("ctrl", "a")
```

`pynput` is installed automatically with `winuia-auto`.
All named `pynput` keys can be passed as lowercase strings, including
`f1` through `f24`, `print_screen`, `caps_lock`, and media keys.

### Screenshot, Highlight, and UI Tree Export

New debugging and reporting helpers:

```python
control.Screenshot("control.png")
control.Screenshot("control_offscreen.png", offscreen=True)
control.Highlight()

data = control.ToDict()
control.ExportTreeToJson("tree.json", maxDepth=5)
control.ExportTreeToMarkdown("tree.md", maxDepth=5)
```

`offscreen=True` uses Windows `PrintWindow` to capture native windows even when they are covered by other top-level windows. Some hardware-accelerated surfaces such as video, games, WebView, and parts of modern browsers may still return blank or stale content because the target application must support offscreen painting.

Module-level tree export helpers:

- `ControlToDict()`
- `ControlTreeToDict()`
- `ExportControlTreeToJson()`
- `ControlTreeToMarkdown()`
- `ExportControlTreeToMarkdown()`

### Practical Browser Automation Example

The following example shows how the newer helpers can be used together in a browser-like window:

```python
import winuia as auto

def get_browser_object(shop_name, class_name=None):
    # 获取对象
    doc_windows = find_windows_by_class_and_title(class_name, shop_name)
    if len(doc_windows) == 0:
        return False
    window = auto.ControlFromHandle(doc_windows[0])
    if window.Exists():
        print(f"成功获取窗口控制对象2: {window2}")
    return window

def get_all_window_controls():
    # 1. Safely enumerate visible top-level windows through Win32.
    top_windows = auto.GetTopLevelControls()
    print(f"共找到 {len(top_windows)} 个顶级窗口：\n")
    # 2. 遍历并打印每个窗口的信息
    for i, window in enumerate(top_windows):
        try:
            print(f"【窗口 {i + 1}】")
            print(f"  窗口标题: {window.Name}")
            print(f"  类名: {window.ClassName}")
            print(f"  句柄: {window.NativeWindowHandle}")
            print(f"  控件类型: {window.ControlType}")
            print(f"  是否可见: {window.Exists()}")
            print("-" * 50)
            print(window.GetChildren())
        except Exception as e:
            print(f"读取窗口失败: {e}")

auto.SetDpiAwareness()

window = auto.WindowControl(ClassName="Chrome_WidgetWin_1", RegexName=".*Google Chrome")

# Enumerate tabs and read their URLs.
tabs = window.FindAll(auto.ControlType.TabItemControl, maxDepth=10)
for tab in tabs:
    url = tab.GetTabUrl(window=window)
    print(tab.Name, url)

# Read all tab URLs in one call. This switches tabs and restores the original tab.
tab_urls = window.GetAllTabUrls()
for item in tab_urls:
    print(item["index"], item["name"], item["url"])

# Switch tabs by title or by URL substring.
window.SwitchToTabByName("PyPI")
window.SwitchToTabByUrl("pypi.org")

# Read the active tab URL from the browser address bar.
url = window.GetCurrentUrl()
print(url)

# Use pynput-backed hotkeys on a focused control.
window = auto.WindowControl(ClassName='Chrome_WidgetWin_1', RegexName='.*Google Chrome')
control = window.EditControl(RegexName='地址和搜索栏')
control.Press_Hotkey("ctrl", "a")
control.Press_Hotkey("backspace")
control.SendKeys('https://pypi.org/project/winuia-auto/')
# control.Press_Hotkey("ctrl", "v")
control.Press_Hotkey("enter")

# Safely invoke a covered button; SafeClick prefers InvokePattern and falls back to mouse click.
window2 = get_browser_object('万达云')
# 控件隐藏也能点击到
control = window2.ButtonControl(RegexName='智能分流') #
print(control.IsVisibleOnScreen())
control.SafeClick()

# Capture a control even when another top-level window covers it.
address_bar.Screenshot("address_bar.png", offscreen=True)
address_bar.Highlight()

# Export UI tree data for debugging or reports.
window.ExportTreeToJson("tree.json", maxDepth=5)
window.ExportTreeToMarkdown("tree.md", maxDepth=5)

```

You can also start from a native window handle, which is useful when a window is easier to locate with Win32 APIs:

```python
import win32gui
import winuia as auto

def find_windows_by_class_and_title(class_name=None, title_keyword=""):
    handles = []

    def callback(hwnd, _):
        current_class = win32gui.GetClassName(hwnd)
        title = win32gui.GetWindowText(hwnd)
        if class_name:
            matched = current_class == class_name and title_keyword in title
        else:
            matched = title_keyword in title
        if matched:
            handles.append(hwnd)
        return True

    win32gui.EnumWindows(callback, 0)
    return handles

handles = find_windows_by_class_and_title("Chrome_WidgetWin_1", "Google Chrome")
if handles:
    window = auto.ControlFromHandle(handles[0])
    print(window.Name, window.ClassName, window.NativeWindowHandle)
```

### Image Template Matching

```python
match = auto.FindImageOnScreen("button.png", confidence=0.9)
match = auto.WaitImageAppear("button.png", timeout=10)
auto.ClickImage("button.png", timeout=10)

# The template can also be encoded image bytes.
with open("button.png", "rb") as f:
    image_bytes = f.read()
match = auto.FindImageOnScreen(image_bytes, confidence=0.9)

# A captured Bitmap can be converted to PNG bytes directly.
root = auto.GetRootControl()
bitmap = root.ToBitmap(100, 200, 500, 300)
if bitmap:
    with bitmap:
        png_bytes = bitmap.Bytes
        # Use bitmap.ToBytes('jpg') or bitmap.ToBytes('BGRA') for other formats.

# DPI/resolution-tolerant multi-scale matching.
match = auto.FindImageOnScreen(
    "button.png",
    confidence=0.8,
    multiScale=True,
    maxScaleDiff=0.3,
    scaleStep=0.05,
)
auto.ClickImage("button.png", timeout=10, multiScale=True)

# Reuse robust matching settings across multiple operations.
matcher = auto.RobustTemplateMatch(threshold=0.7, maxScaleDiff=0.3)
matcher.click_by_template("button.png", timeout=10)


def image_click():
    path = r'C:\Users\YHCX\Desktop\旧\ScreenShot_2026-07-27_165832_374.png'
    match_result = auto.FindImageOnScreen(path, confidence=0.85)
    print(match_result)
    
    # 1. 解包数据
    x, y, w, h, confidence = match_result

    # 2. 计算中心点
    center_x = x + w // 2  # 780
    center_y = y + h // 2  # 161
    
    # 3. 移动鼠标
    # 方法 A: 如果你的 auto 库有 MoveTo 方法 (根据你之前的代码风格推测)
    auto.MoveTo(center_x, center_y)
    
    # 4. 移动鼠标到坐标点进行点击
    auto.Click(center_x, center_y)

    # 5. 静默点击这个坐标
    auto.SafeClick(center_x, center_y)

    # 6. 传入图片路径 -> 找到图片所在坐标 -> 进行点击
    auto.ClickImage(path)
```

Multi-scale matching searches around the current Windows DPI scale and also tests the
original template size. It returns screen coordinates correctly for restricted regions
and virtual desktops. This is useful for custom-rendered UI that does not expose reliable
UIAutomation controls. OpenCV and NumPy are installed automatically with the package.

### RapidOCR Text Position Clicks

OCR helpers are useful for UIA-invisible text, self-drawn controls, web pages, Canvas, and image-like buttons. Install the OCR extra when needed:

```powershell
pip install "winuia-auto[ocr]"
```

Create a reusable RapidOCR instance. `params` are initialization parameters for
`RapidOCR(params=...)`; they are merged with the library defaults, which use
ONNXRuntime for detection, classification, and recognition:

```python
ocr = auto.CreateRapidOCR(
    params={
        "Global.text_score": 0.8,
        "Global.log_level": "warning",
        "Rec.lang_type": "en",
    },
)
```

Call `rapidOCR_img()` directly when the raw RapidOCR `result_json` is needed.
Its `params` argument configures engine creation, while keyword arguments are
passed to the recognition call:

```python
result_json = auto.rapidOCR_img(
    "control.png",
    params={"Global.text_score": 0.8},
    use_cls=False,
    text_score=0.8,
)
print(result_json)
# [{"box": [[10, 20], [120, 20], [120, 50], [10, 50]],
#   "txt": "Login", "score": 0.998}]
```

Recognize normalized text positions from an image file:

```python
positions = auto.OCRImageTextPositions(
    "control.png",
    text="Login",
    minScore=0.8,
    ocr=ocr,
    predictArgs={"use_cls": False, "text_score": 0.8},
)

for item in positions:
    print(item["text"], item["score"], item["box"], item["screen_center"])
```

One-step OCR text click on a control:

```python

window = auto.WindowControl(ClassName="Chrome_WidgetWin_1", RegexName=".*Google Chrome")

match = auto.OCRClickText("登录", window, minScore=0.8, offscreen=True)
print(match["text"], match["screen_center"]) if match else print("not found")

# Equivalent control method.
match = window.OCRClickText("登录", minScore=0.8, offscreen=True)
```

For tuning or reuse:

```python
def show_window(handle):
    """ cmdShow 值及效果
    SW_HIDE     0   隐藏窗口（最常用，让程序在后台运行不显示界面）
    SW_SHOW     5   显示窗口（如果之前被隐藏了，就恢复显示）
    SW_MINIMIZE 6   最小化窗口到任务栏
    SW_MAXIMIZE 3   最大化窗口（全屏）
    SW_RESTORE  9   还原窗口（从最小化或最大化恢复到之前的大小）
    :return:
    """
    # 展示最初的桌面
    # auto.ShowDesktop()
    # 显示窗口的状态
    # auto.ShowWindow(handle, cmdShow)

def text_click():
    # 使用文本进行点击
    window = auto.WindowControl(ClassName='Chrome_WidgetWin_1', RegexName='.*Google Chrome')
    control = window.DocumentControl(Name='搜索 - Microsoft 必应')
    match = auto.OCRClickText(
        "国际版",
        control,
        minScore=0.8,
        # 如果控件/窗口被其他顶层窗口遮挡
        offscreen=True,
        ocrArgs={
            "Global.text_score": 0.8,
            "Rec.lang_type": "ch",
        },
        # RapidOCR 本次识别参数
        predictArgs={
            "use_cls": False,
            "text_score": 0.8,
        },
        # SafeClick 开启由Click进行兜底点击
        safeClickArgs={
            "fallbackToClick": True,
        },
    )
    print(match)
```

`OCRClickText()` screenshots the control through `Screenshot()` / `ToBitmap()`, runs RapidOCR, matches text, converts OCR image coordinates to screen coordinates, and clicks with `SafeClick()`. `center` is the OCR center inside the screenshot image; `screen_center` is `center` plus the screenshot's actual desktop offset, so it can be used for Windows screen clicks. `ocrArgs={...}` are passed to `RapidOCR(params=...)`, or create and reuse an OCR instance with `CreateRapidOCR(params=...)`. `predictArgs={...}` are passed to the RapidOCR recognition call, for example `use_cls`, `use_det`, `use_rec`, `text_score`, `box_thresh`, and `unclip_ratio`. OCR results are normalized to dictionaries with `text`, `score`, `box`, `center`, `screen_box`, and `screen_center`.

### XPath Queries with `UIXPath`

Version 0.1.5 adds `UIXPath`, an XPath-style query engine for the UI Automation
tree. Simple XPaths (descendant paths with attribute predicates) are resolved by
walking `Control` objects directly, which is fast; anything using axes, complex
predicates, or structural `and`/`or` falls back to building an XML snapshot of
the tree and evaluating it with `lxml`. The `lxml` fallback is needed for those
complex queries, so install it with `pip install lxml` if it is not already
available.

```python
import winuia as auto
from winuia import UIXPath

window = auto.WindowControl(ClassName='Chrome_WidgetWin_1', RegexName='.*Google Chrome')
xpath = UIXPath(window)

# exists() is the cheapest way to test a condition.
if xpath.exists("//Button[@AutomationId='signInSubmit']"):
    xpath.find("//Button[@AutomationId='signInSubmit']").Click()

# find_all() returns every match, in tree order.
edits = xpath.find_all("//Edit[contains(@AutomationId, 'ap_')]")
print([e.AutomationId for e in edits])

# Attribute predicates match the UIA properties exposed as XML attributes, and a
# single predicate can accept both English and localized names.
continue_btn = xpath.find("//Button[@Name='Continue' or @Name='继续']")
```

`UIXPath(root=None, max_depth=None, cache=True)` starts from `root`, or from
`auto.GetRootControl()` when no root is given. `find()` / `find_first()` return
the first matching `Control` or `None`; `find_all()` / `find_all_by_xpath()`
return a list; `exists()` returns a bool; and `refresh()` discards the cached XML
tree so the next complex query rebuilds it. `dump_xml()` and `save_xml()` export
the current tree as XML for debugging, and `get_xpath(control, mode="short")`
derives an XPath for an existing control. The same helpers are available as
module-level functions in `winuia.uia_xpath`: `find_by_xpath()`,
`find_all_by_xpath()`, `xpath_exists()`, and `dump_ui_xml()`.

A full example that drives a Google sign-in flow entirely through XPath:

```python
import time

import winuia as auto
from winuia import UIXPath


def login_judge_xpath():
    # 判断是否需要登录
    window = auto.WindowControl(ClassName='Chrome_WidgetWin_1', RegexName='.*Google Chrome')
    xpath = UIXPath(window)

    text2 = xpath.find("//Text[contains(@Name, 'Sign in or crea') or contains(@Name, '登录')]")
    time.sleep(5)
    if not text2.Exists(1):
        return

    # 输入邮箱
    email = 'xxxxxx'
    input_email = xpath.find("//Edit[@AutomationId='ap_email_login']")

    input_email.Press_Hotkey('ctrl', 'a')
    input_email.Press_Hotkey('backspace')

    auto.SetClipboardText(email)
    input_email.SetFocus()
    input_email.Press_Hotkey('ctrl', 'v')

    next_btn = xpath.find("//Button[@Name='Continue' or @Name='继续']")
    next_btn.Click()

    time.sleep(1.5)
    # windows 安全中心
    cancel_btn = xpath.find("//Button[@Name='Cancel' or @Name='取消']")
    if cancel_btn.Exists(1):
        cancel_btn.Click()
        time.sleep(0.5)
        next_btn.Click()

    for i in range(10):
        pwd_edit = xpath.find("//Edit[@AutomationId='ap_password']")
        if pwd_edit.Exists(1):
            break
        time.sleep(1)

    # 输入密码
    password = 'xxxxxx'
    pwd_edit = xpath.find("//Edit[@AutomationId='ap_password']")
    pwd_edit.SetFocus()
    auto.SetClipboardText(password)
    pwd_edit.Press_Hotkey('ctrl', 'v')
    # 点击登录
    log_btn = xpath.find("//Button[@AutomationId='signInSubmit']")
    log_btn.Click()

    # 保存密码弹窗
    for i in range(8):
        no_use = window.ButtonControl(Name='不用了')
        if no_use.Exists(1):
            no_use.Click()
            break
        time.sleep(1)
```

`find()` returns `None` when nothing matches, so either guard the result (`if control: ...`)
or prefer `xpath.exists("//...")` when you only need to test for a condition.
Predicates use the same property names as the `Control` API, so
`@AutomationId`, `@Name`, and `@ClassName` can be mixed with `contains()` for
partial matching, and a single predicate can accept several localized strings
with `or`.

### Element Inspector (`InspectElement`)

Version 0.1.5 adds a built-in element inspector, which replaces the external
Windows SDK `Inspect.exe` that older releases shipped in an `inspect/` folder.

```python
import winuia

winuia.InspectElement()
```

It opens a Tkinter window with the live UI Automation tree on the left and the
selected element's details on the right:

- Hold `Ctrl` and hover the mouse over a control. After a short hover pause the
  inspector captures the control under the cursor and loads the full UI tree of
  its top-level window.
- The left pane shows the UI tree and filters it live through the search box;
  selecting a node selects the matching element.
- The right pane lists properties such as `Name`, `AutomationId`, `ClassName`,
  `ControlType`, and `BoundingRectangle`, and highlights the element on screen.
- The XPath box generates an XPath for the selected element and runs
  find / find-all queries live, with previous/next navigation between matches.
  Right-click the tree or the property list to copy an XPath or a property value.

The generated XPath values come from the same properties that `UIXPath` matches
on, so the usual workflow is: inspect the element, copy its XPath, then reuse that
XPath in the automation script.
