Metadata-Version: 2.4
Name: alphawindow
Version: 0.1.1
Summary: Windows-first window automation abstractions with pluggable input and output modes.
Requires-Python: >=3.11
Description-Content-Type: text/markdown
Provides-Extra: native
Requires-Dist: pywin32>=306; platform_system == "Windows" and extra == "native"
Requires-Dist: pynput>=1.7.6; platform_system == "Windows" and extra == "native"
Requires-Dist: winrt-Windows.Foundation>=3.2.1; platform_system == "Windows" and extra == "native"
Requires-Dist: winrt-Windows.Graphics>=3.2.1; platform_system == "Windows" and extra == "native"
Requires-Dist: winrt-Windows.Graphics.Capture>=3.2.1; platform_system == "Windows" and extra == "native"
Requires-Dist: winrt-Windows.Graphics.Capture.Interop>=3.2.1; platform_system == "Windows" and extra == "native"
Requires-Dist: winrt-Windows.Graphics.DirectX>=3.2.1; platform_system == "Windows" and extra == "native"
Requires-Dist: winrt-Windows.Graphics.DirectX.Direct3D11>=3.2.1; platform_system == "Windows" and extra == "native"
Requires-Dist: winrt-Windows.Graphics.DirectX.Direct3D11.Interop>=3.2.1; platform_system == "Windows" and extra == "native"
Provides-Extra: test
Requires-Dist: pytest>=8; extra == "test"

# AlphaWindow

AlphaWindow is a Windows-first automation abstraction layer. It exposes one session API for window operations while allowing callers to choose different input and output modes.

AlphaWindow is meant to model target-window automation, not global "computer use" control. The same high-level operations can be backed by no-op inspection, window messages, UI Automation, foreground input, guarded foreground input, or optional hook-assisted virtual input.

## Scope

AlphaWindow separates four concerns:

- Window discovery and state: `WindowResolver`, `WindowSelector`, and `WindowSnapshot`.
- Output: state inspection, direct window capture, desktop-region capture, hook telemetry, or no output.
- Input: dry-run, window messages, UIA, global foreground input, guarded foreground input, isolated hook input, or virtualized hook input.
- Safety contracts: profile compatibility, explicit fallback modes, hook requirements, prefetch TTL, and fake backends for deterministic tests.

The project currently focuses on the abstraction and compatibility layer. Native Win32, pynput, UIA, and hook implementations are intentionally kept behind protocols so callers can plug in their own adapters without changing automation code. The exception is an experimental PyWinRT-backed Windows Graphics Capture path that can capture a native Direct3D surface, read it back to CPU memory, or keep a persistent frame cache for low-latency repeated screenshots.

## Supported Today

- Built-in I/O profiles for every declared input and output mode, including `inspect`, `observe`, `desktop_observe`, `dry_run`, `win32_message`, `message_only`, `uia_control`, `uia_visual`, `global_input`, `foreground_visual`, `guarded_input`, `guarded_desktop`, `isolated_observe`, `isolated_telemetry`, `virtual_window`, `virtual_telemetry`, and `virtual_input_only`.
- A `WindowSession` API for `prefetch`, `capture`, `observe`, `click`, `key_down`, `key_up`, and `type_text`.
- Generic delegation backends for message, UIA, foreground, guarded foreground, state, desktop-region, hook-telemetry, and no-output modes.
- Compatibility checks so a backend cannot silently fall back to a broader side-effect mode.
- Hook-aware `isolated` and `virtualized` input backends that accept pluggable hook managers and virtual input transports.
- Deterministic prefetch behavior: snapshots are reused within `prefetch_ttl_seconds`, rejected when stale, and revalidated through a resolver `is_valid` hook when provided.
- DPI-aware snapshots and capture requests: `WindowSnapshot.dpi` is carried into operation normalization and DirectX capture requests.
- Relative point normalization: `session.click(0.5, 0.5)` resolves against the current target window rectangle.
- HDR-aware DirectX/Windows Graphics Capture model: `DirectXCaptureOutputBackend` builds capture requests for an injected `DirectXCapturePipeline`, selects FP16/scRGB for HDR targets, selects BGRA8/sRGB for SDR, and preserves frame DPI and SDR white-level metadata.
- CPU frame conversion: `CpuFrame` supports FP16/scRGB to SDR BGRA8 tone mapping and pure-Python PNG export.
- Experimental native WGC pipeline: `WinrtWgcCapturePipeline` uses PyWinRT, creates a Direct3D11 device, creates a `GraphicsCaptureItem` from `hwnd`, starts a free-threaded WGC frame pool, returns a `CaptureFrame` containing the native `IDirect3DSurface`, and can read it back with `capture_cpu()`.
- Persistent native WGC frame cache: `WinrtWgcFrameCache` keeps the WGC session and D3D11 device alive, tracks the newest frame, can wait for a bounded fresh frame, and can run a `before_readback` hook while holding that frame so external labels can be sampled against the same image moment.
- PyPI packaging for `pip install alphawindow` and typed package data via `py.typed`.
- Tests that cover profile coverage, compatibility failures, prefetch reuse/refresh behavior, hook backend contracts, DirectX/HDR capture contracts, native WGC dependency behavior, generic backends, packaging import behavior, and the publish workflow.

## Built-in Profiles

| Profile | Input mode | Output mode | Current behavior |
| --- | --- | --- | --- |
| `inspect` | `dry_run` | `state` | Inspect resolver state only. |
| `observe` | `dry_run` | `window_capture` | Observe/capture without input. |
| `desktop_observe` | `dry_run` | `desktop_region` | Observe through desktop-region output. |
| `dry_run` | `dry_run` | `none` | Record planned operations without side effects. |
| `win32_message` | `message` | `window_capture` | Adapter slot for window-message input plus window capture. |
| `message_only` | `message` | `none` | Adapter slot for message input without output. |
| `uia_control` | `uia` | `state` | Adapter slot for UIA control operations. |
| `uia_visual` | `uia` | `window_capture` | Adapter slot for UIA input plus visual capture. |
| `global_input` | `foreground` | `desktop_region` | Adapter slot for pynput/SendInput-style foreground input. |
| `foreground_visual` | `foreground` | `window_capture` | Foreground input with target-window output. |
| `guarded_input` | `guarded_foreground` | `window_capture` | Adapter slot for foreground input with focus/cleanup guards. |
| `guarded_desktop` | `guarded_foreground` | `desktop_region` | Guarded foreground input with desktop-region capture. |
| `isolated_observe` | `isolated` | `window_capture` | Requires a hook-capable input backend; can isolate real mouse input before optional delegated input. |
| `isolated_telemetry` | `isolated` | `hook_telemetry` | Requires hook support for isolation and telemetry. |
| `virtual_window` | `virtualized` | `window_capture` | Requires hook support and a virtual input transport. |
| `virtual_telemetry` | `virtualized` | `hook_telemetry` | Requires hook support for virtual input and telemetry. |
| `virtual_input_only` | `virtualized` | `none` | Requires hook support; routes virtual input without output capture. |

## Considered Design Surface

These capabilities are represented in the model and can be supplied by external adapters today:

- Native window resolution by `hwnd`, title, class name, PID, visibility, DPI, and process/thread metadata.
- Direct window output such as `PrintWindow`, DWM thumbnails, BitBlt, or application-specific capture.
- HDR-aware DirectX/Windows Graphics Capture output with explicit pixel format, color space, buffer count, free-threaded frame-pool preference, DPI, and SDR white-level metadata.
- Native WGC capture through PyWinRT for callers that want an `IDirect3DSurface`, a tone-mapped CPU image, or a persistent latest-frame cache for high-frequency capture loops.
- Desktop-region output for modes where the target must be foreground or compositor-visible.
- Window-message input for controls that accept `PostMessage`/`SendMessage` style input.
- Windows UI Automation input for standard controls.
- Foreground input through `pynput`, `SendInput`, or equivalent global input libraries.
- Guarded foreground input that restores focus/cursor state after an operation.
- Optional hook-assisted mouse and keyboard windowization, including target-local telemetry.
- Per-window or per-process virtual input transports for multi-window automation without taking over the primary mouse.

## Not Supported Yet

The package still keeps most native adapters out of core. In particular, it does not currently include:

- A built-in pywin32 `WindowResolver`.
- Color-managed HDR preview or HDR-preserving image/video encoders.
- Advanced tone-mapping controls beyond the built-in ACES-style SDR conversion.
- Built-in `PostMessage`, UIA, pynput, or SendInput input backends.
- A production DLL injection or hook manager.
- A ready-made virtual input IPC transport.
- OCR, image matching, accessibility tree querying, or high-level wait/retry helpers.
- Non-Windows native backends.
- Kernel drivers, privilege escalation, protected-process bypasses, anti-cheat bypasses, or game-specific automation logic.

Hook-capable profiles fail closed unless the provided backend advertises hook capability. `VirtualizedInputBackend` also requires a `VirtualInputTransport`.

## Planned Features

- pywin32-based resolver and state backend for common window selectors.
- Native capture adapters for classic window capture and desktop-region capture.
- Broader Windows Graphics Capture resize handling.
- HDR-preserving image encoders and richer color-management metadata.
- pynput/SendInput foreground input backend and guarded foreground focus restoration.
- Window-message and UIA adapters for common controls.
- A documented hook manager interface with a sample user-mode hook implementation.
- A sample virtual input transport suitable for target-window mouse/keyboard windowization.
- More examples showing how to share one automation script across dry-run, message, foreground, isolated, and virtualized modes.
- Expanded integration tests for real Windows adapters while keeping unit tests deterministic through fake backends.

## Install

From PyPI:

```powershell
pip install alphawindow
```

From this checkout:

```powershell
pip install .
```

Native Windows adapters are optional:

```powershell
pip install "alphawindow[native]"
```

## Import

```python
import alphawindow

from alphawindow import (
    AutomationConfig,
    DirectXCaptureConfig,
    WgcFrameReadbackResult,
    WinrtWgcCapturePipeline,
    WinrtWgcFrameCache,
    InputMode,
    OutputMode,
    WindowSelector,
    WindowSession,
    get_profile,
)

print(alphawindow.__version__)
print(get_profile("observe").input_mode is InputMode.DRY_RUN)
print(DirectXCaptureConfig().hdr_mode.value)
print(WgcFrameReadbackResult.__name__)
print(WinrtWgcCapturePipeline.__name__)
print(WinrtWgcFrameCache.__name__)
```

## Native WGC Screenshot

```python
from alphawindow import DirectXCaptureConfig, HdrCaptureMode, WindowSnapshot
from alphawindow.native_wgc import WinrtWgcCapturePipeline

target = WindowSnapshot(
    hwnd=123456,
    title="Target",
    rect=(0, 0, 1920, 1080),
    dpi=144,
    metadata={"hdr_enabled": True, "sdr_white_level_nits": 203.0},
)
request = DirectXCaptureConfig(hdr_mode=HdrCaptureMode.AUTO).create_request(target)

pipeline = WinrtWgcCapturePipeline(timeout_seconds=2.0)
frame = pipeline.capture_cpu(target, request, tone_map=True)
frame.save_png("target.png")
pipeline.close()
```

For repeated capture, keep the WGC session alive and read back the latest frame:

```python
from alphawindow.native_wgc import WinrtWgcFrameCache

def sample_labels(frame_info):
    return {"sequence": frame_info["sequence"], "labels": []}

cache = WinrtWgcFrameCache.start_for(target, request, timeout_seconds=2.0)
try:
    result = cache.readback_latest_cpu(
        tone_map=True,
        fresh_max_age_ms=8.0,
        fresh_wait_ms=16.0,
        before_readback=sample_labels,
    )
    result.cpu.save_png("target.png")
    print(result.info["size"], result.info["content_size"], result.hook_result)
finally:
    cache.close()
```
