Metadata-Version: 2.4
Name: aliax
Version: 1.0.0
Summary: Aliax SDK — real-time interceptor middleware for visual web agents (Set-of-Mark + deterministic execution).
Author-email: Aliax <hello@aliax.xyz>
License: MIT License
        
        Copyright (c) 2026 Aliax
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in
        all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
        THE SOFTWARE.
        
Project-URL: Homepage, https://aliax.xyz
Project-URL: Documentation, https://aliax.xyz/docs
Project-URL: Source, https://github.com/aliax/aliax-sdk
Project-URL: Tracker, https://github.com/aliax/aliax-sdk/issues
Project-URL: Changelog, https://github.com/aliax/aliax-sdk/blob/main/CHANGELOG.md
Keywords: ai,agents,web-agents,playwright,vlm,set-of-mark,browser-automation,llm-tools
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx<2,>=0.24
Requires-Dist: playwright<2,>=1.30
Provides-Extra: test
Requires-Dist: pytest>=7; extra == "test"
Requires-Dist: pytest-asyncio>=0.21; extra == "test"
Requires-Dist: respx>=0.20; extra == "test"
Dynamic: license-file

# Aliax SDK

[![PyPI version](https://img.shields.io/pypi/v/aliax.svg?style=flat-square&color=10B981)](https://pypi.org/project/aliax/)
[![Python versions](https://img.shields.io/pypi/pyversions/aliax.svg?style=flat-square&color=3B82F6)](https://pypi.org/project/aliax/)
[![CI](https://img.shields.io/github/actions/workflow/status/aliax/aliax-sdk/test.yml?branch=main&style=flat-square&label=tests)](https://github.com/aliax/aliax-sdk/actions/workflows/test.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-10B981.svg?style=flat-square)](LICENSE)

**Aliax** is the real-time interceptor middleware for visual web agents.
Translate the live page into a numbered Set-of-Mark image, hand it to
your VLM, and let Aliax execute the action it picks — clicks, types,
scrolls, the works — with deterministic accuracy.

When the agent still gets stuck, ship the failure to the offline
annotation queue with `capture_failure()`.

**Current version:** `1.0.0` · **Python:** 3.8 – 3.12 · [Changelog](CHANGELOG.md) · [Contributing](CONTRIBUTING.md)

## Installation

```bash
pip install aliax
playwright install chromium
```

## The 3-line agent loop

```python
from aliax import Aliax
from playwright.async_api import async_playwright

aliax = Aliax(api_key="sk_live_...")   # mint one at /api-keys

async with async_playwright() as p:
    page = await (await p.chromium.launch()).new_page()
    await page.goto("https://shop.example.com/cart")

    # 1. Translate the live DOM into a VLM-friendly Set-of-Mark.
    ctx = await aliax.parse_ui(page)

    # 2. Ask your VLM. It sees a numbered screenshot + a tiny JSON map.
    decision = await ask_llm(image=ctx.image_bytes, map=ctx.elements)
    # e.g. {"action": "CLICK", "element_id": "el_41"}

    # 3. Aliax executes it natively — scroll-into-view, React onChange,
    #    iframe coords, retina DPR, all handled.
    await aliax.execute(page, decision)
```

That's the entire happy path. No XPath wrangling, no Playwright
boilerplate, no pixel guessing.

## The safety net — `capture_failure()`

When the agent loops on a popup or hallucinates past recovery, snapshot
the full state and ship it to the Aliax annotation queue for human
correction:

```python
await aliax.capture_failure(
    page,
    goal="Close newsletter popup",
    thoughts=agent.current_reasoning,
    last_attempted_action={"action": "CLICK", "element_id": "el_41"},
    failure_reason="modal_blocked",
    step=agent.loop_step,
)
```

Each failure becomes a (negative, positive) DPO pair — the agent's
"stupid move" + the annotator's corrected tap — ready for fine-tuning.

## `parse_ui()` — what you get back

```python
ctx = await aliax.parse_ui(page)
```

| Attribute        | Type            | Description |
|------------------|-----------------|-------------|
| `image_bytes`    | `bytes`         | Native browser screenshot (JPEG by default — see `render_config`) with numbered Set-of-Mark boxes painted by the bundled JS overlay. Zero Python image processing. |
| `image_mime`     | `str`           | `"image/jpeg"` by default, `"image/png"` if you opt in. |
| `image_size`     | `(int, int)`    | `(width, height)` in physical pixels, parsed from the image header. |
| `elements`       | `list[dict]`    | Each element: `element_id`, `tag`, `role`, `text`, `bounds`, `editable`, `is_canvas`, `attrs`. |
| `viewport`       | `dict`          | `{width, height, dpr, scroll_x, scroll_y}` in CSS px. |
| `url`            | `str`           | Page URL at capture time. |

### Power-user: shrink your VLM bill with `render_config`

VLM providers bill image tokens by file weight + dimensions. JPEG at
`quality=40` costs roughly **10× fewer tokens** than the lossless
default. The Set-of-Mark IDs stay readable because they're rendered as
crisp DOM text by Chromium *before* the JPEG encoder runs.

```python
# Default — pristine JPEG at q=80.
ctx = await aliax.parse_ui(page)

# Token-optimised — same readable IDs, ~10× cheaper per VLM call.
ctx = await aliax.parse_ui(
    page,
    render_config={"format": "jpeg", "quality": 40},
)

# Lossless when you need it (research / fine-tuning datasets).
ctx = await aliax.parse_ui(
    page,
    render_config={"format": "png"},
)
```

Quality is clamped to `[30, 100]` so a typo like `quality=5` can't turn
the numbered boxes into illegible mush and crash your agent loop.

Convenience helper:

```python
prompt = f"""
{ctx.llm_text_block()}
Pick the element_id to click. Reply JSON {{action, element_id}}.
"""
```

## `execute()` — the action verbs

```python
await aliax.execute(page, {"action": "CLICK",    "element_id": "el_41"})
await aliax.execute(page, {"action": "TYPE",     "element_id": "el_7", "value": "Nike Shoes"})
await aliax.execute(page, {"action": "HOVER",    "element_id": "el_14"})
await aliax.execute(page, {"action": "PRESS",    "element_id": "el_7", "key": "Enter"})
await aliax.execute(page, {"action": "SCROLL",   "dy": 600})
await aliax.execute(page, {"action": "NAVIGATE", "url": "https://..."})
await aliax.execute(page, {"action": "WAIT",     "ms": 1500})
await aliax.execute(page, {"action": "DONE"})
```

Raw coordinates are also accepted as an escape hatch:
```python
await aliax.execute(page, {"action": "CLICK", "x": 905, "y": 150})
```

Under the hood, `execute()` scrolls the target into view, fires
React-friendly events (focus → keystrokes with a 50ms cadence), and
returns a small dict like `{"ok": True, "coords": [905, 150]}`. It
never raises — your agent loop stays alive.

## How it works under the hood

1. The bundled DOM mapper (`dom-mapper.min.js`, shipped inside the
   wheel) traverses the live page including shadow DOMs and same-origin
   iframes. It returns only **truly interactable** elements with a
   minimum size of 12 CSS pixels — no clutter, no hallucinated boxes.
2. The same JS module paints a `position:fixed; pointer-events:none;
   contain:strict` overlay of numbered colored boxes at the top of the
   z-stack. Chromium renders the boxes in microseconds and the host
   page's layout / hover / IntersectionObserver state is untouched.
3. Playwright snaps **one** native screenshot via its C++ CDP path at
   the format / quality you asked for in `render_config`. No Python
   image processing, no Pillow dependency.
4. The overlay layer is torn down in a `try/finally` so a crash mid-
   capture still leaves the live page exactly as we found it.
5. Coordinates are cached so `execute(page, {"element_id": "el_41"})`
   resolves instantly against the most recent `parse_ui()`.
6. Every call fires a non-blocking telemetry ping for usage tracking.

## What's new in v1.0.0 — first PyPI release

The 1.0.0 cut is the first official PyPI release. The SDK has been
hardened for enterprise deployment — zipped wheels, serverless layers,
long-running agents — and the install name now matches the import
(`pip install aliax` → `import aliax`).

- **`async with Aliax() as aliax:`** — full async context manager support
  so the `httpx` connection pool is torn down deterministically even if
  the agent loop crashes. No more socket leaks in 24/7 bots.
- **`ALIAX_API_KEY` env-var fallback** — `Aliax()` with no args reads
  the key from the environment; 12-factor Docker / Lambda / GitHub
  Actions deployments don't have to thread it through code.
- **Zip-safe asset loading** — `dom-mapper.min.js` is read via
  `importlib.resources`, so the SDK runs inside AWS Lambda layers and
  Google Cloud Run container images where `__file__` is a virtual path.
- **PEP 561 typed** — ships `py.typed` so `mypy`, Pyright, VS Code, and
  PyCharm consume the inline type hints out of the box.
- **PEP 621 packaging** — pure `pyproject.toml`, dynamic version pulled
  from `aliax/_version.py`, MIT-licensed, classified as
  `Development Status :: 5 - Production/Stable`.
- **`render_config={"format", "quality"}`** — the only knob AI engineers
  get for VLM token economics. Defaults are pristine JPEG at
  `quality=80`; drop to `quality=40` for roughly 10× cheaper VLM calls
  without sacrificing Set-of-Mark ID legibility. Quality is clamped to
  `[30, 100]` so a typo can't crash the agent.
- **Browser-native overlay** — Set-of-Mark boxes are painted by the
  bundled `dom-mapper.min.js` as a `position:fixed; pointer-events:none;
  contain:strict` DOM layer. Chromium renders them in microseconds;
  Playwright snaps the screenshot via its native C++ CDP path. The SDK
  never touches the pixel buffer — no Pillow, no OpenCV, no Cairo.
- **`parse_ui(page)` / `execute(page, decision)` / `capture_failure(...)`**
  — the canonical 3-call agent loop, with a typed `ParseContext`,
  `AttemptedAction` dataclass, and the full action catalog
  (`CLICK`, `TYPE`, `TYPE_AND_ENTER`, `HOVER`, `SCROLL_{DOWN,UP,LEFT,RIGHT}`).
- **Hardened transport** — bearer-only auth, idempotency keys on
  `capture_failure`, race-safe HTTP client init, non-blocking usage
  telemetry, automatic `*.workers.dev` fallback if the apex DNS / TLS
  flakes, opt-out via `fallback_endpoint=""`.

See [CHANGELOG.md](CHANGELOG.md) for the full changelog.


