Metadata-Version: 2.4
Name: aliax
Version: 1.0.3
Summary: Aliax SDK — real-time interceptor middleware for visual web agents (Set-of-Mark + deterministic execution).
Author-email: Aliax <hello@aliax.xyz>
License: MIT License
        
        Copyright (c) 2026 Aliax
        
        Permission is hereby granted, free of charge, to any person obtaining a copy
        of this software and associated documentation files (the "Software"), to deal
        in the Software without restriction, including without limitation the rights
        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
        copies of the Software, and to permit persons to whom the Software is
        furnished to do so, subject to the following conditions:
        
        The above copyright notice and this permission notice shall be included in
        all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
        THE SOFTWARE.
        
Project-URL: Homepage, https://aliax.xyz
Project-URL: Documentation, https://aliax.xyz/docs
Project-URL: Source, https://github.com/aliax/aliax-sdk
Project-URL: Tracker, https://github.com/aliax/aliax-sdk/issues
Project-URL: Changelog, https://github.com/aliax/aliax-sdk/blob/main/CHANGELOG.md
Keywords: ai,agents,web-agents,playwright,vlm,set-of-mark,browser-automation,llm-tools
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx<2,>=0.24
Requires-Dist: playwright<2,>=1.30
Provides-Extra: test
Requires-Dist: pytest>=7; extra == "test"
Requires-Dist: pytest-asyncio>=0.21; extra == "test"
Requires-Dist: respx>=0.20; extra == "test"
Dynamic: license-file

### ⚡ See it work before you install — [run the live browser benchmark →](https://aliax.xyz/benchmark)

> Paste any URL. Watch Aliax shrink the page into a tiny, numbered map your model can actually read — side-by-side with the raw HTML, in seconds. No signup, no install.

---

# Aliax — Python SDK

[![PyPI version](https://img.shields.io/pypi/v/aliax.svg?style=flat-square&color=10B981)](https://pypi.org/project/aliax/)
[![Python versions](https://img.shields.io/pypi/pyversions/aliax.svg?style=flat-square&color=3B82F6)](https://pypi.org/project/aliax/)
[![CI](https://img.shields.io/github/actions/workflow/status/aliax/aliax-sdk/test.yml?branch=main&style=flat-square&label=tests)](https://github.com/aliax/aliax-sdk/actions/workflows/test.yml)
[![License: MIT](https://img.shields.io/badge/License-MIT-10B981.svg?style=flat-square)](LICENSE)

**The production-grade reliability layer for VLM web agents.**

Stop your AI agents from guessing browser coordinates and failing on
responsive layouts. Aliax sits between your VLM and the live page,
translates the DOM into a numbered Set-of-Mark image, executes the
action the model picks deterministically, and ships every real-world
failure into a labelled dataset you can fine-tune on.

It is the layer that turns *"works in my notebook"* into *"holds up at
3am for paying corporate clients."*

Python: 3.8 – 3.12 · [Changelog](CHANGELOG.md) · [Contributing](CONTRIBUTING.md)

## Install

```bash
pip install aliax
playwright install chromium
```

`aliax` is the install name AND the import — no aliases, no version
pinning required. No signup either: with no key, `Aliax()` bootstraps a
free sandbox credential (500 parses; 100 from cloud egress) and caches it
in `~/.aliax/credentials`. `Aliax()` reads `ALIAX_API_KEY` from the environment
so 12-factor deployments (Docker, Lambda, GitHub Actions) don't have
to thread the key through code.

## The 3-line agent loop

```python
from aliax import Aliax, SYSTEM_INSTRUCTIONS
from playwright.async_api import async_playwright

async with Aliax() as aliax:                     # no key needed on first run
    async with async_playwright() as p:
        page = await (await p.chromium.launch()).new_page()
        await page.goto("https://shop.example.com/cart")

        # 1. Translate the live page into a Set-of-Mark image + node map.
        ctx = await aliax.parse_ui(page)

        # 2. Hand both to your VLM. Drop SYSTEM_INSTRUCTIONS into the system prompt.
        decision = await ask_llm(
            system=SYSTEM_INSTRUCTIONS,
            image=ctx.image_bytes,
            map=ctx.llm_text_block(),
        )
        # e.g. {"action": "CLICK", "element_id": "el_41"}

        # 3. Aliax executes it natively — scroll-into-view, React onChange,
        #    iframe coords, retina DPR, hydration races — all handled.
        await aliax.execute(page, decision)
```

That's the entire happy path. No XPath wrangling, no Playwright
locator boilerplate, no pixel guessing.

---

## Pillar 1 — The Steady Hand

Deterministic VLM-to-DOM translation. Your model never sees raw
coordinates; it sees `el_41` and Aliax handles the browser physics.

`parse_ui(page)` returns a `ParseContext`:

| Attribute     | Type         | Description |
|---------------|--------------|-------------|
| `image_bytes` | `bytes`      | Native Chromium screenshot (JPEG by default) with numbered Set-of-Mark boxes painted by the bundled JS overlay. Zero Python image processing — no Pillow, no OpenCV. |
| `image_mime`  | `str`        | `"image/jpeg"` by default, `"image/png"` if you opt in. |
| `image_size`  | `(int, int)` | `(width, height)` in physical pixels, parsed from the image header. |
| `elements`    | `list[dict]` | Each element: `element_id`, `tag`, `role`, `text`, `bounds`, `editable`, `is_canvas`, `state`, `attrs`, `links_to`. Only **truly interactable** nodes ≥12 CSS px. |
| `viewport`    | `dict`       | `{width, height, dpr, scroll_x, scroll_y, page_scrollable_x, page_scrollable_y}` in CSS px. |
| `url`         | `str`        | Page URL at capture time. |
| `path`        | `str`        | URL pathname only — e.g. `"/settings/billing"`. |
| `title`       | `str`        | `document.title` at capture time. |
| `truncated`   | `bool`       | `True` when the DOM mapper hit its element cap; `llm_text_block()` appends a notice so the VLM knows the map is partial. |

Convenience helpers: `ctx.llm_text_block()` (drop-in for your prompt)
and `ctx.route_context_block()` (URL + title summary).

### Execute — the full verb catalog

Every action `execute()` recognises:

```python
await aliax.execute(page, {"action": "CLICK",          "element_id": "el_41"})
await aliax.execute(page, {"action": "HOVER",          "element_id": "el_14"})
await aliax.execute(page, {"action": "TYPE",           "element_id": "el_7", "value": "Nike Shoes"})
await aliax.execute(page, {"action": "TYPE_AND_ENTER", "element_id": "el_7", "value": "Nike Shoes"})
await aliax.execute(page, {"action": "PRESS",          "element_id": "el_7", "key": "Enter"})
await aliax.execute(page, {"action": "SCROLL_DOWN",    "dy": 600})       # also: dx / delta_x / delta_y
await aliax.execute(page, {"action": "SCROLL_UP",      "dy": 600})
await aliax.execute(page, {"action": "SCROLL_LEFT",    "dx": 400})
await aliax.execute(page, {"action": "SCROLL_RIGHT",   "dx": 400})
await aliax.execute(page, {"action": "NAVIGATE",       "url": "https://..."})
await aliax.execute(page, {"action": "WAIT",           "ms": 1500})
await aliax.execute(page, {"action": "COMBO",          "actions": [...]})  # compound action pipeline
await aliax.execute(page, {"action": "BATCH_TYPE",     "fields": [...]})   # multi-field fill in one turn
await aliax.execute(page, {"action": "REPORT_ISSUE",   "reason": "stuck"}) # routes through the Gatekeeper
await aliax.execute(page, {"action": "NOOP"})                              # terminal alias
await aliax.execute(page, {"action": "DONE"})                              # `FINISH` is accepted as an alias
```

Raw coordinates are accepted as an escape hatch:

```python
await aliax.execute(page, {"action": "CLICK", "x": 905, "y": 150})
```

Under the hood, `execute()` scrolls the target into view, fires
React-friendly events (focus → keystrokes with a 50ms cadence),
refuses to click disabled controls, and **never raises** — your
agent loop stays alive.

### Token-aware rendering

VLM providers bill on file weight + dimensions. Drop to `quality=40`
for roughly **10× cheaper** VLM calls without sacrificing Set-of-Mark
ID legibility (the IDs are crisp DOM text rendered *before* the JPEG
encoder runs).

```python
ctx = await aliax.parse_ui(
    page,
    render_config={"format": "jpeg", "quality": 40},
)
```

Quality is clamped to `[30, 100]` so a typo like `quality=5` can't
turn the numbered boxes into illegible mush and crash your loop.

---

## Pillar 2 — The Flight Recorder

Every agent eventually loops on a popup or hallucinates past
recovery. Aliax ships two escalation entry points that double as
your agentic telemetry stream:

### `report_issue()` — the gated escalation

```python
await aliax.report_issue(
    page,
    reason="submit_button_dead_zone",
    expected_outcome="navigate to /dashboard",
    actual_outcome="still on /login after 3 tries",
)
```

A strict Gatekeeper rejects the call unless the SDK's per-page
failure history *proves* the agent is stuck:

- **3 identical state hashes** of `(url, spatial_map)` → frozen DOM, and
- **period-2 / 3 / 4 action cycle detection** → radio-group / checkbox
  alternation loops the state hash is blind to.

Rejected calls never hit the API, never debit credits, and never
flood the annotation queue with slow-network panic. Pass `force=True`
only from a developer-side assertion (Playwright timeout, business
invariant violation).

### `capture_failure()` — the airbag

```python
await aliax.capture_failure(
    page,
    goal="Close newsletter popup",
    thoughts=agent.current_reasoning,
    last_attempted_action={"action": "CLICK", "element_id": "el_41"},
    failure_reason="modal_blocked",
    step=agent.loop_step,
)
```

`POST /v1/capture` with idempotency UUID, 3 attempts and exponential
backoff on 429/502/503/504, automatic transport-error fallback. The
full state — screenshot, DOM map, viewport, the agent's own
reasoning trail — lands in the **Aliax Inbox** for human triage.

---

## Pillar 3 — The Closed Loop

Every captured failure becomes a `(negative, positive)` DPO pair:
the agent's wrong move plus the annotator's corrected tap. Export
the dataset and fine-tune; your next deployment fails less often on
the exact failure modes that bit you in production.

That's the loop:

```
parse_ui → VLM → execute → (95% happy path)
                            ↓ stuck
                       report_issue → Inbox → DPO dataset → fine-tune
```

---

## Architecture

1. The bundled encrypted DOM mapper (`dom-mapper.dat`, shipped inside
  the wheel) is opened only after the authenticated licensing session,
  then walks the live page including shadow DOMs and same-origin
   iframes, returning only interactable elements ≥12 CSS px.
2. The same JS module paints a `position:fixed; pointer-events:none;
   contain:strict` overlay of numbered boxes. The host page's
   layout / hover / IntersectionObserver state is untouched.
3. Playwright snaps **one** native screenshot via its C++ CDP path
   at the format / quality you asked for. No Python pixel processing.
4. The overlay is torn down in a `try/finally` so a mid-capture
   crash leaves the live page exactly as we found it.
5. Coordinates are cached so `execute(page, {"element_id": "el_41"})`
   resolves instantly against the most recent `parse_ui()`.
6. Every call fires a non-blocking telemetry ping for billing.

### Hardening for long-running agents

- **`async with Aliax() as aliax:`** — full async context manager
  support so the `httpx` connection pool is torn down deterministically
  even if the loop crashes. No socket leaks in 24/7 bots.
- **Zip-safe asset loading** — `dom-mapper.dat` is loaded via
  `importlib.resources`, so the SDK runs inside AWS Lambda layers
  and Cloud Run images where `__file__` is virtual.
- **PEP 561 typed** — ships `py.typed`; `mypy`, Pyright, VS Code and
  PyCharm consume the inline type hints out of the box.
- **Automatic `*.workers.dev` fallback** — if the apex DNS / TLS
  flakes, the SDK transparently retries against the worker origin.
  Opt out with `fallback_endpoint=""`.
- **Self-healing billing latches** — HTTP 402 sets
  `credits_exhausted` and clears on any successful 200. HTTP 401
  latches `invalid_key_reason` (terminal — mint a new key).

## Public surface

```python
from aliax import (
    Aliax,
    ParseContext,
    AttemptedAction,
    AliaxError,
    AliaxOutOfCreditsError,
    AliaxInvalidKeyError,
    ALIAX_SYSTEM_INSTRUCTIONS,  # primary — drop into your VLM system prompt
    SYSTEM_INSTRUCTIONS,        # short alias for the same string
)
```

`Aliax` constructor (all keyword-only, all optional):

| Kwarg               | Default                      | Notes |
|---------------------|------------------------------|-------|
| `api_key`           | `os.getenv("ALIAX_API_KEY")`, else a free anonymous sandbox key | Must start with `sk_`. |
| `allow_anonymous`   | `True` | With no key, bootstrap a free machine-scoped sandbox credential (cached in `~/.aliax/credentials`): 500 parses on a home/office network, 100 from cloud egress, 30-day life, no crash captures. Set `False` (or `ALIAX_DISABLE_ANONYMOUS=1`) to hard-fail instead. |
| `endpoint`          | `https://api.aliax.xyz/v1`   | Override for self-hosted. |
| `fallback_endpoint` | workers.dev origin (auto)    | Pass `""` to disable. |
| `debug_mode`        | `False`                      | Writes payloads locally instead of POSTing. |
| `redact_selectors`  | `[]`                         | Merged with built-in PII baseline (passwords, emails, card numbers). |
| `check_for_updates` | `True`                       | Pings `/v1/version` once at init; pass `False` in airgapped envs. |
| `max_image_dim`     | `0`                          | **Deprecated — no-op since v1.0.** Use `render_config={"quality": N}` on `parse_ui` for VLM token economy. Kept in the signature so pre-1.0 callers keep importing; a one-shot deprecation log fires only when explicitly set. |

Methods: `parse_ui`, `execute`, `report_issue`, `capture_failure`,
`get_element_coords`, `billing_status`, `refresh_billing_status`,
`aclose`.

---

[Get an API key](https://aliax.xyz/auth) · [Docs](https://aliax.xyz/docs) · [Inbox](https://aliax.xyz/captures)
