Skip to content

Browser Agent

BrowserAgent drives a web browser through Playwright.

Use it for tasks that are naturally completed in websites or web apps, such as searching, comparing pages, filling web forms, navigating dashboards, or extracting visible information from browser sessions.

Requirements

  • Playwright browser dependencies installed
  • Browser binaries installed with uv run playwright install
  • Provider credentials configured for the selected model provider

What It Owns

  • Playwright browser setup and cleanup
  • screenshots of the current browser viewport
  • browser actions such as navigation, clicking, typing, scrolling, hovering, and keyboard shortcuts
  • conversion from normalized 0-1000 coordinates to browser pixels

Reasoning Loop

BrowserAgent follows the shared observe/reason/act loop:

  1. Observe: capture a browser screenshot and read the current page URL.
  2. Encode: send the task, screenshot, and URL metadata through the configured model adapter.
  3. Reason: ask the model for either a final answer or browser tool calls.
  4. Act: execute browser actions through BrowserController, such as navigating, clicking, typing, scrolling, or pressing keys.
  5. Report: capture a fresh screenshot and URL after each action and return them to the model as tool results.
  6. Repeat: continue until the model returns a final answer or the step limit is reached.

The browser loop is best when the main state is page-centric: URLs, web forms, DOM-backed interactions, and visual page content.

Run With Python

import asyncio

from uisurf_agent import BrowserAgent


async def main() -> None:
    async with BrowserAgent(headless=True) as agent:
        async for event in agent.run(
            "Open example.com and summarize the page",
            max_steps=20,
        ):
            print(event.eventType, event.payload)


asyncio.run(main())

Run With The CLI

uv run uisurf_agent run browser_agent \
  --task "Open example.com and summarize the page" \
  --headless \
  --fast-mode \
  --max-steps 20

Run As An A2A Server

uv run uisurf_agent run browser_agent \
  --mode a2a \
  --host 0.0.0.0 \
  --port 8001

If --port is omitted, the browser A2A server defaults to 8001.

Browser A2A mode also reads BROWSER_AGENT_MAX_STEPS, BROWSER_AGENT_AUTO_MODE, BROWSER_INCLUDE_THOUGHTS, BROWSER_OBSERVATION_SCALE, BROWSER_FAST_MODE, and BROWSER_AGENT_PUBLIC_URL from the environment.

Common Runtime Options

Option Description
--headless Launch Chromium without a visible browser window when Playwright owns the browser process.
--fast-mode / --no-fast-mode Use faster browser settling behavior.
--auto-mode Automatically approve safety-gated actions.
--max-steps Maximum observe/reason/act iterations.
--max-observation-images Number of recent image-bearing items that keep image payloads.
--observation-scale Scale screenshots before sending them to the model.

Model and provider configuration is documented in the Models section.

Notes

headless=True only affects the path where Playwright launches Chromium. If the controller connects to an already-running browser through CDP, the existing browser process decides whether it is headless or headed.