Tool Extensions¶
Browser, desktop, and mobile tools can be extended with the same registry pattern used by models.
A tool extension has two parts:
- a declaration function, which describes the function schema exposed to the model
- an optional handler, which executes the tool when an agent receives that tool call
The built-in Gemini model reads this registry when building its browser,
desktop, or mobile tool configuration. BrowserAgent, DesktopAgent, and
MobileAgent read the same registry when dispatching model-emitted tool calls.
Register a Browser Tool¶
Use register_browser_tool(...) for browser-only tools.
from typing import Any
from uisurf_agent import BrowserAgent, register_browser_tool
async def save_page_title(agent: BrowserAgent, args: dict[str, Any]) -> None:
title = await agent._browser_controller.page.title()
print(f"{args['label']}: {title}")
@register_browser_tool(handler=save_page_title)
def save_page_title(label: str) -> dict[str, str]:
"""Save the current page title with a label."""
return {"label": label}
The declaration function name becomes the model-visible tool name. In this
example, the model sees a save_page_title function with one label argument.
End-To-End Browser Tool Example¶
This script registers a custom browser tool, starts a browser agent, and streams
agent events. Save it as application code that imports uisurf_agent before the
agent is created.
import asyncio
from typing import Any
from uisurf_agent import BrowserAgent, register_browser_tool
async def save_page_title_handler(
agent: BrowserAgent,
args: dict[str, Any],
) -> None:
title = await agent._browser_controller.page.title()
print(f"{args['label']}: {title}")
@register_browser_tool(handler=save_page_title_handler)
def save_page_title(label: str) -> dict[str, str]:
"""Save the current page title with a label."""
return {"label": label}
async def main() -> None:
async with BrowserAgent(
provider_name="gemini",
model_id="gemini-3-flash-preview",
auto_mode=True,
) as agent:
async for event in agent.run(
(
"Open https://example.com, then call save_page_title "
"with label 'Example page'."
),
max_steps=5,
):
print(event.eventType, event.payload)
if __name__ == "__main__":
asyncio.run(main())
agent.run(...) is an async generator, so consume it with async for. The tool
is registered when the module is imported, and the provider sees the tool when
the BrowserAgent creates its model configuration.
Register a Desktop Tool¶
Use register_desktop_tool(...) for desktop-only tools.
from typing import Any
from uisurf_agent import DesktopAgent, register_desktop_tool
async def open_notes_workspace(agent: DesktopAgent, args: dict[str, Any]) -> None:
await agent._desktop_controller.open_app("Notes", intent=args.get("folder"))
@register_desktop_tool(handler=open_notes_workspace)
def open_notes_workspace(folder: str | None = None) -> dict[str, str | None]:
"""Open Notes, optionally targeting a folder."""
return {"folder": folder}
Register a Mobile Tool¶
Use register_mobile_tool(...) for Android-only tools.
from typing import Any
from uisurf_agent import MobileAgent, register_mobile_tool
async def open_settings_handler(agent: MobileAgent, args: dict[str, Any]) -> None:
await agent._mobile_controller.open_app("com.android.settings")
@register_mobile_tool(handler=open_settings_handler)
def open_android_settings() -> dict[str, str]:
"""Open Android Settings."""
return {"status": "settings_requested"}
Use register_tool("browser", ...), register_tool("desktop", ...), or
register_tool("mobile", ...) when the target environment is selected
dynamically.
Handler Contract¶
Handlers receive the agent instance and the decoded tool-call arguments:
Handlers may be sync or async. If a handler returns an awaitable, the agent
awaits it. Exceptions are caught by the agent and returned to the model as
error: ... tool results.
If a registered tool has no handler, the agent falls back to a controller method with the same name:
from uisurf_agent import register_desktop_tool
@register_desktop_tool
def custom_controller_action(value: str) -> dict[str, str]:
"""Call DesktopController.custom_controller_action(value=...)."""
return {"value": value}
This is useful when an application subclasses or wraps the built-in controller and already exposes the action method there.
Custom Tool Names¶
Pass name=... to expose a different model-visible function name from the
Python callable name:
from uisurf_agent import register_browser_tool
@register_browser_tool(name="capture_page_summary")
def page_summary(label: str) -> dict[str, str]:
"""Capture a summary of the current page."""
return {"label": label}
Installed Package Entry Points¶
Installed packages can expose tools through the uisurf_agent.tools entry-point
group.
Entry points may expose a registration function:
def register(registry):
registry.register("browser", my_browser_tool, handler=my_browser_handler)
registry.register("desktop", my_desktop_tool, handler=my_desktop_handler)
registry.register("mobile", my_mobile_tool, handler=my_mobile_handler)
They may also expose a ToolRegistration object, a sequence of
ToolRegistration objects, or a declaration callable named with an environment
prefix:
[project.entry-points."uisurf_agent.tools"]
"browser.save_page_title" = "my_package.tools:save_page_title"
"desktop.open_notes_workspace" = "my_package.tools:open_notes_workspace"
"mobile.open_android_settings" = "my_package.tools:open_android_settings"
When using the browser.name, desktop.name, or mobile.name form, the
entry-point name is used as the model-visible tool name.
Relationship to Models¶
The tool registry is provider-neutral. A provider implementation decides how to convert registered declaration callables into provider-specific tool schemas.
The built-in gemini provider already does this for all built-in environments. Custom
provider implementations can call:
from uisurf_agent import tool_registry
browser_tools = tool_registry.declarations("browser")
desktop_tools = tool_registry.declarations("desktop")
mobile_tools = tool_registry.declarations("mobile")
Then convert those declarations into the provider's tool schema.
Public API¶
The main exports are:
tool_registryregister_toolregister_browser_toolregister_desktop_toolregister_mobile_toolToolRegistrationToolRegistryToolEnvironmentToolDeclarationToolHandler