Metadata-Version: 2.4
Name: anyweb
Version: 0.6.3
Summary: CLI-first browser automation with platform intelligence
Project-URL: Homepage, https://github.com/Mixiaomiupup/anyweb
Project-URL: Repository, https://github.com/Mixiaomiupup/anyweb
Project-URL: Bug Tracker, https://github.com/Mixiaomiupup/anyweb/issues
Author: mixiaomi
License-Expression: MIT
License-File: LICENSE
Keywords: automation,browser,cli,playwright,scraping
Classifier: Development Status :: 3 - Alpha
Classifier: Environment :: Console
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: MacOS
Classifier: Operating System :: POSIX :: Linux
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Testing
Requires-Python: >=3.11
Requires-Dist: aiohttp>=3.9.0
Requires-Dist: playwright-stealth>=1.0.6
Requires-Dist: playwright~=1.59.0
Requires-Dist: websockets>=13.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.0.0; extra == 'dev'
Requires-Dist: ruff>=0.4.0; extra == 'dev'
Provides-Extra: mcp
Requires-Dist: mcp[cli]>=1.0.0; extra == 'mcp'
Requires-Dist: pydantic>=2.0.0; extra == 'mcp'
Description-Content-Type: text/markdown

# anyweb

**English** | [简体中文](README.zh-CN.md)

[![PyPI](https://img.shields.io/pypi/v/anyweb.svg)](https://pypi.org/project/anyweb/)
[![Python](https://img.shields.io/pypi/pyversions/anyweb.svg)](https://pypi.org/project/anyweb/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

CLI-first browser automation for AI agents. One command reads a webpage, one command clicks a button — no boilerplate, no browser lifecycle to manage.

Built on Playwright with anti-detection, a persistent daemon, and platform-specific extractors for X, Zhihu, XHS, and more.

**PyPI**: https://pypi.org/project/anyweb/

## Why anyweb

AI coding agents (Claude Code, Codex, etc.) need to interact with the web but can't drive a browser natively. anyweb fills that gap:

- **Atomic commands** — `open`, `click`, `input`, `keys`, `read` — each one maps to a single action
- **Persistent daemon** — browser stays alive between commands, ~50ms latency vs ~3s cold start
- **Lazy browser** — the daemon starts the browser only when a command needs it; `status`, `sessions`, and `log` work with no browser running
- **Multi-page parallelism** — `--page` flag runs named tabs concurrently, perfect for batch queries
- **Accessibility tree** — `state --ax` returns a compact, structured page representation with text, URLs, and element indices — ideal for LLM context
- **Content-stable waiting** — `wait content-stable` detects when dynamic content (Grok responses, streaming results) finishes loading
- **Platform intelligence** — `read` auto-detects X/Zhihu/XHS and applies platform-specific extraction

## Install

```bash
pip install anyweb        # or: uv tool install anyweb
anyweb install            # install Chromium
anyweb init               # optional: write ~/.anyweb/config.toml
anyweb doctor             # verify dependencies, config, and runtime
```

## Quick start

```bash
# Read any webpage — auto-detects platform
anyweb read "https://x.com/karpathy"

# JSON output for programmatic use
anyweb --json read "https://x.com/karpathy"

# Search within a platform
anyweb search x "embodied AI news" --limit 10

# System health check
anyweb doctor
```

## Commands

### High-level

| Command | What it does |
|---------|-------------|
| `read URL` | Smart extract: auto-detect platform, render JS, return clean content |
| `search PLATFORM QUERY` | Search within X / Zhihu / XHS / WeChat |
| `login PLATFORM` | Save session cookies (opens real Chrome) |
| `intercept URL --pattern P` | Open a page and capture API responses matching a pattern |
| `status` | Daemon, browser, pages, sessions, recommended actions |
| `doctor [--fix]` | Dependency + config + runtime + credential check with auto-repair |

### Atomic browser

| Command | What it does |
|---------|-------------|
| `open URL` | Navigate (starts daemon + browser if needed) |
| `state` | List interactive elements with indices |
| `state --ax` | Accessibility tree — compact text + URLs for LLMs |
| `click IDX` | Click element by index, AX ref (`e5`), or CSS selector |
| `input IDX "text"` | Click element, then type |
| `type "text"` | Type via keyboard events (no click) |
| `keys Enter` | Send keyboard events |
| `hover IDX` | Hover over element |
| `scroll down` | Scroll the page |
| `eval "js"` | Execute JavaScript (supports Promises) |
| `screenshot` | Capture screenshot |
| `get text\|title\|url\|html` | Get page info |
| `wait selector "h1"` | Wait for CSS selector |
| `wait content-stable` | Wait for dynamic content to settle |
| `back` | Navigate back |
| `close` | Close browser; daemon stays alive |
| `shutdown` | Stop everything (browser + daemon) |

All atomic commands accept `--page NAME` for parallel named tabs.

### Setup & diagnostics

| Command | What it does |
|---------|-------------|
| `init` | Create `~/.anyweb/config.toml` (`--mode`, `--port`, `--proxy`, `--force`) |
| `install` | Install Playwright browsers |
| `doctor [--fix]` | Dependencies, config, runtime, credentials |
| `log [-f] [--errors]` | View, follow, or filter daemon command log |
| `sessions` | List active browser sessions and pages |
| `status` | Daemon PID/RSS, browser, pages, cookie expiry |
| `cookies get\|export\|import\|clear` | Cookie management |

## Configuration

anyweb works with zero config. To pin behavior, write `~/.anyweb/config.toml`:

```bash
anyweb init                              # mode=auto, port=9222, no proxy
anyweb init --mode headless --proxy http://127.0.0.1:7890 --force
```

```toml
# ~/.anyweb/config.toml
[browser]
mode  = "auto"        # auto | headed | headless
port  = 9222          # CDP port used in headed mode
proxy = ""            # e.g. http://127.0.0.1:7890
```

**Resolution order**: environment variables > `config.toml` > defaults.

| Setting | Env override | Default |
|---------|-------------|---------|
| Browser mode | `ANYWEB_MODE` | `auto` |
| Proxy | `ANYWEB_PROXY` (falls back to `ALL_PROXY`/`HTTPS_PROXY`/`HTTP_PROXY`) | none |

`mode = auto` resolves to **headed** on a desktop (macOS, or Linux with `DISPLAY`/`WAYLAND_DISPLAY`) and **headless** on a headless server. `doctor` reports the resolved mode under its Config and Runtime checks.

### WSL

On WSL, `mode = auto` uses a layered browser strategy:

1. Prefer Linux Chrome inside WSL with WSLg/X11/Wayland.
2. Fall back to Windows Chrome launched from WSL when Linux Chrome is missing.
3. Use headless Playwright when no headed browser path is available.

Run `anyweb doctor` to see which path your machine is using. For server-style WSL sessions, pin headless mode:

```bash
ANYWEB_MODE=headless anyweb read https://example.com
```

## Multi-page parallelism

`--page NAME` runs multiple named tabs concurrently — essential for batch operations like querying Grok 4 times in parallel:

```bash
# Open 4 pages
anyweb open --page g1 "https://x.com/i/grok?new=true"
anyweb open --page g2 "https://x.com/i/grok?new=true" &
anyweb open --page g3 "https://x.com/i/grok?new=true" &
anyweb open --page g4 "https://x.com/i/grok?new=true" &
wait

# Submit queries in parallel
anyweb input --page g1 textarea "Query 1" && anyweb keys --page g1 Enter &
anyweb input --page g2 textarea "Query 2" && anyweb keys --page g2 Enter &
wait

# Wait for all responses to finish
anyweb wait --page g1 content-stable --duration 5 --timeout 120000 &
anyweb wait --page g2 content-stable --duration 5 --timeout 120000 &
wait

# Read results
anyweb state --page g1 --ax > /tmp/result1.txt
anyweb state --page g2 --ax > /tmp/result2.txt
```

`read` manages pages internally — no `--page` needed for batch reads:

```bash
anyweb --json read "https://x.com/AnthropicAI" > /tmp/a.json &
anyweb --json read "https://x.com/OpenAI" > /tmp/b.json &
anyweb --json read "https://x.com/karpathy" > /tmp/c.json &
wait
```

## Accessibility tree

`state --ax` returns the page structure optimized for LLM consumption — text, links, and interactive elements in a compact format:

```
URL: https://x.com/karpathy
Title: Andrej Karpathy (@karpathy) / X
Refs: 42 interactive elements
---
RootWebArea "Andrej Karpathy (@karpathy) / X"
  e0:link "Home" -> https://x.com/home
  e1:link "Search" -> https://x.com/explore
  ...
  StaticText "The hottest new programming language is English"
  e12:link "View tweet" -> https://x.com/karpathy/status/1617979122625712128
```

Options: `-i` / `--interactive-only` (clickable/typable elements only), `--depth N` (limit tree depth). Click any ref directly: `anyweb click e12`.

## Platform intelligence

`read` auto-detects the platform from the URL and applies the right extraction:

| Platform | Detection | Extraction |
|----------|-----------|------------|
| X/Twitter | `x.com`, `twitter.com` | Tweets, threads, profiles, engagement stats, media URLs |
| Zhihu | `zhihu.com` | Auto-expand folded content, answers |
| XHS | `xiaohongshu.com` | Notes, comments, images |
| Generic | everything else | Article body, metadata |

Force a platform: `anyweb read -p x "https://example.com"`

## Session management

Login once, use forever. Cookies persist across daemon restarts:

```bash
anyweb login x                    # opens real Chrome, you log in, cookies saved
anyweb login x --method import    # import from system Chrome (no manual login)
anyweb login x --method cookie    # paste cookies directly
```

`doctor` and `status` show session health:

```
=== Sessions ===
  x         : saved 2026-05-18 | cookies: 465 | expires 2027-05-25
  zhihu     : saved 2026-05-08 | cookies: 36  | expires 2026-11-03
  xhs       : no auth cookie ❌

=== Recommended Actions ===
  anyweb login xhs
```

## Output formats

```bash
anyweb read URL              # plain text (default, pipe-friendly)
anyweb --json read URL       # structured JSON with metadata
anyweb --markdown read URL   # markdown with frontmatter
```

## Architecture

```
CLI ──▶ Unix socket ──▶ Daemon (persistent) ──▶ one browser
MCP ──▶ stdio JSON-RPC ─┘    │
                             ├── mode=headed   → real Chrome via CDP (login state, watchable)
                             └── mode=headless → Playwright Chromium (servers, fast)
```

- **Single browser, auto-detected mode** — `config.toml` sets `mode` (`auto`/`headed`/`headless`); `auto` picks headed on a desktop, headless on a server. No more engine routing or `--headed`/`--headless` flags.
- **Lazy startup** — the daemon launches the browser only when a command needs it. `status`, `sessions`, and `log` respond instantly with no browser running.
- **Daemon** auto-starts on first command (guarded by a file lock), stays alive across invocations, ~50ms per command.
- **Headed = CDP to a dedicated Chrome** on the configured port — so cookies/login state are real and you can watch it work. **Headless = Playwright Chromium** — faster, no display required.
- **Anti-detection**: playwright-stealth + behavior simulation applied automatically.
- **Session isolation**: per-platform cookie storage in `~/.anyweb/sessions/`.
- **Runtime layout**: `~/.anyweb/` holds `config.toml`, `sessions/`, the daemon socket/PID, and `daemon.log`.

Full design doc: [`docs/architecture.md`](docs/architecture.md).

## Requirements

- Python 3.11+
- macOS or Linux

## License

MIT
