Metadata-Version: 2.4
Name: sela-browse-sdk
Version: 1.0.13
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Rust
Classifier: Topic :: Internet :: WWW/HTTP :: Browsers
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Summary: Sela Network Python SDK - P2P client for browser automation agents
Keywords: sela,p2p,libp2p,browser-automation,web-scraping,semantic
Author-email: Sela Network <dev@sela.network>
License: MIT
Requires-Python: >=3.9
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
Project-URL: documentation, https://github.com/sela-network/sela-node-v2/tree/main/client-sdk
Project-URL: homepage, https://github.com/sela-network/sela-node-v2
Project-URL: repository, https://github.com/sela-network/sela-node-v2

# Sela Network Python SDK

Python bindings for the Sela Network client SDK. Browse any website and get back structured data through distributed browser agents — no CAPTCHAs, no bot detection.

## Installation

```bash
pip install sela-browse-sdk
```

## Quick Start

```python
import asyncio
import json
from sela_browse_sdk import SelaClient, BrowseOptions

async def main():
    # 1. Create client with your API key (get one at https://app.selanet.ai)
    client = await SelaClient.with_api_key("sk_live_xxx")

    # 2. Browse a URL
    result = await client.browse(
        url="https://x.com/search?q=ai&f=live",
        options=BrowseOptions(parse_only=True),
    )

    # 3. Extract structured data
    for item in result.page.content:
        fields = json.loads(item.fields_json)
        print(fields["text"])
        print(fields["author_username"], "—", fields.get("like_count"), "likes")

    await client.shutdown()

asyncio.run(main())
```

Example response:

```json
{
  "page": {
    "page_type": "x_com::search",
    "content": [
      {
        "content_type": "tweet",
        "fields": {
          "text": "AI is transforming how we build software...",
          "author_username": "johndoe",
          "like_count": 142,
          "retweet_count": 38
        }
      }
    ]
  }
}
```

> **Note:** `fields_json` and `collection_stats` are JSON strings in Python. Always use `json.loads()` to parse them.

---

## Examples

All examples below assume you've created a client as shown in Quick Start:

```python
client = await SelaClient.with_api_key("sk_live_xxx")
```

### Browse Any Website (Markdown Extraction)

For general websites (not X or Xiaohongshu), use `extract_format='markdown'` to get readable content:

```python
result = await client.browse(
    url="https://en.wikipedia.org/wiki/Python_(programming_language)",
    options=BrowseOptions(parse_only=True, extract_format="markdown"),
)

print(result.extracted_content)  # Clean markdown text
```

> **Important:** `extract_format='schema'` (default) only works for supported platforms (X, Xiaohongshu). For all other sites, you **must** use `'markdown'` or `'html'` — otherwise you'll get an empty response.

### Collect More Items (Infinite Scroll)

Use `count` to collect multiple items by scrolling the page:

```python
import json

# Collect 50 tweets by scrolling
result = await client.browse(
    url="https://x.com/search?q=python&f=live",
    options=BrowseOptions(parse_only=True, count=50),
)

for item in result.page.content:
    fields = json.loads(item.fields_json)
    print(fields["text"][:80])

# Check collection stats
if result.collection_stats:
    stats = json.loads(result.collection_stats)
    print(f"Collected: {stats}")
```

### Xiaohongshu (RedNote)

#### Search Posts

When using platform params (`xiaohongshu_params`, `x_params`), pass `url=None` — the URL is determined by the params.

```python
import json
from sela_browse_sdk import BrowseOptions, XiaohongshuParams

# Search and collect 30 posts
result = await client.browse(
    url=None,
    options=BrowseOptions(
        xiaohongshu_params=XiaohongshuParams(
            type="search",
            query="맛집 추천",
        ),
        count=30,
        parse_only=True,
    ),
)

for item in result.page.content:
    fields = json.loads(item.fields_json)
    print(fields.get("title", ""), "—", fields.get("like_count", ""))
```

#### Browse a Note by URL

Use `type="note"` with a `url` to browse a note directly by URL:

```python
result = await client.browse(
    url=None,
    options=BrowseOptions(
        xiaohongshu_params=XiaohongshuParams(
            type="note",
            url="https://www.xiaohongshu.com/explore/682e7c72000000000b03a68a",
        ),
        parse_only=True,
    ),
)
```

#### Browse a Note by ID

Use `type="note"` with a `note_id` to browse a specific note:

```python
result = await client.browse(
    url=None,
    options=BrowseOptions(
        xiaohongshu_params=XiaohongshuParams(
            type="note",
            note_id="682e7c72000000000b03a68a",
        ),
        parse_only=True,
    ),
)
```

#### XiaohongshuParams Types

| Type | Required Fields | Description |
|------|----------------|-------------|
| `"search"` | `query` | Search posts by keyword |
| `"note"` | `note_id` or `url` | Browse a specific note by ID or URL |
| `"profile"` | `user_id` | Browse a user profile |
| `"url"` | `url` | Browse any Xiaohongshu URL directly |

### X (Twitter)

#### Search Tweets

```python
from sela_browse_sdk import BrowseOptions, XParams

result = await client.browse(
    url=None,
    options=BrowseOptions(
        x_params=XParams(
            type="search",
            query="sela network",
            min_likes=100,
        ),
        count=30,
        parse_only=True,
    ),
)
```

#### Browse a Profile

```python
from sela_browse_sdk import BrowseOptions, XParams

result = await client.browse(
    url=None,
    options=BrowseOptions(
        x_params=XParams(type="profile", username="elonmusk", tab="media"),
        parse_only=True,
    ),
)
```

#### XParams Types

| Type | Required Fields | Description |
|------|----------------|-------------|
| `"search"` | `query` | Advanced search with filters |
| `"profile"` | `username` | User profile (optional `tab`: posts/replies/media/likes) |
| `"post"` | `username`, `tweet_id` | Single tweet |

### Browse Multiple URLs in Parallel

`browse_parallel_collect()` distributes URLs across agents for concurrent browsing:

```python
import asyncio
from sela_browse_sdk import (
    SelaClient, BrowseOptions,
    XiaohongshuParams, ParallelBrowseItem,
)

NOTE_URLS = [
    "https://www.xiaohongshu.com/explore/682e7c72000000000b03a68a",
    "https://www.xiaohongshu.com/explore/682e0a26000000000d007040",
    "https://www.xiaohongshu.com/explore/682e5f1d000000001203ae3e",
    "https://www.xiaohongshu.com/explore/682dd62f000000000c037bfc",
    "https://www.xiaohongshu.com/explore/682e4e43000000000b03a04e",
]

async def main():
    client = await SelaClient.with_api_key("sk_live_xxx")

    # Build parallel items
    items = [
        ParallelBrowseItem(
            url=None,
            options=BrowseOptions(
                xiaohongshu_params=XiaohongshuParams(type="note", url=note_url),
                parse_only=True,
            ),
        )
        for note_url in NOTE_URLS
    ]

    # Browse all concurrently (max 5 per agent)
    results = await client.browse_parallel_collect(items, max_concurrent_per_agent=5)

    for r in results:
        if r.error:
            print(f"[{r.index}] FAIL: {r.error}")
        else:
            title = r.response.page.metadata.title or "N/A"
            print(f"[{r.index}] {title[:40]} ({r.elapsed_ms}ms)")

    await client.shutdown()

asyncio.run(main())
```

#### Batching Large Requests

For many URLs, batch them to avoid overwhelming the network:

```python
batch_size = 5
for i in range(0, len(all_urls), batch_size):
    batch = all_urls[i : i + batch_size]
    items = [
        ParallelBrowseItem(
            url=None,
            options=BrowseOptions(
                xiaohongshu_params=XiaohongshuParams(type="note", url=url),
                parse_only=True,
            ),
        )
        for url in batch
    ]
    results = await client.browse_parallel_collect(items, max_concurrent_per_agent=5)
    # process results...
```

`ParallelBrowseResult` fields:

| Field | Type | Description |
|-------|------|-------------|
| `index` | `int` | Original position in the input list |
| `url` | `str` | The URL that was browsed |
| `response` | `SemanticResponse?` | Result (None if error) |
| `error` | `str?` | Error message (None if success) |
| `agent_peer_id` | `str` | Which agent handled this URL |
| `elapsed_ms` | `int` | Time taken in milliseconds |

---

## API Reference

### SelaClient

#### Creating a Client

```python
from sela_browse_sdk import SelaClient

# Recommended — just pass your API key
client = await SelaClient.with_api_key("sk_live_xxx")
```

#### Methods

| Method | Description |
|--------|-------------|
| `browse(url, options?)` | Browse a URL and get semantic content |
| `browse_parallel_collect(items, max_concurrent?)` | Browse multiple URLs in parallel |
| `search(query, target_url?, options?)` | Search on a platform |
| `shutdown(timeout_ms?)` | Gracefully shutdown the client |
| `state()` | Get current client state |

---

### Configuration

#### BrowseOptions

```python
BrowseOptions(
    parse_only=True,            # Always recommended — skip AI planning, extract only
    count=None,                 # Items to collect (enables infinite scroll)
    extract_format=None,        # "schema" (default), "markdown", "html"
    session_id=None,            # Reuse existing browser session
    timeout_ms=None,            # Request timeout in ms
    x_params=None,              # X (Twitter) search/profile/post params
    xiaohongshu_params=None,    # Xiaohongshu search/note/profile/url params
    agent_id=None,              # Target a specific agent by peer ID
    query=None,                 # Hint for semantic extraction
    include_html=None,          # Return raw HTML in response
    api_key=None,               # Override API key per-request
)
```

> **Important:** `extract_format='schema'` only works for supported platforms (X, Xiaohongshu). For all other sites, you **must** use `'markdown'` or `'html'`.

#### Advanced: SelaClientConfig

For fine-grained control, create a client with `SelaClientConfig`:

```python
from sela_browse_sdk import SelaClient, SelaClientConfig

config = SelaClientConfig(
    bootstrap_nodes=[],
    api_key="sk_live_xxx",               # Required — your API key
    api_server_url=None,                 # Default: https://api.selanet.ai
    connection_timeout_secs=30,
    request_timeout_secs=120,
)
client = await SelaClient.create(config)
```

---

### Response Types

#### SemanticResponse

```python
result = await client.browse(url, options)

result.session_id           # Browser session ID
result.page                 # SemanticPage
result.page.page_type       # e.g., "x_com::search", "generic"
result.page.content         # list[SemanticContent]
result.page.metadata.title  # Page title
result.collection_stats     # JSON string (when count is used) — use json.loads()
result.extracted_content    # Markdown/HTML text (when extract_format is set)
result.extract_format       # "schema", "html", or "markdown"
result.agent_version        # Agent node version (e.g., "0.2.6")
result.agent_id             # Agent peer ID that processed the request
```

#### SemanticContent

```python
for item in result.page.content:
    item.content_type       # e.g., "tweet", "note", "user"
    item.content_id         # Unique ID (if available)
    item.fields_json        # JSON string — use json.loads()
    item.actions            # list[AvailableAction]
```

---

### Error Handling

```python
from sela_browse_sdk import SelaError

try:
    result = await client.browse(url, options)
except SelaError.ConfigurationError as e:
    print(f"Bad config: {e}")
except SelaError.TimeoutError as e:
    print(f"Request timed out: {e}")
except SelaError.BrowseError as e:
    print(f"Browse failed: {e}")
except SelaError as e:
    print(f"Error: {e}")
```

| Error | When |
|-------|------|
| `ConfigurationError` | Invalid config (bad API key format, missing fields) |
| `TimeoutError` | Request timed out |
| `BrowseError` | Browse operation failed on agent |
| `InternalError` | Unexpected internal error |

---

## Development

### Building from Source

Requires: Rust 1.70+, Python 3.9+, maturin

```bash
pip install maturin

cd client-sdk/crates/sela-py
maturin develop --release   # Install in dev mode
maturin build --release     # Build wheel
```

### Running the Example

```bash
cd client-sdk
python examples/browse_example.py
```

### Running Tests

```bash
cargo test -p sela-py
```

## License

MIT License - see [LICENSE](../../../LICENSE) for details.

