Metadata-Version: 2.4
Name: camoufox_spider
Version: 0.1.0
Summary: 反检测 Web 爬虫库，基于 Camoufox 反检测浏览器
Home-page: https://github.com/violettoolssite/camoufox_spider
Author: violet
Author-email: violet <violettools.site@gmail.com>
License: MIT
Project-URL: Homepage, https://github.com/violettoolssite/camoufox_spider
Project-URL: Repository, https://github.com/violettoolssite/camoufox_spider
Project-URL: Issues, https://github.com/violettoolssite/camoufox_spider/issues
Keywords: scraping,anti-detection,camoufox,cloudflare,akamai,perimeterx,bot-detection
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Requires-Python: >=3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: requests>=2.25.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov>=4.0; extra == "dev"
Dynamic: author
Dynamic: home-page
Dynamic: license-file
Dynamic: requires-python

# 🕷️ SuperSpider

**反检测 Web 爬虫库** — 基于 Camoufox 反检测浏览器，轻松绕过 Cloudflare、Akamai、PerimeterX 等反爬系统。

[![Python](https://img.shields.io/badge/python-3.8+-blue.svg)](https://www.python.org/downloads/)
[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)
[![PyPI](https://img.shields.io/badge/pypi-super_requests-orange.svg)](https://pypi.org/project/super_requests/)

## ✨ 特性

- 🛡️ **反检测** — 基于 Camoufox，自动绕过主流反爬系统
- 🔌 **REST API** — 通过 camofox-browser 服务，支持远程控制
- 🎯 **简洁 API** — 一行代码获取页面，类似 Selenium/Playwright
- 🌐 **代理支持** — 内置 SOCKS5/HTTP 代理配置
- 📊 **结构化提取** — 支持 CSS 选择器和 JSON Schema 提取
- 🔍 **搜索引擎** — 内置 Google/Bing/DuckDuckGo 搜索

## 📦 安装

```bash
pip install super_requests
```

## 🚀 快速开始

### 基础用法

```python
from super_requests import SuperSpider

with SuperSpider() as spider:
    # 获取页面 HTML
    html = spider.fetch("https://example.com")
    
    # 获取页面信息
    title = spider.get_title()
    url = spider.get_url()
    
    # 爬取链接
    links = spider.scrape_links()
    for link in links:
        print(f"{link['text']} -> {link['href']}")
    
    # 爬取表格
    data = spider.scrape_table("table")
    for row in data:
        print(row)
```

### 带代理

```python
from super_requests import SuperSpider, BrowserConfig, ProxyConfig

proxy = ProxyConfig(
    host="50.3.64.229",
    port=443,
    username="your-username",
    password="your-password",
    protocol="socks5"
)

config = BrowserConfig(proxy=proxy)

with SuperSpider(config) as spider:
    html = spider.fetch("https://example.com")
```

### 搜索引擎

```python
with SuperSpider() as spider:
    # Google 搜索
    spider.google_search("Camoufox 反检测浏览器")
    results = spider.scrape_links("#search a")
    
    # Bing 搜索
    spider.bing_search("Python 爬虫")
    results = spider.scrape_links("#b_results a")
```

### 交互操作

```python
with SuperSpider() as spider:
    # 导航
    spider.navigate("https://example.com/login")
    
    # 填写表单
    spider.type_text('input[name="username"]', "myuser")
    spider.type_text('input[name="password"]', "mypass")
    
    # 点击按钮
    spider.click('button[type="submit"]')
    
    # 等待页面加载
    spider.wait_for("#dashboard")
    
    # 滚动加载
    spider.scroll(500)
    
    # 获取内容
    html = spider.get_html()
```

### JavaScript 执行

```python
with SuperSpider() as spider:
    spider.navigate("https://example.com")
    
    # 执行任意 JavaScript
    result = spider.evaluate("""
        Array.from(document.querySelectorAll('h2'))
            .map(el => el.textContent.trim())
    """)
    print(result["result"])  # ['标题1', '标题2', ...]
    
    # 结构化提取
    schema = {
        "type": "object",
        "properties": {
            "title": {"type": "string", "x-ref": "e1"},
            "description": {"type": "string", "x-ref": "e2"}
        }
    }
    data = spider.extract(schema)
```

## 📋 API 参考

### SuperSpider

| 方法 | 说明 |
|------|------|
| `fetch(url, wait=2.0, auto_scroll=False)` | 一步获取页面 HTML |
| `navigate(url, wait=2.0)` | 导航到 URL |
| `click(selector)` | 点击元素 |
| `type_text(selector, text, mode="fill")` | 输入文字 |
| `scroll(pixels=500)` | 滚动页面 |
| `wait_for(selector, timeout=10.0)` | 等待元素出现 |
| `evaluate(expression)` | 执行 JavaScript |
| `get_html()` | 获取页面 HTML |
| `get_text(selector="body")` | 获取元素文本 |
| `get_url()` | 获取当前 URL |
| `get_title()` | 获取页面标题 |
| `scrape_links()` | 爬取所有链接 |
| `scrape_table(selector="table")` | 爬取表格 |
| `google_search(query)` | Google 搜索 |
| `bing_search(query)` | Bing 搜索 |
| `duckduckgo_search(query)` | DuckDuckGo 搜索 |

### BrowserConfig

| 参数 | 类型 | 默认值 | 说明 |
|------|------|--------|------|
| `server_url` | str | `http://localhost:9377` | camofox-browser 服务地址 |
| `user_id` | str | `super_requests` | 用户标识 |
| `timeout` | int | `30` | 请求超时（秒） |
| `proxy` | ProxyConfig | `None` | 代理配置 |

### ProxyConfig

| 参数 | 类型 | 默认值 | 说明 |
|------|------|--------|------|
| `host` | str | - | 代理主机 |
| `port` | int | - | 代理端口 |
| `username` | str | `None` | 代理用户名 |
| `password` | str | `None` | 代理密码 |
| `protocol` | str | `socks5` | 代理协议 |

## 🏗️ 依赖

SuperSpider 需要 [camofox-browser](https://github.com/jo-inc/camofox-browser) 服务运行：

```bash
# 安装 camofox-browser
npm install -g @askjo/camofox-browser

# 启动服务
camofox-browser --port 9377
```

## 📄 License

MIT License - 详见 [LICENSE](LICENSE)

## 🔗 相关项目

- [Camoufox](https://github.com/nicoreed/camoufox) — 反检测浏览器核心
- [camofox-browser](https://github.com/jo-inc/camofox-browser) — REST API 服务
