Metadata-Version: 2.4
Name: scrapegraphai-proxyhat
Version: 0.1.0
Summary: ProxyHat residential proxies for ScrapeGraphAI — inject a geo-targeted, sticky-or-rotating proxy straight into your graph_config.
Project-URL: Homepage, https://proxyhat.com
Project-URL: Documentation, https://docs.proxyhat.com
Project-URL: Repository, https://github.com/ProxyHatCom/scrapegraphai-proxyhat
Author-email: ProxyHat <support@proxyhat.com>
License-Expression: MIT
License-File: LICENSE
Keywords: llm-scraper,playwright,proxy,proxyhat,residential-proxy,scrapegraphai,sticky-session,web-scraping
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: Proxy Servers
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Requires-Python: >=3.11
Requires-Dist: proxyhat>=0.2.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == 'dev'
Requires-Dist: ruff>=0.4; extra == 'dev'
Provides-Extra: scrapegraphai
Requires-Dist: scrapegraphai>=1.30; extra == 'scrapegraphai'
Description-Content-Type: text/markdown

# scrapegraphai-proxyhat

Route [ScrapeGraphAI](https://github.com/ScrapeGraphAI/Scrapegraph-ai) LLM scraping graphs through [ProxyHat](https://proxyhat.com?utm_source=github&utm_medium=readme&utm_campaign=scrapegraphai) residential proxies — a geo-targeted, sticky-by-default residential IP dropped straight into your `graph_config`.

[![CI](https://github.com/ProxyHatCom/scrapegraphai-proxyhat/actions/workflows/ci.yml/badge.svg)](https://github.com/ProxyHatCom/scrapegraphai-proxyhat/actions/workflows/ci.yml)
[![Compatible with scrapegraphai latest](https://github.com/ProxyHatCom/scrapegraphai-proxyhat/actions/workflows/compat.yml/badge.svg)](https://github.com/ProxyHatCom/scrapegraphai-proxyhat/actions/workflows/compat.yml)
[![PyPI](https://img.shields.io/pypi/v/scrapegraphai-proxyhat)](https://pypi.org/project/scrapegraphai-proxyhat/)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

## Why

ScrapeGraphAI drives a real browser (Playwright) to render pages before the LLM reads them. From a datacenter IP that browser gets flagged, CAPTCHA-walled, and blocked. This package plugs ProxyHat's residential IPs (50M+ across 148+ countries) into ScrapeGraphAI through the exact hook it already supports — `graph_config["loader_kwargs"]["proxy"]` — with one pinned residential IP for the whole scrape by default, so pagination and JS interactions stay on one consistent identity. No fork, no boilerplate.

## Install

```bash
pip install scrapegraphai-proxyhat scrapegraphai
```

`scrapegraphai` is a heavy peer dependency (LLM + Playwright stack) — bring your own version. The config builders (`proxyhat_graph_config()`, `proxyhat_proxy()`) work on their own; you only need `scrapegraphai` installed to actually run a graph.

## Quick start

```python
from scrapegraphai.graphs import SmartScraperGraph
from scrapegraphai_proxyhat import proxyhat_graph_config

# An API key auto-selects an active residential sub-user:
graph_config = proxyhat_graph_config(
    {"llm": {"model": "openai/gpt-4o-mini"}},
    country="us",   # sticky US residential IP for the whole scrape
)

scraper = SmartScraperGraph(
    prompt="List every project with its title and description.",
    source="https://perinim.github.io/projects/",
    config=graph_config,
)
print(scraper.run())
```

Get an API key at [proxyhat.com](https://proxyhat.com?utm_source=github&utm_medium=readme&utm_campaign=scrapegraphai).

`proxyhat_graph_config(base_config, ...)` merges the proxy into `loader_kwargs.proxy` and **preserves everything else** in your config (`llm`, `verbose`, `headless`, any existing `loader_kwargs`). Your `base_config` is never mutated.

Just want the proxy dict for a config you build yourself?

```python
from scrapegraphai_proxyhat import proxyhat_proxy

graph_config = {
    "llm": {"model": "openai/gpt-4o-mini"},
    "loader_kwargs": {"proxy": proxyhat_proxy(country="de", sticky="1h")},
}
# proxyhat_proxy(...) -> {"server": "http://gate.proxyhat.com:8080", "username": ..., "password": ...}
```

Or the raw URL string (`proxyhat_proxy_url(...)`) for any other tool:

```python
from scrapegraphai_proxyhat import proxyhat_proxy_url

url = proxyhat_proxy_url(country="us")
# http://<user>-country-us-sid-<id>-ttl-30m:<pass>@gate.proxyhat.com:8080
```

## Credentials

Pass them explicitly or via environment variables — options win over env:

| Option | Env var | Notes |
|---|---|---|
| `api_key` | `PROXYHAT_API_KEY` | Auto-selects an active sub-user with remaining traffic |
| `sub_user` | `PROXYHAT_SUBUSER` | Pick a specific sub-user by uuid or name (with an API key) |
| `username` | `PROXYHAT_USERNAME` | Explicit gateway `proxy_username` (skips the API) |
| `password` | `PROXYHAT_PASSWORD` | Explicit gateway `proxy_password` |

## Targeting

```python
proxyhat_graph_config(
    base_config,
    protocol="http",   # or "socks5"
    country="us",      # ISO code or "any" (default)
    region="california",
    city="los_angeles",
    filter="high",     # AI IP-quality tier
    sticky="30m",      # session lifetime (default); sticky=False rotates every request
)
```

All three builders (`proxyhat_graph_config`, `proxyhat_proxy`, `proxyhat_proxy_url`) take the same targeting keyword arguments.

### Sticky IP per scrape (default)

A single scrape often hits the same site many times — rendering, scrolling, following pagination, retrying. If the exit IP changed mid-scrape the site would see a visitor teleporting between cities and block it. So this package is **sticky by default**: one residential IP is pinned for the whole run (`sticky="30m"`), keeping the session coherent.

Want a fresh IP on **every** request instead (e.g. a `SearchGraph` fanning out across many independent pages)? Turn stickiness off:

```python
proxyhat_graph_config(base_config, country="us", sticky=False)  # rotating residential IP per connection
```

Set a custom lifetime with `sticky="2h"`.

## How it works

ScrapeGraphAI reads a proxy at `graph_config["loader_kwargs"]["proxy"]` and hands it to its `ChromiumLoader` (and on to Playwright) as a Playwright-style `{server, username, password}` mapping. `proxyhat_proxy(...)` resolves your gateway credentials (via the official [`proxyhat`](https://pypi.org/project/proxyhat/) SDK — an API key auto-picks an active sub-user, or pass `username`/`password`), then builds exactly that mapping: `server` is the ProxyHat gateway (`http://gate.proxyhat.com:8080`, or `socks5://…:1080`) and `username` carries ProxyHat's targeting grammar (`<user>-country-us-sid-<id>-ttl-30m`). `proxyhat_graph_config(...)` just merges that dict into `loader_kwargs.proxy` over your config. A sticky username pins one residential IP; a rotating one makes the gateway hand out a fresh IP per connection.

## License

MIT © [ProxyHat](https://proxyhat.com)
