Metadata-Version: 2.4
Name: scrape-do-python
Version: 0.3.1
Summary: A Python SDK for the Scrape.do API
Author-email: svdC1 <svdc1mail@gmail.com>
License: MIT
Project-URL: Documentation, https://svdc1.github.io/scrape-do-python
Project-URL: Repository, https://github.com/svdC1/scrape-do-python
Project-URL: Issues, https://github.com/svdC1/scrape-do-python/issues
Project-URL: Changelog, https://github.com/svdC1/scrape-do-python/blob/main/.github/CHANGELOG.md
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.9
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: httpx>=0.25.0
Requires-Dist: pydantic>=2.0.0
Requires-Dist: typing_extensions>=4.0
Provides-Extra: dev
Requires-Dist: pytest>=7.0; extra == "dev"
Requires-Dist: pytest-cov; extra == "dev"
Requires-Dist: pytest-mock; extra == "dev"
Requires-Dist: pytest-asyncio; extra == "dev"
Requires-Dist: respx; extra == "dev"
Requires-Dist: mypy; extra == "dev"
Requires-Dist: ruff; extra == "dev"
Provides-Extra: docs
Requires-Dist: mkdocs; extra == "docs"
Requires-Dist: mkdocs-material; extra == "docs"
Requires-Dist: mkdocstrings; extra == "docs"
Requires-Dist: mkdocstrings-python; extra == "docs"
Requires-Dist: mkdocs-autorefs; extra == "docs"
Requires-Dist: black; extra == "docs"
Dynamic: license-file

# scrape-do-python

[![PyPI](https://img.shields.io/pypi/v/scrape-do-python?style=flat&logo=pypi&logoColor=white)](https://pypi.org/project/scrape-do-python/)
[![Python](https://img.shields.io/pypi/pyversions/scrape-do-python?style=flat&logo=python&logoColor=white)](https://pypi.org/project/scrape-do-python/)
[![License](https://img.shields.io/badge/license-MIT-green?style=flat)](LICENSE)
[![Docs](https://img.shields.io/badge/docs-lightgrey?style=flat&logo=materialformkdocs&logoColor=black&logoSize=auto)](https://svdc1.github.io/scrape-do-python)
[![codecov](https://img.shields.io/codecov/c/github/svdC1/scrape-do-python?flag=unit&style=flat&logo=codecov&label=unit)](https://app.codecov.io/gh/svdC1/scrape-do-python?flags%5B0%5D=unit)
[![codecov](https://img.shields.io/codecov/c/github/svdC1/scrape-do-python?flag=integration&style=flat&logo=codecov&label=integration)](https://app.codecov.io/gh/svdC1/scrape-do-python?flags%5B0%5D=integration)


A Python SDK for the [`Scrape.do`](https://scrape.do) web-scraping proxy API, built on [`httpx`](https://github.com/encode/httpx) and [`pydantic v2`](https://github.com/pydantic/pydantic) 

---

## Quickstart

### Install
```bash
pip install scrape-do-python
```

### Make a Request

```python
from scrape_do import ScrapeDoClient

# API token is pulled from `SCRAPE_DO_API_KEY` 

# (override with `api_token=...`)

with ScrapeDoClient() as client:
    response = client.get(
        "https://example.com",
        super=True,
        render=True,
        return_json=True,
        screenshot=True,
    )

response.raise_for_status()
print(response.target_status_code) # 200
print(response.remaining_credits)  # 300000.0
response.screenshots[0].to_file("page.png")
```

---

## Features

<table>
  <tr>
    <td><b>Sync & Async</b></td>
    <td>Sync and Async clients for both <a href="https://svdc1.github.io/scrape-do-python/client/"><code>API Gateway</code></a> and <a href="https://svdc1.github.io/scrape-do-python/proxy_client/"><code>Proxy Mode</code></a></td>
  </tr>
  <tr>
    <td><b>Async API Support</b></td>
    <td>Submit, poll, and fetch jobs against <code>q.scrape.do</code> with the <a href="https://svdc1.github.io/scrape-do-python/async_api/client/"><code>Async API Client</code></a></td>
  </tr>
    <tr>
    <td><b>Parameter Validation</b></td>
    <td>Every request parameter is validated locally by the <a href="https://svdc1.github.io/scrape-do-python/models/parameters/#scrape_do.models.parameters.RequestParameters"><code>RequestParameters</code></a> pydantic model</td>
  </tr>
    <tr>
    <td><b>Typed Responses</b></td>
    <td><a href="https://svdc1.github.io/scrape-do-python/models/response/#scrape_do.models.response.ScrapeDoResponse"><code>ScrapeDoResponse</code></a> exposes response fields as <code>pydantic</code> models for easy access</td>
  </tr>
    <tr>
    <td><b>Browser Automation</b></td>
    <td>Build <code>playWithBrowser</code> sequences from typed <a href="https://svdc1.github.io/scrape-do-python/models/browser_actions/"><code>Browser Action Models</code></a></td>
  </tr>
    <tr>
    <td><b>Session Management</b></td>
    <td>Use a custom <a href="https://svdc1.github.io/scrape-do-python/client/#scrape_do.client.SyncSessionValidator"><code>session_validator</code></a> function to track proxy rotations when using the <code>session_id</code> parameter
  </tr>
    <tr>
    <td><b>Native Event Hooks</b></td>
    <td>Provide custom <a href="https://svdc1.github.io/scrape-do-python/client/#scrape_do.client.SyncClientEventHooks"><code>request / response / retry / poll</code></a> hooks that fire at the logical execution boundary</td>
  </tr>
    </tr>
    <tr>
    <td><b>Automatic Retries</b></td>
    <td>All clients implement automatic retries on transient gateway/network errors with a customisable <a href="https://svdc1.github.io/scrape-do-python/client/#scrape_do.client.default_backoff_strategy"><code>Backoff Strategy</code></td>
  </tr>
</table>

---

## Help

<table>
    <tr>
        <td><b><a href="https://svdc1.github.io/scrape-do-python">Documentation</a></b></td>
        <td>Full API Reference</td>
    </tr>
        <tr>
        <td><b><a href="https://svdc1.github.io/scrape-do-python/examples">Examples</a></b></td>
        <td>Curated Usage Examples</td>
    </tr>
    </tr>
        <tr>
        <td><b><a href="https://svdc1.github.io/scrape-do-python/roadmap">Roadmap</a></b></td>
        <td>What's Planned
        </tr>
        <tr>
        <td><b><a href="https://svdc1.github.io/scrape-do-python/changelog">Changelog</a></b></td>
        <td>What's Already Shipped</td>
    </tr>
</table>

---

## Contributing

> Pull Requests, bug reports, and feature requests are all welcome

> See [`CONTRIBUTING`](.github/CONTRIBUTING.md) for local setup, test commands, and PR conventions
