Metadata-Version: 2.5
Name: scrapeunblocker-cloud
Version: 0.1.0
Summary: Official Python client and CLI for ScrapeUnblocker Cloud - deploy, run, schedule and pull data from Scrapy spiders hosted on ScrapeUnblocker Cloud. Traffic goes through ScrapeUnblocker by default, so spiders that get blocked elsewhere run here with no proxy or fingerprint code.
Project-URL: Homepage, https://scrapeunblocker.com
Project-URL: Documentation, https://developers.scrapeunblocker.com
Project-URL: Source, https://github.com/ScrapeUnblocker/scrapeunblocker-cloud-python
Author-email: ScrapeUnblocker <support@scrapeunblocker.com>
License-Expression: MIT
License-File: LICENSE
Keywords: anti-bot,crawler,deploy scrapy,scraper,scraping api,scrapinghub alternative,scrapy,scrapy cloud,spider hosting,web scraping,zyte alternative
Classifier: Development Status :: 4 - Beta
Classifier: Framework :: Scrapy
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Typing :: Typed
Requires-Python: >=3.8
Description-Content-Type: text/markdown

# ScrapeUnblocker Cloud - Python client & CLI

Deploy, run, schedule and pull data from Scrapy spiders hosted on **ScrapeUnblocker Cloud**,
from Python or the command line.

ScrapeUnblocker Cloud is managed Scrapy hosting: you push an unmodified Scrapy project, buy
concurrency slots and run it. The difference from other Scrapy clouds is that traffic goes
through ScrapeUnblocker by default, so a spider that gets blocked elsewhere runs here without
you writing any proxy or fingerprint code (`--unblock`).

```bash
pip install scrapeunblocker-cloud
```

No runtime dependencies: this package is pure standard library, so installing it next to your
Scrapy project never pulls in a conflicting dependency tree.

## Get a token

Create an API token in the [ScrapeUnblocker portal](https://app.scrapeunblocker.com) under
Spider Cloud. Then either run `su-cloud login`, or set the environment:

```bash
export SU_CLOUD_TOKEN="sk_..."
export SU_CLOUD_ORG="acme"
export SU_CLOUD_PROJECT="shop"     # defaults to the current directory name
```

## CLI

```bash
su-cloud login                              # store url/token/org/project in ~/.su-cloud.json
su-cloud deploy                             # package the current Scrapy project and build it
su-cloud run products -a category=shoes --unblock --wait
su-cloud jobs --state running
su-cloud logs j1699999999abc --lines 200
su-cloud items j1699999999abc > out.jsonl   # page scraped items to stdout as JSON lines
su-cloud download j1699999999abc -o out.jsonl.gz   # download the whole dataset
su-cloud stats j1699999999abc              # crawl statistics
su-cloud schedules
su-cloud schedule-add nightly "0 2 * * *" products --unblock
su-cloud destinations
su-cloud destination-add mymongo mongodb -s uri=mongodb+srv://... -s database=shop -s collection=products
```

## Library

```python
from scrapeunblocker_cloud import SpiderCloudClient

client = SpiderCloudClient(token="sk_...", org="acme", project="shop")

# deploy the current directory, then run a spider through ScrapeUnblocker
client.deploy(".", notes="add price field")
job = client.run("products", args={"category": "shoes"}, unblock=True)

job.wait()                       # block until it finishes
print(job.state, job.items_total)

for item in job.items():         # stream the scraped data straight back
    print(item)

for line in job.logs(lines=100):
    print(line)
```

Org and project set on the client are the defaults for every call, so single-project users
never repeat themselves. Every method also accepts `org=` / `project=` to override per call.

### What you can reach

| Namespace | Methods |
|-----------|---------|
| `client.orgs` | `list`, `create` |
| `client.projects` | `list`, `create`, `settings`, `update_settings` |
| `client.deploys` | `list`, `upload`, `from_git` |
| `client.jobs` | `run`, `list`, `get` |
| `Job` | `wait`, `refresh`, `logs`, `items`, `items_page`, `download`, `stats`, `cancel`, `.state`, `.done` |
| `client.schedules` | `list`, `create`, `toggle`, `delete` |
| `client.destinations` | `list`, `create`, `delete` |

## Errors

Everything raises a subclass of `SpiderCloudError`: `AuthError` (bad token), `NotFoundError`
(unknown org/project/job), `APIError` (other non-2xx, with `.status` and `.detail`),
`ConnectionFailed` (control plane unreachable) and `ConfigError` (missing token/org/project).

## License

MIT
