Metadata-Version: 2.5
Name: litescrape-sdk
Version: 0.5.0
Summary: Python SDK for the Litescrape API: validated, batched, retrying calls to search, maps, and reviews endpoints.
Project-URL: Homepage, https://litescrape.com
Project-URL: Source, https://github.com/litescrape/litescrape-sdk
Author-email: Litescraper <support@litescrape.com>
License-Expression: MIT
License-File: LICENSE
Keywords: api,google,litescrape,maps,reviews,scraping,serp
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: httpx<1,>=0.27
Requires-Dist: pydantic<3,>=2.7
Requires-Dist: tqdm<5,>=4.66
Description-Content-Type: text/markdown

# litescrape-sdk

Python SDK for the [Litescrape API](https://litescrape.com): validated, concurrent, retrying calls to the
Google, Bing, DuckDuckGo, Yelp, Tripadvisor, Apple Maps, Google Play, and Apple App Store endpoints, with results returned in input order.

```
pip install litescrape-sdk
```

```python
import os
from litescrape_sdk import GoogleMaps, scrape

os.environ["LITESCRAPE_API_KEY"] = "ls_live_..."

results = scrape(
    [
        {"endpoint": "google_search", "q": "coffee grinders", "gl": "us"},
        GoogleMaps(q="coffee", type="search", ll="@40.745,-73.988,14z"),
    ]
)
for result in results:
    print(result.ok, result.data or result.error)
```

`help(litescrape_sdk.scrape)` documents every argument, the retry and concurrency rules, and the error
types. `ascrape` is the same function for asyncio code, and `litescrape_sdk.REQUEST_TYPES` maps each
`endpoint` slug to its request class.

Development: `uv sync`, then `uv run pytest`, `uv run ruff check .`, and `uv run ruff format .`.

## Durable batches and recovery

Requires an API deployment with durable batch support. Set `batched=True` to submit every query
before polling for completion. The server continues processing accepted jobs if your computer
disconnects or the Python process exits. Responses are retained for 24 hours after completion.

```python
from litescrape_sdk import GoogleSearch, scrape

queries = [GoogleSearch(q="coffee grinders"), GoogleSearch(q="espresso machines")]
results = scrape(queries, batched=True, cache_path="./my-batch.sqlite3")

# After a crash or disconnect, rerun with the same inputs, key and cache path:
results = scrape(queries, batched=True, useCache=True, cache_path="./my-batch.sqlite3")
for result in results:
    print(result.job_id, result.data or result.error)
```

The SDK always saves submission intentions and job IDs locally in batch mode. `useCache=False`
is the default and creates fresh jobs. `useCache=True` retrieves matching saved jobs; it creates
replacements only when the server confirms a job is missing or expired. A polling outage does
not trigger a fresh charge. Reordering distinct queries preserves their cache matches; identical
queries are matched by occurrence. Keep separate cache files for independently resumable batches.
The default path is `~/.cache/litescrape/jobs.sqlite3`, configurable with `LITESCRAPE_JOB_CACHE`.
The cache stores job IDs and request fingerprints, without raw API keys or query text.

New jobs reserve one credit; terminal failures refund that reservation. Polling and retrying the
same submission do not consume more credits. Batch mode skips the synchronous whole-list balance
check so previously paid results can be recovered even with no credits remaining. Check every
`Result` for submission errors, including insufficient credits for new jobs.

`concurrency` controls simultaneous submission/poll requests (default 32); execution uses separate
server batch capacity. Network retries and polling use exponential backoff with jitter, capped at
30 seconds; retries honor `Retry-After`. All results are returned in input order. `ascrape` accepts
the same options. A server `request_timeout` applies to each worker attempt after queueing.

## Google Search fast mode

Google Search supports `GoogleSearch(q="coffee grinders", fast_mode=True)` to
return only organic results plus search metadata and parameters. This skips
AI Overview and other result groups. The default is the full response; fast mode
is unavailable on the dedicated AI Overview endpoint. Requires an API deployment
that supports `fast_mode`.

## Request deadlines

Set `request_timeout` on `scrape` or `ascrape` to apply a server deadline to every item.
Set `timeout` on an individual request to override that default. Both accept seconds greater than
0 and at most 90, including fractional seconds; omitting them preserves the API's standard behavior.

```python
from litescrape_sdk import GoogleSearch, RequestDeadlineExceededError, scrape

(result,) = scrape(
    [GoogleSearch(q="coffee grinders")],
    request_timeout=15,
    attempts=1,
)
if isinstance(result.error, RequestDeadlineExceededError):
    print(result.error.request_id, result.error.retryable, result.error.message)
else:
    print(result.raise_for_error())

# A per-item deadline also works in dictionaries.
results = scrape(
    [
        GoogleSearch(q="coffee", timeout=10),
        {"endpoint": "google_search", "q": "tea", "timeout": 20},
    ],
    attempts=1,
)
```

The deadline covers the entire API request from gateway receipt, including admission, scraping,
and billing. On expiry, the API returns HTTP 503 with the new `request_deadline_exceeded` code,
`retryable: true`, a request ID, and a description. That attempt is not charged. The API cancels
the underlying scrape to release concurrency; credit and concurrency cleanup can finish shortly
after the response.

The SDK retries this response using its usual `attempts` and `Retry-After` rules. Each attempt gets
its own deadline; use `attempts=1` as above to receive the first timeout immediately. SDK queueing,
the status check, network transit, retry waits, and the full batch are outside the server deadline.

The existing `scrape(..., timeout=120)` argument remains the HTTP transport timeout. Keep it longer
than the server deadline so the API can return its error. A client-side `TransportError` alone does
not guarantee that the request was unbilled.

## Store APIs (Alpha)

All nine Store operations use your existing key. Alpha fields depend on what the storefront supplies.

| Request class | Endpoint slug |
| --- | --- |
| `GooglePlayApps` | `google_play_apps` |
| `GooglePlayGames` | `google_play_games` |
| `GooglePlayBooks` | `google_play_books` |
| `GooglePlayMovies` | `google_play_movies` |
| `GooglePlayProduct` | `google_play_product` |
| `GooglePlayReviews` | `google_play_reviews` |
| `AppleAppStoreSearch` | `apple_app_store_search` |
| `AppleAppStoreProduct` | `apple_app_store_product` |
| `AppleAppStoreReviews` | `apple_app_store_reviews` |

```python
from litescrape_sdk import GooglePlayApps, GooglePlayProduct, AppleAppStoreSearch, scrape

results = scrape(
    [
        GooglePlayApps(q="coffee", hl="en", gl="us"),
        GooglePlayProduct(product_id="com.duolingo"),
        AppleAppStoreSearch(term="coffee", country="us", num=10),
    ]
)
for result in results:
    print(result.raise_for_error())
```

Follow `litescrape_pagination.next` or use the returned continuation with the same operation
and parameters. Each successful page consumes one call; failures do not. For Google Play,
`chart`, `next_page_token`, `section_page_token`, and `see_more_token` are mutually exclusive.
Queries exclude category filters and charts. Search text is limited to 2,048 UTF-8 bytes,
so non-ASCII characters can consume more than one byte. Apple search also limits
URL-encoded terms to 4,096 bytes. Apps/games charts require
`store_device="phone"` or an omitted device; other device storefronts are browsable without a chart. Omit `store_device` when using an apps or games query or category.
`GooglePlayGames(q=...)` uses the shared Android app search; omit `q` or choose `games_category`
to browse games. Apple review pages
are one-based; exhausted pages return an empty list. Mac reviews use newest-first ordering.

`search_metadata.raw_file` and `prettify_file`, when returned, link to authenticated response
artifacts retained for at least seven days (today and the previous seven UTC date buckets). Download them with the same bearer key; downloads are unbilled.
Apple search applies category and case-insensitive developer-name filters before the `num` ceiling.
The native search window can contain fewer matching results than that ceiling.
See the [API reference](https://litescrape.com/docs) for every parameter and response group.
