Metadata-Version: 2.4
Name: scraping-ai
Version: 0.2.1
Summary: Python SDK for the Scraping AI data extraction pipeline service
Author: SDT Bizdev Team
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: httpx>=0.28.1
Requires-Dist: pydantic>=2.0.0
Provides-Extra: dev
Requires-Dist: pytest>=8.0.0; extra == "dev"
Requires-Dist: pytest-asyncio>=0.23.0; extra == "dev"
Requires-Dist: respx>=0.21.0; extra == "dev"
Requires-Dist: ruff>=0.1.0; extra == "dev"

# Scraping AI Python SDK (`scraping-ai`)

A modern, type-safe Python SDK for interacting with the Scraping AI core service API endpoints. It supports both synchronous and asynchronous operations using `httpx` and `pydantic`.

This SDK focuses strictly on **core service execution endpoints** (v1 pipeline orchestration, data extraction, and v2 standalone modules), excluding account management, billing, payments, and administrative routes.

## Installation

```bash
pip install scraping-ai
```

## Core Service Usage

### 1. v1 Unified Pipeline Flow

```python
from scraping_ai import ScrapingAIClient

# Initialize client using API Key
client = ScrapingAIClient(api_key="your_api_key_here")

# Create an InputState
state = client.input_states.create(
    base_url="https://example.com",
    user_instruction="dummy_user_instruction",
    schema_instruction="dummy_schema_instruction"
)
print(f"Created input state ID: {state.id}")

# Trigger the auto-flow pipeline
flow = client.input_states.run_flow(state.id)
print(f"Flow started. First step: {flow.first_step}")

# Wait for extraction step to finish
state = client.input_states.wait_for_task(state.id, "extractor", poll_interval=5.0)
print(f"Extraction status: {state.extraction_status}")

# Fetch extracted data rows
data_res = client.data.get_by_state(state.id)
for row in data_res.results:
    print(row.data)
```

### 2. v2 Standalone Async Modules

```python
import asyncio
from scraping_ai import AsyncScrapingAIClient

async def main():
    async with AsyncScrapingAIClient(api_key="your_api_key_here") as client:
        # Start a v2 keyword generation run
        run_info = await client.v2_keywords.run(context="dummy_context", num_keywords=10)
        print(f"Accepted run for State ID {run_info.state_id}")

        # Poll status
        status_info = await client.v2_keywords.status(run_info.state_id)
        print(f"Current status: {status_info.status}")

        # Retrieve result when finished
        res = await client.v2_keywords.result(run_info.state_id)
        print(f"Keywords generated: {res.keywords}")

asyncio.run(main())
```

## Core Service Operations Summary

The SDK exposes service operations under core namespaces:

### v1 Pipeline & Data Operations
1. `client.input_states`: Create, list, retrieve, update, patch, trigger `run_flow`, and `wait_for_task` helper.
2. `client.tasks`: List available background tasks (`list_available`) and trigger tasks (`run`).
3. `client.task_entries`: Create an InputState and run a task in one call (`create`), or query task status (`get_status`).
4. `client.data`: Retrieve extracted rows (`get_by_state`) and download CSV exports (`export_by_state`).

### v2 Standalone Service Modules
1. `client.v2_keywords`: `run`, `status`, `result` for standalone keyword generation.
2. `client.v2_finder`: `run`, `status`, `result` for URL discovery.
3. `client.v2_crawler`: `run`, `status`, `result` for page crawling.
4. `client.v2_extractor`: `run`, `status`, `result` for LLM data extraction.
5. `client.v2_exports`: `create`, `list`, `get` for presigned S3 exports.
6. `client.v2_ranker`: `run`, `status`, `result` for URL relevance ranking.
7. `client.v2_schema`: `run`, `status`, `result` for JSON Schema generation.
8. `client.v2_search_url`: `run`, `status`, `result` for search URL generation.
9. `client.v2_search_results`: `run`, `status`, `result` for search results finder.
