Metadata-Version: 2.4
Name: crewai-alterlab
Version: 0.1.0
Summary: CrewAI integration for AlterLab web scraping API
License: MIT
License-File: LICENSE
Keywords: crewai,alterlab,web-scraping,ai-agents,web-crawler,scraping-tool
Author: AlterLab
Author-email: dev@alterlab.io
Requires-Python: >=3.10,<4.0
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Programming Language :: Python :: 3.15
Classifier: Topic :: Internet :: WWW/HTTP :: Indexing/Search
Classifier: Topic :: Software Development :: Libraries
Requires-Dist: alterlab (>=2.4.0)
Requires-Dist: crewai[tools] (>=0.80.0)
Project-URL: Documentation, https://docs.alterlab.io/integrations/crewai
Project-URL: Homepage, https://alterlab.io
Project-URL: Repository, https://github.com/RapierCraftStudios/AlterLab
Description-Content-Type: text/markdown

# crewai-alterlab

CrewAI integration for the [AlterLab](https://alterlab.io) web scraping API.

## Installation

```bash
pip install crewai-alterlab
```

## Setup

Set your AlterLab API key:

```bash
export ALTERLAB_API_KEY="your-api-key"
```

Get your API key at [alterlab.io/dashboard](https://alterlab.io/dashboard).

## Scrape Tool

Scrape individual web pages with `AlterLabScrapeTool`:

```python
from crewai_alterlab import AlterLabScrapeTool

tool = AlterLabScrapeTool()
result = tool.run("https://example.com")
print(result)
```

### With a CrewAI Agent

```python
from crewai import Agent, Task, Crew
from crewai_alterlab import AlterLabScrapeTool

researcher = Agent(
    role="Web Researcher",
    goal="Research topics by scraping web pages",
    backstory="You are an expert web researcher.",
    tools=[AlterLabScrapeTool()],
)

task = Task(
    description="Scrape https://example.com and summarise the content",
    expected_output="A summary of the page content",
    agent=researcher,
)

crew = Crew(agents=[researcher], tasks=[task])
result = crew.kickoff()
```

### Custom Parameters

```python
tool = AlterLabScrapeTool(
    scrape_params={
        "formats": ["markdown"],
        "mode": "js",  # JavaScript rendering
    },
)
```

## Crawl Tool

Crawl multiple pages from a website with `AlterLabCrawlTool`:

```python
from crewai_alterlab import AlterLabCrawlTool

tool = AlterLabCrawlTool()
result = tool.run(url="https://docs.example.com", max_pages=5)
print(result)
```

### Research Agent with Both Tools

```python
from crewai import Agent
from crewai_alterlab import AlterLabScrapeTool, AlterLabCrawlTool

researcher = Agent(
    role="Deep Researcher",
    goal="Thoroughly research topics using web data",
    backstory="You gather comprehensive information from the web.",
    tools=[AlterLabScrapeTool(), AlterLabCrawlTool()],
)
```

## API Reference

See [docs.alterlab.io](https://docs.alterlab.io) for the full AlterLab API documentation.

## Search Tool

Run web searches with `AlterLabSearchTool` (numbered title, URL and snippet):

```python
from crewai_alterlab import AlterLabSearchTool

tool = AlterLabSearchTool(num_results=5, search_params={"engine": "google"})
print(tool.run(query="web scraping api"))
```

## Blocked-Fetch Fallback

`BlockedFetchFallbackTool` runs your own fetch first and retries through AlterLab only when
the result is a 403/429, a bot-protection challenge page, a CAPTCHA, or an empty JavaScript
shell. Normal pages, 404s and network errors are returned or raised untouched, so credits are
spent only on blocked fetches:

```python
import requests
from crewai_alterlab import BlockedFetchFallbackTool

tool = BlockedFetchFallbackTool(fetch=lambda url: requests.get(url, timeout=20))
print(tool.run(url="https://example.com"))
```

