Metadata-Version: 2.4
Name: langchain-scrapingpros
Version: 0.1.0
Summary: LangChain integration for Scraping Pros — web scraping tools for AI agents.
Project-URL: Homepage, https://scrapingpros.com
Project-URL: Documentation, https://api.scrapingpros.com/llms-full.txt
Project-URL: Repository, https://gitlab.com/7Puentes/scrapingpros-python-sdk
Project-URL: API Reference, https://api.scrapingpros.com/docs
Project-URL: Scraping Pros, https://scrapingpros.com
Author-email: Scraping Pros <dev@scrapingpros.com>
License-Expression: MIT
Keywords: ai-agents,langchain,llm,rag,scraping,scrapingpros,web-scraping
Classifier: Development Status :: 4 - Beta
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Topic :: Internet :: WWW/HTTP
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
Classifier: Typing :: Typed
Requires-Python: >=3.10
Requires-Dist: langchain-core>=0.2.0
Requires-Dist: scrapingpros>=0.1.0
Provides-Extra: dev
Requires-Dist: pytest-asyncio>=0.23.0; extra == 'dev'
Requires-Dist: pytest>=8.0; extra == 'dev'
Description-Content-Type: text/markdown

# langchain-scrapingpros

LangChain integration for [Scraping Pros](https://scrapingpros.com) — web scraping tools for AI agents.

## Installation

```bash
pip install langchain-scrapingpros
```

## Quick Start

```python
from langchain_scrapingpros import ScrapingProsScraper

# Use demo token (5,000 credits/month, no signup)
tool = ScrapingProsScraper(api_key="demo_6x595maoA6GdOdVb")
result = tool.invoke({"url": "https://example.com"})
print(result)
```

## Tools

### ScrapingProsScraper

Scrape a web page and return clean markdown text. Ideal for RAG pipelines and LLM consumption.

```python
from langchain_scrapingpros import ScrapingProsScraper

tool = ScrapingProsScraper()  # uses SCRAPINGPROS_API_KEY env var

# Simple scrape
result = tool.invoke({"url": "https://example.com"})

# JavaScript-heavy site
result = tool.invoke({"url": "https://spa-site.com", "browser": True})
```

### ScrapingProsExtractor

Extract structured data using CSS selectors. Returns JSON.

```python
from langchain_scrapingpros import ScrapingProsExtractor

tool = ScrapingProsExtractor()
result = tool.invoke({
    "url": "https://quotes.toscrape.com/",
    "fields": {
        "quotes": "css:.text",
        "authors": "css:.author",
    },
})
```

## Use with LangChain Agents

```python
from langchain_scrapingpros import ScrapingProsScraper, ScrapingProsExtractor

tools = [ScrapingProsScraper(), ScrapingProsExtractor()]
# Pass tools to your LangChain agent
```

## Authentication

Set `SCRAPINGPROS_API_KEY` environment variable, or pass `api_key=` explicitly.

Demo token for quick testing: `demo_6x595maoA6GdOdVb` (5,000 credits/month).

Get a dedicated token at [scrapingpros.com](https://scrapingpros.com).

## Features

- Clean markdown output for AI/LLM consumption
- Automatic proxy rotation (950+ proxies, 200+ countries)
- Headless browser rendering for JavaScript sites
- CSS/XPath data extraction
- Anti-bot bypass (Camoufox anti-detection)
- CAPTCHA detection with clear error messages
- Sync and async support

## Links

- [Scraping Pros](https://scrapingpros.com)
- [API Documentation](https://api.scrapingpros.com/docs)
- [Python SDK](https://pypi.org/project/scrapingpros/)
- [Full Reference](https://api.scrapingpros.com/llms-full.txt)

## License

MIT
