Metadata-Version: 2.5
Name: llama-index-readers-scrapeless
Version: 0.1.0
Summary: Scrapeless Python SDK readers for LlamaIndex
Project-URL: SDK Reference, https://docs.scrapeless.com/en/docs/developer-tools/sdks/python-sdk/?utm_source=pypi&utm_medium=referral&utm_campaign=llamaindex-scrapeless&utm_content=package-description
License-Expression: MIT
License-File: LICENSE
Keywords: llamaindex,rag,readers,scrapeless,web-scraping
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 3.14
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=3.10
Requires-Dist: beautifulsoup4<5,>=4.12
Requires-Dist: llama-index-core<0.15,>=0.14.25
Requires-Dist: python-dotenv<2,>=1
Requires-Dist: scrapeless==1.3.0
Provides-Extra: dev
Requires-Dist: build>=1; extra == 'dev'
Requires-Dist: markdown-it-py<5,>=4; extra == 'dev'
Requires-Dist: packaging>=24; extra == 'dev'
Requires-Dist: pytest-cov>=6; extra == 'dev'
Requires-Dist: pytest>=8; extra == 'dev'
Requires-Dist: ruff>=0.11; extra == 'dev'
Requires-Dist: tomli>=2; (python_version < '3.11') and extra == 'dev'
Requires-Dist: twine>=6; extra == 'dev'
Description-Content-Type: text/markdown

# Scrapeless Reader for LlamaIndex

Load completed Scrapeless results as LlamaIndex `Document` objects. The Reader uses the Scrapeless Python SDK and retains source information for downstream indexing and retrieval.

## Install

Use Python 3.10 or later:

```bash
python -m pip install llama-index-readers-scrapeless==0.1.0
```

Set `SCRAPELESS_API_KEY` to a key from your [Scrapeless dashboard](https://app.scrapeless.com/?utm_source=pypi&utm_medium=referral&utm_campaign=llamaindex-scrapeless&utm_content=package-description). On macOS or Linux:

```bash
export SCRAPELESS_API_KEY="YOUR_SCRAPELESS_API_KEY"
```

## Load a page

Save the following as `quickstart.py` in your working directory:

```python
from llama_index.readers.scrapeless import ScrapelessReader

reader = ScrapelessReader(timeout=120, request_timeout=30)
documents = reader.load_data(
    "scrape",
    params={
        "url": "https://example.com",
        "options": {"formats": ["markdown"], "onlyMainContent": True},
    },
)

for document in documents:
    print("Source:", document.metadata.get("url"))
    print(document.text[:300])
```

Run `python quickstart.py`. This sends a live Scrapeless request. Check that the returned text describes the example domain and that the source URL matches the requested page. An empty list means no usable documents were returned. Failed or malformed task results raise an error.

## Supported services

| Product | Reader service | Input |
| --- | --- | --- |
| Scraping API | `google_search`, `scraping` | Search input, or an actor and its input |
| Crawl | `scrape`, `batch_scrape`, `crawl` | Page URL or batch URL list, with optional Crawl settings |
| Web Unlocker | `web_unlocker` | Actor and request input |
| AI Scraper | `ai_scraper` | Actor and engine-specific input |

See the [Scrapeless Python SDK reference](https://docs.scrapeless.com/en/docs/developer-tools/sdks/python-sdk/?utm_source=pypi&utm_medium=referral&utm_campaign=llamaindex-scrapeless&utm_content=package-description) for product inputs. The Reader returns `list[Document]` across all seven routes. Choose your own LlamaIndex indexing, storage, and model configuration for downstream use.

On a polling timeout or a task result error, retain the original task ID when available. Polling timeout does not cancel the remote job. Inspect the original job before deciding whether to resume it; the Reader does not automatically submit a replacement.

The source distribution includes the repository examples, usage notes, and tests. Download the source archive from the project's PyPI files if you want to run those examples. The installed wheel is sufficient for the standalone quick start above.

For support, contact [support@scrapeless.com](mailto:support@scrapeless.com) with the version, reproduction steps, and a sanitized error message.
