Metadata-Version: 2.4
Name: scrapy-hrequests
Version: 0.1.9
Summary: Scrapy download handler powered by hrequests
Author: Nischal Poudel
Project-URL: Homepage, https://github.com/yourusername/scrapy-hrequests
Requires-Python: >=3.8
Description-Content-Type: text/markdown
Requires-Dist: scrapy
Requires-Dist: hrequests

# scrapy-hrequests

A Scrapy download handler that uses **hrequests** instead of Scrapy's default HTTP downloader. This allows you to use `hrequests` seamlessly within your existing Scrapy spiders without changing your crawling logic.

## Features

- Simple integration with Scrapy
- Supports both HTTP and HTTPS requests
- Drop-in replacement for Scrapy's default download handler
- Uses the `hrequests` library for making requests

## Installation

```bash
pip install scrapy-hrequests
```

## Usage

Configure your spider (or project) to use the `HRequestsDownloadHandler`.

### Per Spider

```python
import scrapy

class ExampleSpider(scrapy.Spider):
    name = "example"

    custom_settings = {
        "DOWNLOAD_HANDLERS": {
            "http": "scrapy_hrequests.handler.HRequestsDownloadHandler",
            "https": "scrapy_hrequests.handler.HRequestsDownloadHandler",
        },
    }

    start_urls = [
        "https://httpbin.org/get",
    ]

    def parse(self, response):
        yield {
            "url": response.url,
            "status": response.status,
        }
```

### Project Wide

Add the following to your `settings.py`:

```python
DOWNLOAD_HANDLERS = {
    "http": "scrapy_hrequests.handler.HRequestsDownloadHandler",
    "https": "scrapy_hrequests.handler.HRequestsDownloadHandler",
}
```

## Requirements

- Python 3.8+
- Scrapy
- hrequests

## License

MIT License
