Metadata-Version: 2.2
Name: scrapy-requests-spider-closed
Version: 0.0.1
Summary: scrapy-requests-spider-closed is an extension to make requests when the spider is closed
Author: Xavier Zambrano
License: Copyright (c) 2012-2024 Scott Chacon and others
        
        Permission is hereby granted, free of charge, to any person obtaining
        a copy of this software and associated documentation files (the
        "Software"), to deal in the Software without restriction, including
        without limitation the rights to use, copy, modify, merge, publish,
        distribute, sublicense, and/or sell copies of the Software, and to
        permit persons to whom the Software is furnished to do so, subject to
        the following conditions:
        
        The above copyright notice and this permission notice shall be
        included in all copies or substantial portions of the Software.
        
        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
        EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
        MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
        NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
        LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
        OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
        WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Classifier: Programming Language :: Python :: 3
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: scrapy
Requires-Dist: requests

# scrapy-requests-spider-closed

scrapy-requests-spider-closed is an extension to make requests when the spider is closed.

Use cases: 
- You must make a request to an API when the spider is closed.
- You make a request to an API when the scraping is finished, your API will create a report with the data scraped.
- Etc.

## Installation
Install scrapy-requests-spider-closed using pip:
```bash
pip install git+https://github.com/XavierZambrano/scrapy-requests-spider-closed.git
```


## Configuration
1. Add the `REQUESTS_SPIDER_CLOSED_REASONS_TRIGGER` in your `settings.py` file.
```python
REQUESTS_SPIDER_CLOSED_REASONS_TRIGGER = ['finished', 'cancelled', 'shutdown']
```
The `REQUESTS_SPIDER_CLOSED_REASONS_TRIGGER` is a list of reasons that will trigger the requests. [reasons](https://docs.scrapy.org/en/latest/topics/signals.html?highlight=finish%20reason#spider-closed).

2. Add the `REQUESTS_SPIDER_CLOSED_REQUESTS_PARAMETERS` in your `settings.py` file.
```python
REQUESTS_SPIDER_CLOSED_REQUESTS_PARAMETERS = [
    {
        'method': 'POST',
        'url': 'https://myapi-xxx.com/endpoint',
    }
]
```
You can use any parameter that the [requests](https://requests.readthedocs.io/en/latest/api/) library accepts.

3. Enable `SpiderClosedRequests` by adding it to the `EXTENSIONS` in your `settings.py` file.
```python
EXTENSIONS = {
   "scrapy_requests_spider_closed.RequestsSpiderClosed": 0,
}
```
