Metadata-Version: 2.1
Name: just-another-imgscrapper
Version: 0.1.2
Summary: A utility for scrapping images from a HTML doc from  a URL.
Home-page: https://github.com/deshrit/just-another-imgscrapper
Author: Deshrit Baral
Author-email: deshritbaral@gmail.com
License: MIT
Keywords: image,scrapper,asyncio,httpx,beautifulsoup4,lxml
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Requires-Python: >=3.7
Description-Content-Type: text/markdown
Provides-Extra: dev
License-File: LICENSE

# just-another-imgscrapper
![](https://github.com/deshrit/just-another-imgscrapper/actions/workflows/tests.yml/badge.svg)

A utility for scrapping images from a HTML doc.

Uses `asyncio` for fast concurrent download.

## Installation
Binary installers for the latest released version are available at the [Python Package Index (PyPI)](https://pypi.org/project/just-another-imgscrapper/).
```bash
$ pip install just-another-imgscrapper
```
## Usage
### 1. From cli
```bash
$ imgscrapper -h
```
To get HTML doc, extract image links from `src` attribute of `<img>` tags and download.
```
$ imgscrapper "http://foo.com/bar"
[2023-06-06 23:22:56] imgscrapper.utils:INFO: ### Initializing Scrapping ###
[2023-06-06 23:23:01] imgscrapper.utils:INFO: ### Downloaded 41 images out of extracted 41 links ###
```
Downloads to `imgs/` dir in working dir. If dir does not exists, creates.

### 2. From module
```python
>>> from imgscrapper import ImgScrapper
>>> d = ImgScrapper()
>>> d.download("http://foo.com/bar") 
>>> 3
```
Specify path to store downloaded images.
```python
>>> d = ImgScrapper()
>>> d.url = "http://foo.com/bar"
>>> d.path = "/path/download"
>>> d.download() # returns no. of successful downloads
>>> 3
```
Some servers will block the scrapping, respect robots.txt and only used in allowed hosts.

You can add request headers.
```python
>>> ...
>>> d.request_header = {
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0',
    'DNT': '1',
    }
>>> ...
```
You can select specific type of `img` tags only by passing `attrs` dict.
```html
<!-- >http://helloworld.com<-->
<html>
    <body>
        <img src="https://foo.com/bar.png" class="apple ball">
        <img src="/foo.jpg" class="cat bar">
    </body>
<html>
```
To select only images with `class: cat`
```python
>>> d = ImgScrapper()
>>> d.url = "http://helloworld.com"
>>> d.attrs = {
    'class': 'cat',
    }
>>> d.download()
>>> 1 # http://helloworld.com/foo.jpg
```
The downloader gives unique `uuid` filename to downloaded images preserving the image extension.
```python
>>> d = ImgScrapper(
    url = "http://helloworld.com",
    attrs = {'class': 'cat'},
    max = 5,
    path = "/home/images"
)
>>> d.download()
>>> 5
```
Limit the no. of image downloads by passing `max` value.

## Liscense
`just-another-imgscrapper` is released under the MIT liscense. See LISCENSE for details.

## Contact
Connect with me on twitter [@deshritbaral](https://twitter.com/deshritbaral)
