A utility for scrapping images from a HTML doc from a URL.
Project description
just-another-imgscrapper
A utility for scrapping images from a HTML doc.
Uses asyncio for fast concurrent download.
Installation
Binary installers for the latest released version are available at the Python Package Index (PyPI).
$ pip install just-another-imgscrapper
Usage
1. From cli
$ imgscrapper -h
To get HTML doc, extract image links from src attribute of <img> tags and download.
$ imgscrapper "http://foo.com/bar"
[2023-06-06 23:22:56] imgscrapper.utils:INFO: ### Initializing Scrapping ###
[2023-06-06 23:23:01] imgscrapper.utils:INFO: ### Downloaded 41 images out of extracted 41 links ###
Downloads to imgs/ dir in working dir. If dir does not exists, creates.
2. From module
>>> from imgscrapper import ImgScrapper
>>> d = ImgScrapper()
>>> d.download("http://foo.com/bar")
>>> 3
Specify path to store downloaded images.
>>> d = ImgScrapper()
>>> d.url = "http://foo.com/bar"
>>> d.path = "/path/download"
>>> d.download() # returns no. of successful downloads
>>> 3
Some servers will block the scrapping, respect robots.txt and only used in allowed hosts.
You can add request headers.
>>> ...
>>> d.request_header = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0',
'DNT': '1',
}
>>> ...
You can select specific type of img tags only by passing attrs dict.
<!-- >http://helloworld.com<-->
<html>
<body>
<img src="https://foo.com/bar.png" class="apple ball">
<img src="/foo.jpg" class="cat bar">
</body>
<html>
To select only images with class: cat
>>> d = ImgScrapper()
>>> d.url = "http://helloworld.com"
>>> d.attrs = {
'class': 'cat',
}
>>> d.download()
>>> 1 # http://helloworld.com/foo.jpg
The downloader gives unique uuid filename to downloaded images preserving the image extension.
>>> d = ImgScrapper(
url = "http://helloworld.com",
attrs = {'class': 'cat'},
max = 5,
path = "/home/images"
)
>>> d.download()
>>> 5
Limit the no. of image downloads by passing max value.
Liscense
just-another-imgscrapper is released under the MIT liscense. See LISCENSE for details.
Contact
Connect with me on twitter @deshritbaral
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for just-another-imgscrapper-0.1.2.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | af2f3405e33c90b017c341e8ef16e8cbd5942111ce8f795a86059e6546019407 |
|
| MD5 | 3db3b547c618c64a9636985f1c00039d |
|
| BLAKE2b-256 | 41332f0b0bf0c6920b045b93891684c33e356ca9ccad1afe87d64a498152b4a9 |
Hashes for just_another_imgscrapper-0.1.2-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 1f0f8143293756a8373d9e443033d5349e911950eb4004a978f5c64baee01c35 |
|
| MD5 | 35d8668ddfb117ce905109d795a8ac00 |
|
| BLAKE2b-256 | 0e47b6a34ff5d4a3ec0d127df541d480f200fe74e9da131211aeabe3e9db4be6 |
