A utility for scrapping images from a HTML doc from a URL.
Project description
just-another-imgscrapper
A utility for scrapping images from a HTML doc.
Uses asyncio for fast concurrent download.
Installation
$ pip install just-another-imgscrapper
Usage
1. From cli
$ imgscrapper -h
To get HTML doc, extract image links from src attribute of <img> tags and download.
$ imgscrapper "http://foo.com/bar"
[2023-06-06 23:22:56] imgscrapper.utils:INFO: ### Initializing Scrapping ###
[2023-06-06 23:23:01] imgscrapper.utils:INFO: ### Downloaded 41 images out of extracted 41 links ###
Downloads to imgs/ dir in working dir. If dir does not exists, creates.
2. From module
>>> from imgscrapper import ImgScrapper
>>> d = ImgScrapper()
>>> d.download("http://foo.com/bar")
>>> 3
Specify path to store downloaded images.
>>> d = ImgScrapper()
>>> d.url = "http://foo.com/bar"
>>> d.path = "/path/download"
>>> d.download() # returns no. of successful downloads
>>> 3
Some servers will block the scrapping, respect robots.txt and only used in allowed hosts.
You can add request headers.
>>> ...
>>> d.request_header = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0',
'DNT': '1',
}
>>> ...
You can specifically select specific type of img tags by specfying attribute of HTML element.
<!-- >http://helloworld.com<-->
<html>
<body>
<img src="https://foo.com/bar.png" class="apple ball">
<img src="/foo.jpg" class="cat bar">
</body>
<html>
To select only images with class: cat
>>> d = ImgScrapper()
>>> d.url = "http://helloworld.com"
>>> d.attrs = {
'class': 'cat',
}
>>> d.download()
>>> 1 # http://helloworld.com/foo.jpg
The downloader gives unique uuid to downloaded images preserving the image extension.
>>> d = ImgScrapper(
url = "http://helloworld.com",
attrs = {'class': 'cat'},
max = 5,
path = "/home/images"
)
>>> d.download()
>>> 5
You can limit no. of image downloads by max value.
Liscense
just-another-imgscrapper is released under the MIT liscense. See LISCENSE for details.
Contact
Follow me on twitter
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for just-another-imgscrapper-0.1.1.tar.gz
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | 86c3c05341848af68a1d32713d493938174787caf14f180f48b1153ef234b9a6 |
|
| MD5 | 45f3bd7038a67d6212ade9e0f07da669 |
|
| BLAKE2b-256 | 6c586b99b7e0f7cdb616dcbcf185759c04158a82a1f64389519fc876ab064836 |
Hashes for just_another_imgscrapper-0.1.1-py3-none-any.whl
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | bad00654ab2cf6e8f1e9af5d4ff3ef6452e1eda4129ae1e87796e091a3dc0b1c |
|
| MD5 | 08d58104e2ea1e6629b2474442d9b596 |
|
| BLAKE2b-256 | 157792f9695ebf14597ed8927c2962e46058dbe97135f4be95f48098f98c8407 |
