A utility for scrapping images from a HTML doc from a URL.
Project description
just-another-imgscrapper
A utility for scrapping images from a HTML doc.
Uses asyncio for fast concurrent download.
Installation
Binary installers for the latest released version are available at the Python Package Index (PyPI).
$ pip install just-another-imgscrapper
Usage
1. From cli
$ imgscrapper -h
To get HTML doc, extract image links from src attribute of <img> tags and download.
$ imgscrapper "http://foo.com/bar"
[2023-06-06 23:22:56] imgscrapper.utils:INFO: ### Initializing Scrapping ###
[2023-06-06 23:23:01] imgscrapper.utils:INFO: ### Downloaded 41 images out of extracted 41 links ###
Downloads to imgs/ dir in working dir. If dir does not exists, creates.
2. From module
>>> from imgscrapper import ImgScrapper
>>> d = ImgScrapper()
>>> d.download("http://foo.com/bar")
>>> 3
Specify path to store downloaded images.
>>> d = ImgScrapper()
>>> d.url = "http://foo.com/bar"
>>> d.path = "/path/download"
>>> d.download() # returns no. of successful downloads
>>> 3
Some servers will block the scrapping, respect robots.txt and only used in allowed hosts.
You can add request headers.
>>> ...
>>> d.request_header = {
'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0',
'DNT': '1',
}
>>> ...
You can select specific type of img tags only by passing attrs dict.
<!-- >http://helloworld.com<-->
<html>
<body>
<img src="https://foo.com/bar.png" class="apple ball">
<img src="/foo.jpg" class="cat bar">
</body>
<html>
To select only images with class: cat
>>> d = ImgScrapper()
>>> d.url = "http://helloworld.com"
>>> d.attrs = {
'class': 'cat',
}
>>> d.download()
>>> 1 # http://helloworld.com/foo.jpg
The downloader gives unique uuid filename to downloaded images preserving the image extension.
>>> d = ImgScrapper(
url = "http://helloworld.com",
attrs = {'class': 'cat'},
max = 5,
path = "/home/images"
)
>>> d.download()
>>> 5
Limit the no. of image downloads by passing max value.
Liscense
just-another-imgscrapper is released under the MIT liscense. See LISCENSE for details.
Contact
Connect with me on twitter @deshritbaral
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file just-another-imgscrapper-0.1.2.tar.gz.
File metadata
- Download URL: just-another-imgscrapper-0.1.2.tar.gz
- Upload date:
- Size: 8.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
af2f3405e33c90b017c341e8ef16e8cbd5942111ce8f795a86059e6546019407
|
|
| MD5 |
3db3b547c618c64a9636985f1c00039d
|
|
| BLAKE2b-256 |
41332f0b0bf0c6920b045b93891684c33e356ca9ccad1afe87d64a498152b4a9
|
File details
Details for the file just_another_imgscrapper-0.1.2-py3-none-any.whl.
File metadata
- Download URL: just_another_imgscrapper-0.1.2-py3-none-any.whl
- Upload date:
- Size: 7.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.6
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
1f0f8143293756a8373d9e443033d5349e911950eb4004a978f5c64baee01c35
|
|
| MD5 |
35d8668ddfb117ce905109d795a8ac00
|
|
| BLAKE2b-256 |
0e47b6a34ff5d4a3ec0d127df541d480f200fe74e9da131211aeabe3e9db4be6
|
