Ecoindex_scraper module provides a way to scrape data from given website while simulating a real web browser
Project description
ECOINDEX SCRAPER PYTHON
This module provides a simple interface to get the Ecoindex of a given webpage using module ecoindex-python
Requirements
- Python ^3.8 with pip
- Google Chrome installed on your computer
Install
pip install ecoindex-scraper
Use
Get a page analysis
You can run a page analysis by calling the function get_page_analysis()
:
(function) get_page_analysis: (url: HttpUrl, window_size: WindowSize | None = WindowSize(width=1920, height=1080), wait_before_scroll: int | None = 1, wait_after_scroll: int | None = 1) -> Coroutine[Any, Any, Result]
Example:
import asyncio
from pprint import pprint
from ecoindex_scraper import get_page_analysis
page_analysis = asyncio.run(get_page_analysis(url="http://ecoindex.fr"))
pprint(page_analysis)
Result example:
Result(width=1920, height=1080, url=HttpUrl('http://ecoindex.fr', scheme='http', host='ecoindex.fr', tld='fr', host_type='domain'), size=422.126, nodes=54, requests=12, grade='A', score=86.0, ges=1.28, water=1.92, date=datetime.datetime(2021, 10, 8, 10, 20, 14, 73831), page_type=None)
Default behaviour: By default, the page analysis simulates:
- Window size of 1920x1080 pixels (can be set with parameter
window_size
)- Wait for 1 second when page is loaded (can be set with parameter
wait_before_scroll
)- Scroll to the bottom of the page (if it is possible)
- Wait for 1 second after having scrolled to the bottom of the page (can be set with parameter
wait_after_scroll
)
Contribute
You need poetry to install and manage dependencies. Once poetry installed, run :
poetry install
Tests
poetry run pytest
Contributing
Code of conduct
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
ecoindex_scraper-1.0.0.tar.gz
(5.2 kB
view hashes)
Built Distribution
Close
Hashes for ecoindex_scraper-1.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1cba7e1fa57dc1c2361624d09fd6463ab5a6381ce0d556d32a791e51aefb1d10 |
|
MD5 | 5a078b971aaa01df80863df0d10b32ec |
|
BLAKE2b-256 | 93b68938d5708cee74dd6e4887dd982439f332b3dd6de145eb7be01b779ce176 |