ecoindex_scraper 1.3.0

Ecoindex_scraper module provides a way to scrape data from given website while simulating a real web browser

ECOINDEX SCRAPER PYTHON

This module provides a simple interface to get the Ecoindex of a given webpage using module ecoindex-python

Requirements

• Python ^3.8 with pip
• Google Chrome installed on your computer

Install

pip install ecoindex-scraper

Use

Get a page analysis

You can run a page analysis by calling the function get_page_analysis():

(function) get_page_analysis: (url: HttpUrl, window_size: WindowSize | None = WindowSize(width=1920, height=1080), wait_before_scroll: int | None = 1, wait_after_scroll: int | None = 1) -> Coroutine[Any, Any, Result]

Example:

import asyncio
from pprint import pprint

from ecoindex_scraper import get_page_analysis

page_analysis = asyncio.run(get_page_analysis(url="http://ecoindex.fr"))
pprint(page_analysis)

Result example:

Result(width=1920, height=1080, url=HttpUrl('http://ecoindex.fr', scheme='http', host='ecoindex.fr', tld='fr', host_type='domain'), size=422.126, nodes=54, requests=12, grade='A', score=86.0, ges=1.28, water=1.92, date=datetime.datetime(2021, 10, 8, 10, 20, 14, 73831), page_type=None)

Default behaviour: By default, the page analysis simulates:

• Window size of 1920x1080 pixels (can be set with parameter window_size)
• Wait for 1 second when page is loaded (can be set with parameter wait_before_scroll)
• Scroll to the bottom of the page (if it is possible)
• Wait for 1 second after having scrolled to the bottom of the page (can be set with parameter wait_after_scroll)

Contribute

You need poetry to install and manage dependencies. Once poetry installed, run :

poetry install

Tests

poetry run pytest

Contributing

Code of conduct