Skip to main content

Scrapping/Automating tools, userSwitching, anti-bot detection and more...

Project description

scrapper-boilerplate

Scrapping functions and for making easy for further scrapping

instalation

(Optional) install enviroment

make sure that you have python installed

then install:

pip install virtualenv

then install the enviroment and activate:

python -m venv venv
.\venv\scripts\activate

Install the dependêncies:

Install chromedriver click here, and get the path location.

Create .env file with:

  • chromedriver location
  • token of telegram bot (if you use telegram function)

Example provided in ".env.example"

install the packages:

pip install -r requirements.txt

and activate:

python index.py

setup enviroment

create ".env" file with following lines:

CHROMEDRIVER_PATH="Insert your chromedriver path"
TELEGRAM_TOKEN="Insert your telegram token..."

Usage

from time import sleep
from utils.setup import setSelenium
import os

ROOT_DIR = os.path.dirname(os.path.abspath(__file__))

def main():
    # import selenium initialization and browser it
    driver = setSelenium()
    driver.get('https://www.google.com')
    sleep(10)
    driver.quit()

if __name__ == "__main__":
    main()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for scrapper-boilerplate, version 0.2.0
Filename, size File type Python version Upload date Hashes
Filename, size scrapper_boilerplate-0.2.0-py3-none-any.whl (8.5 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size scrapper-boilerplate-0.2.0.tar.gz (7.3 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page