Skip to main content

Scrapping/Automating tools, userSwitching, anti-bot detection and more...

Project description

scrapper-boilerplate

Scrapping functions and for making easy for further scrapping

instalation

(Optional) install enviroment

make sure that you have python installed

then install:

pip install virtualenv

then install the enviroment and activate:

python -m venv venv
.\venv\scripts\activate

Install the dependêncies:

Install chromedriver click here, and get the path location.

Create .env file with:

  • chromedriver location
  • token of telegram bot (if you use telegram function)

Example provided in ".env.example"

install the packages:

pip install -r requirements.txt

and activate:

python index.py

setup enviroment

create ".env" file with following lines:

CHROMEDRIVER_PATH="Insert your chromedriver path"
TELEGRAM_TOKEN="Insert your telegram token..."

Usage

from time import sleep
from utils.setup import setSelenium
import os

ROOT_DIR = os.path.dirname(os.path.abspath(__file__))

def main():
    # import selenium initialization and browser it
    driver = setSelenium()
    driver.get('https://www.google.com')
    sleep(10)
    driver.quit()

if __name__ == "__main__":
    main()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapper-boilerplate-0.2.6.tar.gz (10.1 kB view hashes)

Uploaded Source

Built Distribution

scrapper_boilerplate-0.2.6-py3-none-any.whl (12.0 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page