Scrapping/Automating tools, userSwitching, anti-bot detection and more...
Project description
scrapper-boilerplate
Scrapping functions and for making easy for further scrapping
instalation
(Optional) install enviroment
make sure that you have python installed
then install:
pip install virtualenv
then install the enviroment and activate:
python -m venv venv
.\venv\scripts\activate
Install the dependêncies:
Install chromedriver click here, and get the path location.
Create .env file with:
- chromedriver location
- token of telegram bot (if you use telegram function)
Example provided in ".env.example"
install the packages:
pip install -r requirements.txt
and activate:
python index.py
setup enviroment
create ".env" file with following lines:
CHROMEDRIVER_PATH="Insert your chromedriver path"
TELEGRAM_TOKEN="Insert your telegram token..."
Usage
from time import sleep
from utils.setup import setSelenium
import os
ROOT_DIR = os.path.dirname(os.path.abspath(__file__))
def main():
# import selenium initialization and browser it
driver = setSelenium()
driver.get('https://www.google.com')
sleep(10)
driver.quit()
if __name__ == "__main__":
main()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapper-boilerplate-0.2.8.tar.gz
(10.2 kB
view hashes)
Built Distribution
Close
Hashes for scrapper-boilerplate-0.2.8.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3f8f9bedd8ddd6ac822dd225f34202ea6195a84b3a3f7821b0b0dc1028383d78 |
|
MD5 | 97ebfd9ef75f2eee40651e192727193c |
|
BLAKE2b-256 | e887921b067ff1460b3abf8f7ccd98a559b7b077d99d030ce0891b183c94e30a |
Close
Hashes for scrapper_boilerplate-0.2.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ebb68dec238bc280eab0de712738c18b7a5b6c9bf7f8919e5bfdee2cc0777b8f |
|
MD5 | fed24c45608a60518ce65b2d48f37a57 |
|
BLAKE2b-256 | 8abac26df426e8e49fdf5552589eb692e0721c922ce2aaafaaca718862615132 |