This library is aimed at speeding up making of scrappers. The 'utils.py' file contains useful functions which allows users to raise alerts, get random user agents and hit URLs at random. This also reduces chances of getting blocked by the server
Project description
This library contains many useful function which will speed up the making of scrappers. The library focusses on efforts to ensure that the scrappers don't get blocked by the servers. It has functions which:
- Returns a random user agent out of 10,000 user agents. The user agents are stored in a csv file inside the package.
- Randomly hit various URLs provided by the users
- Raise Alert in the form of e-mail and/or beeps
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapper-cf-1.0.6.tar.gz
(714.7 kB
view details)
File details
Details for the file scrapper-cf-1.0.6.tar.gz.
File metadata
- Download URL: scrapper-cf-1.0.6.tar.gz
- Upload date:
- Size: 714.7 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.2.0 pkginfo/1.6.1 requests/2.22.0 setuptools/45.2.0 requests-toolbelt/0.9.1 tqdm/4.54.1 CPython/3.8.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
9c17afdb51b2af5450a45705d928115b6e70253bf38f8a33a801091c5348f04e
|
|
| MD5 |
28733a69c192e32f2961c70fb9520471
|
|
| BLAKE2b-256 |
556390aa5bd93a64300952a549a82c69fa05ba24e2724da2422d86a0256a98ca
|
