Modular scraping convenience framework written in Python.
Project description
scrapekit
Modular scraping convenience framework.
Convenience Methods:
scrapekit.common.get_user_agent(os, browser)
: Returns a random User-Agent string.- Can filter by OS and browser
Proxy Provider Module List:
Installation
pip install scrapekit
Usage Examples
Simple proxified session
import scrapekit
session = scrapekit.ipburger.make_session('MyIPBurgerUsername")
res = session.get('https://icanhazip.com')
print(res.status_code, res.text)
# 200 89.46.62.37
Proxified session with random Windows Firefox User-Agent:
import scrapekit
user_agent = scrapekit.common.get_user_agent(os='Windows', browser='Firefox')
session = scrapekit.ipburger.make_session(
'MyIPBurgerUsername',
headers={'User-Agent': user_agent}
)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapekit-0.4.0.tar.gz
(137.6 kB
view hashes)
Built Distribution
scrapekit-0.4.0-py3-none-any.whl
(153.3 kB
view hashes)
Close
Hashes for scrapekit-0.4.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d21193b1c5bfa77055f91bcb9be730c4d6ab88dda385c9b6c5384536148f9cf4 |
|
MD5 | bdc296ec54d02b331a41f0e9b80cbecc |
|
BLAKE2b-256 | 80d9cf35f95a2d55baee67724251fb0fa3723097c21aaa46a0a058f3d6f95e2c |