Modular scraping convenience framework written in Python.
Project description
scrapekit
Modular scraping convenience framework.
Convenience Methods:
scrapekit.common.get_user_agent(os, browser)
: Returns a random User-Agent string.- Can filter by OS and browser
Proxy Provider Module List:
Installation
pip install scrapekit
Usage Examples
Simple proxified session
import scrapekit
session = scrapekit.ipburger.make_session('MyIPBurgerUsername")
res = session.get('https://icanhazip.com')
print(res.status_code, res.text)
# 200 89.46.62.37
Proxified session with random Windows Firefox User-Agent:
import scrapekit
user_agent = scrapekit.common.get_user_agent(os='Windows', browser='Firefox')
session = scrapekit.ipburger.make_session(
'MyIPBurgerUsername',
headers={'User-Agent': user_agent}
)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapekit-0.4.2.tar.gz
(137.8 kB
view hashes)
Built Distribution
scrapekit-0.4.2-py3-none-any.whl
(153.4 kB
view hashes)
Close
Hashes for scrapekit-0.4.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b9e612ca7d450ef4cf302b26105f85daf2d40b48c7f88c3016ffd8fd754d93cc |
|
MD5 | 34b85a57507fa0c62f37b7dc6fc15a0c |
|
BLAKE2b-256 | 88ccb26c35ccbecb253c85bdda836e6aae551793597320d31165724f6888cc60 |