Modular scraping convenience framework written in Python.
Project description
scrapekit
Modular scraping convenience framework.
Convenience Methods:
scrapekit.common.get_user_agent(os, browser)
: Returns a random User-Agent string.- Can filter by OS and browser
Proxy Provider Module List:
Installation
pip install scrapekit
Usage Examples
Simple proxified session
import scrapekit
session = scrapekit.ipburger.make_session('MyIPBurgerUsername")
res = session.get('https://icanhazip.com')
print(res.status_code, res.text)
# 200 89.46.62.37
Proxified session with random Windows Firefox User-Agent:
import scrapekit
user_agent = scrapekit.common.get_user_agent(os='Windows', browser='Firefox')
session = scrapekit.ipburger.make_session(
'MyIPBurgerUsername',
headers={'User-Agent': user_agent}
)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapekit-0.3.0.tar.gz
(137.7 kB
view hashes)
Built Distribution
scrapekit-0.3.0-py3-none-any.whl
(153.3 kB
view hashes)
Close
Hashes for scrapekit-0.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b7fe20d5626de98079182ed09bd19ac63b9fd588798dbfb5a997883c7ec4264a |
|
MD5 | b9cc18de656f3a9567e73cb1e0a4881a |
|
BLAKE2b-256 | b66ab33ff25906393d9652e68f500c201840e78d0aeac8cd69e5df51bdccd808 |