Modular scraping convenience framework written in Python.
Project description
scrapekit
Modular scraping convenience framework.
Convenience Methods:
scrapekit.common.get_user_agent(os, browser)
: Returns a random User-Agent string.- Can filter by OS and browser
Proxy Provider Module List:
Installation
pip install scrapekit
Usage Examples
Simple proxified session
import scrapekit
session = scrapekit.ipburger.make_session('MyIPBurgerUsername")
res = session.get('https://icanhazip.com')
print(res.status_code, res.text)
# 200 89.46.62.37
Proxified session with random Windows Firefox User-Agent:
import scrapekit
user_agent = scrapekit.common.get_user_agent(os='Windows', browser='Firefox')
session = scrapekit.ipburger.make_session(
'MyIPBurgerUsername',
headers={'User-Agent': user_agent}
)
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
scrapekit-0.4.3.tar.gz
(137.8 kB
view hashes)
Built Distribution
scrapekit-0.4.3-py3-none-any.whl
(153.4 kB
view hashes)
Close
Hashes for scrapekit-0.4.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 29309e4a8525271c989fc231908aa82f50c52777bfd1bbca37bff1866e8752b3 |
|
MD5 | dbf93b48d2d44423372cfa19e58a1e7f |
|
BLAKE2b-256 | 580d11c7f8266524c654ea77ae0b97c395d71127ada2ebb539d8c16c125a5547 |