scrapekit 0.4.3

Modular scraping convenience framework written in Python.

scrapekit

Modular scraping convenience framework.

Convenience Methods:

• scrapekit.common.get_user_agent(os, browser): Returns a random User-Agent string.
  • Can filter by OS and browser

Proxy Provider Module List:

• IP Burger

Installation

pip install scrapekit

Usage Examples

Simple proxified session

import scrapekit

session = scrapekit.ipburger.make_session('MyIPBurgerUsername")

res = session.get('https://icanhazip.com')
print(res.status_code, res.text)
# 200 89.46.62.37

Proxified session with random Windows Firefox User-Agent:

import scrapekit

user_agent = scrapekit.common.get_user_agent(os='Windows', browser='Firefox')
session = scrapekit.ipburger.make_session(
    'MyIPBurgerUsername',
    headers={'User-Agent': user_agent}
)