ak_requests 0.0.2

Requests package with QOL improvements and anti-bot detection measures

ak_requests

Requests package with QOL improvements and anti-bot detection measures

View Demo · Documentation · Report Bug · Request Feature

Table of Contents

• 1. About the Project
  • 1.1. Features
• 2. Getting Started
  • 2.1. Installation
    • 2.1.1. Production
    • 2.1.2. Development
• 3. Usage
• 4. Roadmap
• 5. License
• 6. Contact
• 7. Acknowledgements

1. About the Project

ak_requests is a Python package that provides an interface for automating requests tasks using requests package. It comes with quality of life improvements like retires, custom redirects, spacing out requests and shuffling requests to not trigger anti-bot measures

1.1. Features

• Bulk requests handling
• Built-in retries and timeouts
• Can log processes to file

2. Getting Started

2.1. Installation

2.1.1. Production

Download the repo and install with flit

pip install flit
flit install --deps production

Alternatively, you can use pip

pip install ak_requests

2.1.2. Development

Install with flit

  flit install --pth-file

3. Usage

from ak_requests import RequestsSession

# Initialize session
session = RequestsSession(log=False, retries=5, log_level='error') 

# Update custom header
session.update_header({'Connection': 'keep-alive'})

# set cookies
session.update_cookies([{'name':'has_recent_activity', 'value':'1'}])

# Get requests
res = session.get('https://reqres.in/api/users?page=2', data={}, proxies = {} ) # Can accept any requests parameters

# Change min time bet. requests
session.MIN_REQUEST_GAP = 1.5 # seconds

# Make bulk requests
urls = ['https://reqres.in/api/users?page=2', 'https://reqres.in/api/unknown']
responses = session.bulk_get(urls)

# use beautifulsoup
from ak_requests import soupify
res = session.get('https://reqres.in/api/users?page=2')
soup = soupify(res)

## or 
soup, res = session.soup('https://reqres.in/api/users?page=2')

## Also works for bulk requests
soups, ress = session.bulk_soup(urls)

4. Roadmap

• Add beautifulsoup integration
• Proxy

5. License

See LICENSE for more information.

6. Contact

Arun Kishore - @rpakishore

Project Link: https://github.com/rpakishore/ak_requests

7. Acknowledgements

Use this section to mention useful resources and libraries that you have used in your projects.

• Awesome README Template
• Shields.io