Skip to main content

Create HTTP sessions that maintain a unique user-agent, proxy, and handle session specific metadata

Project description

CustomSession

Overview

CustomSession is a Python package that provides a flexible and customizable HTTP client for making synchronous and asynchronous HTTP requests. This package is built on top of the popular HTTP client librarys, httpx and requests, and offers several features to simplify and enhance your HTTP requests.

Features

  1. Synchronous and Asynchronous Support: CustomSession provides both synchronous and asynchronous HTTP clients

  2. Random User-Agent: Randomly generated desktop user-agent for all requests sent by an HTTP client

  3. Proxy Support: CustomSession allows you to specify an HTTP/s proxy for all requests sent by an HTTP client

  4. Retry Mechanism: Configure the number of retries for each request to handle exceptions gracefully.

  5. Exception Handling: Customize the exceptions to be ignored during retries, allowing you to manage specific error conditions.

  6. Session Metadata: Store and manage miscellaneous session data as a dictionary, making it easier to maintain context between requests.

Installation

You can install CustomSession using pip:

$ pip install custom-session

Usage

Synchronous Client

Here's how you can use CustomSession synchronously:

from CustomSession import SyncSession

# Create a synchronous session
with SyncSession() as session:
    response = session.get('https://example.com')
    print(response.text)

Read more here

Asynchronous Client

To use CustomSession asynchronously, you use it within an asynchronous context:

import asyncio
from CustomSession import AsyncSession


async def main():
    async with AsyncSession() as session:
        response = await session.get('https://example.com')
        print(response.text)


# Run the asynchronous code
loop = asyncio.run(main())

Read more here

API Reference

SyncSession

Overview

SyncSession is an synchronous HTTP client class in Python, inherited from requests.Session from requests. It is designed to simplify making HTTP requests with features such as maintaining a unique user-agent, handling metadata, and optional proxy support.
This class is particularly useful for building synchronous web scraping or web automation scripts.

Usage

To use the SyncSession class, you should first import it and then create an instance of it. You can customize the behavior of the session by passing optional parameters during initialization.

from CustomSession import SyncSession

# Create an instance of SyncSession
session = SyncSession(proxy="1.1.1.1", ignore_exceptions=(Exception,))
Parameters
  • proxy (optional): Specifies an HTTP/s proxy to use when sending requests.
  • ignore_exceptions (optional): A tuple of exceptions to be ignored when sending requests. If an exception specified in this tuple occurs during a request, it will be caught, and the request will be retried.
  • user_agent (UserAgent, optional): A user agent for the session. Default is a randomly generated user agent for a desktop device.
  • meta_data (SessionMetaData | dict, optional): Additional metadata for the session. Default is an instance of SessionMetaData or an empty dictionary.
Methods

SyncSession provides several asynchronous HTTP request methods, including get, post, put, delete, patch, head, and options. These methods allow you to make various types of HTTP requests and handle retries automatically in case of specified exceptions.

  • get: Sends an HTTP GET request.

  • post: Sends an HTTP POST request.

  • put: Sends an HTTP PUT request.

  • delete: Sends an HTTP DELETE request.

  • patch: Sends an HTTP PATCH request.

  • head: Sends an HTTP HEAD request.

  • options: Sends an HTTP OPTIONS request.

Each of these methods accepts parameters for request configuration and allows you to specify the number of retries in case of exceptions.

Example

Here is an example of how to use the SyncSession class to send a GET request:

from CustomSession import SyncSession

# Create an instance of SyncSession
session = SyncSession()

# Send a GET request
response = session.get("https://example.com")

# Check the response status code
if response.status_code == 200:
    print("Request was successful")
else:
    print("Request failed")
Exception Handling

The SyncSession class handles exceptions during HTTP requests by retrying the request up to the specified number of times (retries parameter) if the exception matches any of the types listed in ignore_exceptions. If the maximum number of retries is reached, it raises a RetriesExceeded exception with an error message indicating the number of failed attempts.

Please make sure to handle RetriesExceeded exceptions appropriately in your code.

from CustomSession import SyncSession, RetriesExceeded

session = SyncSession()
try:
    response = session.get("https://example.com", retries=5)
    print("Request was successful")
except RetriesExceeded as e:
    print(f"Request failed after {e.retries_attempted} attempts")

AsyncSession

Overview

AsyncSession is an asynchronous HTTP client class in Python, inherited from httpx.AsyncClient from httpx. It is designed to simplify making HTTP requests with features such as maintaining a unique user-agent, handling metadata, and optional proxy support.
This class is particularly useful for building asynchronous web scraping or web automation scripts.

Usage

To use the AsyncSession class, you can create an instance of it using the async with statement, which allows you to manage the lifecycle of the client. Here's a basic example of how to use it:

import asyncio
from CustomSessions import AsyncSession


async def main():
    async with AsyncSession() as client:
        response = await client.get('https://example.org')
        print(response.text)


if __name__ == "__main__":
    asyncio.run(main())
Parameters

The AsyncSession class accepts several optional parameters:

  • proxy (optional): Specifies an HTTP/s proxy to use when sending requests.

  • ignore_exceptions (optional): A tuple of exceptions to be ignored when sending requests. If an exception specified in this tuple occurs during a request, it will be caught, and the request will be retried.

  • auth (optional): An authentication class to use when sending requests. This allows you to provide authentication credentials for requests, such as username and password.

  • meta_data (optional): A dictionary or an instance of SessionMetaData that can be used to store miscellaneous session data. This data persists across requests and can be useful for storing session-specific information.

Methods

AsyncSession provides several asynchronous HTTP request methods, including get, post, put, delete, patch, head, and options. These methods allow you to make various types of HTTP requests and handle retries automatically in case of specified exceptions.

  • get: Sends an HTTP GET request.

  • post: Sends an HTTP POST request.

  • put: Sends an HTTP PUT request.

  • delete: Sends an HTTP DELETE request.

  • patch: Sends an HTTP PATCH request.

  • head: Sends an HTTP HEAD request.

  • options: Sends an HTTP OPTIONS request.

Each of these methods accepts parameters for request configuration and allows you to specify the number of retries in case of exceptions.

Example

Here's an example of using the get method with retries:

async with AsyncSession() as client:
    response = await client.get('https://example.org', retries=3)
    if response.status_code == 200:
        print(response.text)
        else:
        print("Request failed")

In this example, the get method is used to make a GET request to https://example.org, and it will retry up to 3 times if certain exceptions specified in the ignore_exceptions parameter occur.

Exception Handling

The AsyncSession class handles exceptions during HTTP requests by retrying the request up to the specified number of times (retries parameter) if the exception matches any of the types listed in ignore_exceptions. If the maximum number of retries is reached, it raises a RetriesExceeded exception with an error message indicating the number of failed attempts.

Please make sure to handle RetriesExceeded exceptions appropriately in your code.

from CustomSession import AsyncSession, RetriesExceeded

session = AsyncSession()
try:
    async with AsyncSession() as client:
        response = await client.get("https://example.com", retries=5)
print("Request was successful")
except RetriesExceeded as e:
print(f"Request failed after {e.retries_attempted} attempts")

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

custom_session-1.0.5.tar.gz (10.1 kB view details)

Uploaded Source

Built Distribution

custom_session-1.0.5-py3-none-any.whl (10.7 kB view details)

Uploaded Python 3

File details

Details for the file custom_session-1.0.5.tar.gz.

File metadata

  • Download URL: custom_session-1.0.5.tar.gz
  • Upload date:
  • Size: 10.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.10.6

File hashes

Hashes for custom_session-1.0.5.tar.gz
Algorithm Hash digest
SHA256 779ae9f42476b68b5645d78c897e02cc7371fd12b623e771df546a9d8211f2b6
MD5 9a29634365b61006ccffad70b5481eb0
BLAKE2b-256 316ce01d1ff32b1d1f3b3fe6dbc3e4a92e05633bd5e1987f9e2ade4d317f28af

See more details on using hashes here.

File details

Details for the file custom_session-1.0.5-py3-none-any.whl.

File metadata

File hashes

Hashes for custom_session-1.0.5-py3-none-any.whl
Algorithm Hash digest
SHA256 62e48706a0853abfe934cf5ecbe01e9134ea8956532b491daa89614d7184804b
MD5 789f20be85c0e89371df8e68fc12d5fb
BLAKE2b-256 435e8ee04e003b067ab18923216c7c674ce3285e571acf792df9664b18bfd315

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page