Create HTTP sessions that maintain a unique user-agent, proxy, and handle session specific metadata
Project description
CustomSession
Overview
CustomSession is a Python package that provides a flexible and customizable HTTP client for making synchronous and asynchronous HTTP requests. This package is built on top of the popular HTTP client librarys, httpx and requests, and offers several features to simplify and enhance your HTTP requests.
Features
-
Synchronous and Asynchronous Support: CustomSession provides both synchronous and asynchronous HTTP clients
-
Random User-Agent: Randomly generated desktop user-agent for all requests sent by an HTTP client
-
Proxy Support: CustomSession allows you to specify an HTTP/s proxy for all requests sent by an HTTP client
-
Retry Mechanism: Configure the number of retries for each request to handle exceptions gracefully.
-
Exception Handling: Customize the exceptions to be ignored during retries, allowing you to manage specific error conditions.
-
Session Metadata: Store and manage miscellaneous session data as a dictionary, making it easier to maintain context between requests.
Installation
You can install CustomSession using pip:
$ pip install custom-session
Usage
Synchronous Client
Here's how you can use CustomSession synchronously:
from CustomSession import SyncSession
# Create a synchronous session
with SyncSession() as session:
response = session.get('https://example.com')
print(response.text)
Read more here
Asynchronous Client
To use CustomSession asynchronously, you use it within an asynchronous context:
import asyncio
from CustomSession import AsyncSession
async def main():
async with AsyncSession() as session:
response = await session.get('https://example.com')
print(response.text)
# Run the asynchronous code
loop = asyncio.run(main())
Read more here
API Reference
SyncSession
Overview
SyncSession
is an synchronous HTTP client class in Python, inherited from requests.Session
from requests. It is designed to simplify making HTTP requests with features such as
maintaining a unique user-agent, handling metadata, and optional proxy support.
This class is particularly useful for building synchronous web scraping or web automation scripts.
Usage
To use the SyncSession
class, you should first import it and then create an instance of it. You can customize the
behavior of the session by passing optional parameters during initialization.
from CustomSession import SyncSession
# Create an instance of SyncSession
session = SyncSession(proxy="1.1.1.1", ignore_exceptions=(Exception,))
Parameters
proxy
(optional): Specifies an HTTP/s proxy to use when sending requests.ignore_exceptions
(optional): A tuple of exceptions to be ignored when sending requests. If an exception specified in this tuple occurs during a request, it will be caught, and the request will be retried.user_agent
(UserAgent, optional): A user agent for the session. Default is a randomly generated user agent for a desktop device.meta_data
(SessionMetaData | dict, optional): Additional metadata for the session. Default is an instance ofSessionMetaData
or an empty dictionary.
Methods
SyncSession provides several asynchronous HTTP request methods,
including get
, post
, put
, delete
, patch
, head
, and options
. These methods allow you to make various types
of HTTP requests and handle retries automatically in case of specified exceptions.
-
get
: Sends an HTTP GET request. -
post
: Sends an HTTP POST request. -
put
: Sends an HTTP PUT request. -
delete
: Sends an HTTP DELETE request. -
patch
: Sends an HTTP PATCH request. -
head
: Sends an HTTP HEAD request. -
options
: Sends an HTTP OPTIONS request.
Each of these methods accepts parameters for request configuration and allows you to specify the number of retries in case of exceptions.
Example
Here is an example of how to use the SyncSession
class to send a GET request:
from CustomSession import SyncSession
# Create an instance of SyncSession
session = SyncSession()
# Send a GET request
response = session.get("https://example.com")
# Check the response status code
if response.status_code == 200:
print("Request was successful")
else:
print("Request failed")
Exception Handling
The SyncSession
class handles exceptions during HTTP requests by retrying the request up to the specified number of
times (retries
parameter) if the exception matches any of the types listed in ignore_exceptions
. If the maximum
number of retries is reached, it raises a RetriesExceeded
exception with an error message indicating the number of
failed attempts.
Please make sure to handle RetriesExceeded
exceptions appropriately in your code.
from CustomSession import SyncSession, RetriesExceeded
session = SyncSession()
try:
response = session.get("https://example.com", retries=5)
print("Request was successful")
except RetriesExceeded as e:
print(f"Request failed after {e.retries_attempted} attempts")
AsyncSession
Overview
AsyncSession
is an asynchronous HTTP client class in Python, inherited from httpx.AsyncClient
from httpx. It is designed to simplify making HTTP requests with features such as
maintaining a unique user-agent, handling metadata, and optional proxy support.
This class is particularly useful for building asynchronous web scraping or web automation scripts.
Usage
To use the AsyncSession
class, you can create an instance of it using the async with
statement, which allows you to
manage the lifecycle of the client. Here's a basic example of how to use it:
import asyncio
from CustomSessions import AsyncSession
async def main():
async with AsyncSession() as client:
response = await client.get('https://example.org')
print(response.text)
if __name__ == "__main__":
asyncio.run(main())
Parameters
The AsyncSession
class accepts several optional parameters:
-
proxy
(optional): Specifies an HTTP/s proxy to use when sending requests. -
ignore_exceptions
(optional): A tuple of exceptions to be ignored when sending requests. If an exception specified in this tuple occurs during a request, it will be caught, and the request will be retried. -
auth
(optional): An authentication class to use when sending requests. This allows you to provide authentication credentials for requests, such as username and password. -
meta_data
(optional): A dictionary or an instance ofSessionMetaData
that can be used to store miscellaneous session data. This data persists across requests and can be useful for storing session-specific information.
Methods
AsyncSession provides several asynchronous HTTP request methods,
including get
, post
, put
, delete
, patch
, head
, and options
. These methods allow you to make various types
of HTTP requests and handle retries automatically in case of specified exceptions.
-
get
: Sends an HTTP GET request. -
post
: Sends an HTTP POST request. -
put
: Sends an HTTP PUT request. -
delete
: Sends an HTTP DELETE request. -
patch
: Sends an HTTP PATCH request. -
head
: Sends an HTTP HEAD request. -
options
: Sends an HTTP OPTIONS request.
Each of these methods accepts parameters for request configuration and allows you to specify the number of retries in case of exceptions.
Example
Here's an example of using the get
method with retries:
async with AsyncSession() as client:
response = await client.get('https://example.org', retries=3)
if response.status_code == 200:
print(response.text)
else:
print("Request failed")
In this example, the get
method is used to make a GET request to https://example.org, and it
will retry up to 3 times if certain exceptions specified in the ignore_exceptions
parameter occur.
Exception Handling
The AsyncSession
class handles exceptions during HTTP requests by retrying the request up to the specified number of
times (retries
parameter) if the exception matches any of the types listed in ignore_exceptions
. If the maximum
number of retries is reached, it raises a RetriesExceeded
exception with an error message indicating the number of
failed attempts.
Please make sure to handle RetriesExceeded
exceptions appropriately in your code.
from CustomSession import AsyncSession, RetriesExceeded
session = AsyncSession()
try:
async with AsyncSession() as client:
response = await client.get("https://example.com", retries=5)
print("Request was successful")
except RetriesExceeded as e:
print(f"Request failed after {e.retries_attempted} attempts")
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file custom_session-1.0.5.tar.gz
.
File metadata
- Download URL: custom_session-1.0.5.tar.gz
- Upload date:
- Size: 10.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 779ae9f42476b68b5645d78c897e02cc7371fd12b623e771df546a9d8211f2b6 |
|
MD5 | 9a29634365b61006ccffad70b5481eb0 |
|
BLAKE2b-256 | 316ce01d1ff32b1d1f3b3fe6dbc3e4a92e05633bd5e1987f9e2ade4d317f28af |
File details
Details for the file custom_session-1.0.5-py3-none-any.whl
.
File metadata
- Download URL: custom_session-1.0.5-py3-none-any.whl
- Upload date:
- Size: 10.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.10.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 62e48706a0853abfe934cf5ecbe01e9134ea8956532b491daa89614d7184804b |
|
MD5 | 789f20be85c0e89371df8e68fc12d5fb |
|
BLAKE2b-256 | 435e8ee04e003b067ab18923216c7c674ce3285e571acf792df9664b18bfd315 |