aresilient 0.0.1a0

Python library focused on common HTTP request functionality

aresilient

Overview

aresilient is a Python library that provides resilient HTTP request functionality with automatic retry logic and exponential backoff. Built on top of the modern httpx library, it simplifies handling transient failures in HTTP communications, making your applications more robust and fault-tolerant.

Key Features

• Automatic Retry Logic: Automatically retries failed requests for configurable HTTP status codes (429, 500, 502, 503, 504 by default)
• Exponential Backoff with Optional Jitter: Implements exponential backoff strategy with optional randomized jitter to prevent thundering herd problems and avoid overwhelming servers
• Retry-After Header Support: Respects server-specified retry delays from Retry-After headers (supports both integer seconds and HTTP-date formats)
• Complete HTTP Method Support: Supports all common HTTP methods (GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS)
• Async Support: Fully supports asynchronous requests for high-performance applications
• Built on httpx: Leverages the modern, async-capable httpx library
• Configurable: Customize timeout, retry attempts, backoff factors, jitter, and retryable status codes
• Enhanced Error Handling: Comprehensive error handling with detailed exception information including HTTP status codes and response objects
• Type-Safe: Fully typed with comprehensive type hints
• Well-Tested: Extensive test coverage ensuring reliability

Installation

uv pip install aresilient

The following is the corresponding aresilient versions and supported dependencies.

aresilient	httpx	python
main	>=0.28,<1.0	>=3.10

Quick Start

Basic GET Request

from aresilient import get_with_automatic_retry

# Simple GET request with automatic retry
response = get_with_automatic_retry("https://api.example.com/data")
print(response.json())

Basic POST Request

from aresilient import post_with_automatic_retry

# POST request with JSON payload
response = post_with_automatic_retry(
    "https://api.example.com/submit", json={"key": "value"}
)
print(response.status_code)

Customizing Retry Behavior

from aresilient import get_with_automatic_retry

# Custom retry configuration
response = get_with_automatic_retry(
    "https://api.example.com/data",
    max_retries=5,  # Retry up to 5 times
    backoff_factor=1.0,  # Exponential backoff factor
    jitter_factor=0.1,  # Add 10% jitter to prevent thundering herd
    timeout=30.0,  # 30 second timeout
    status_forcelist=(429, 503),  # Only retry on these status codes
)

Using a Custom httpx Client

import httpx
from aresilient import get_with_automatic_retry

# Use your own httpx.Client for advanced configuration
with httpx.Client(headers={"Authorization": "Bearer token"}) as client:
    response = get_with_automatic_retry(
        "https://api.example.com/protected", client=client
    )

Other HTTP Methods

from aresilient import (
    put_with_automatic_retry,
    delete_with_automatic_retry,
    patch_with_automatic_retry,
    head_with_automatic_retry,
    options_with_automatic_retry,
)

# PUT request to update a resource
response = put_with_automatic_retry(
    "https://api.example.com/resource/123", json={"name": "updated"}
)

# DELETE request to remove a resource
response = delete_with_automatic_retry("https://api.example.com/resource/123")

# PATCH request to partially update a resource
response = patch_with_automatic_retry(
    "https://api.example.com/resource/123", json={"status": "active"}
)

# HEAD request to check resource existence and get metadata
response = head_with_automatic_retry("https://api.example.com/large-file.zip")
if response.status_code == 200:
    print(f"File size: {response.headers.get('Content-Length')} bytes")

# OPTIONS request to discover allowed methods
response = options_with_automatic_retry("https://api.example.com/resource")
print(f"Allowed methods: {response.headers.get('Allow')}")

Error Handling

from aresilient import get_with_automatic_retry, HttpRequestError

try:
    response = get_with_automatic_retry("https://api.example.com/data")
except HttpRequestError as e:
    print(f"Request failed: {e}")
    print(f"Method: {e.method}")
    print(f"URL: {e.url}")
    print(f"Status Code: {e.status_code}")

Using Async

All HTTP methods have async versions for concurrent request processing:

import asyncio
from aresilient import get_with_automatic_retry_async


async def fetch_data():
    response = await get_with_automatic_retry_async("https://api.example.com/data")
    return response.json()


# Run the async function
data = asyncio.run(fetch_data())
print(data)

Concurrent Async Requests

Process multiple requests concurrently for better performance:

import asyncio
from aresilient import get_with_automatic_retry_async


async def fetch_multiple():
    urls = [
        "https://api.example.com/data1",
        "https://api.example.com/data2",
        "https://api.example.com/data3",
    ]
    tasks = [get_with_automatic_retry_async(url) for url in urls]
    responses = await asyncio.gather(*tasks)
    return [r.json() for r in responses]


# Fetch multiple URLs concurrently
results = asyncio.run(fetch_multiple())

Configuration

Default Settings

• Timeout: 10.0 seconds
• Max Retries: 3 (4 total attempts including the initial request)
• Backoff Factor: 0.3
• Retryable Status Codes: 429 (Too Many Requests), 500 (Internal Server Error), 502 (Bad Gateway), 503 (Service Unavailable), 504 (Gateway Timeout)

Exponential Backoff Formula

The wait time between retries is calculated as:

base_wait_time = backoff_factor * (2 ** retry_number)
# If jitter_factor is set (e.g., 0.1 for 10% jitter):
jitter = random(0, jitter_factor) * base_wait_time
total_wait_time = base_wait_time + jitter

For example, with backoff_factor=0.3 and jitter_factor=0.1:

• 1st retry: 0.3-0.33 seconds (base 0.3s + up to 10% jitter)
• 2nd retry: 0.6-0.66 seconds (base 0.6s + up to 10% jitter)
• 3rd retry: 1.2-1.32 seconds (base 1.2s + up to 10% jitter)

Note: Jitter is optional (disabled by default with jitter_factor=0). When enabled, it's randomized for each retry to prevent multiple clients from retrying simultaneously (thundering herd problem). Set jitter_factor=0.1 for 10% jitter, which is recommended for production use.

Retry-After Header Support

When a server returns a Retry-After header (commonly with 429 or 503 status codes), the library automatically uses the server's suggested wait time instead of exponential backoff. This ensures compliance with rate limiting and helps avoid overwhelming the server.

The Retry-After header supports two formats:

• Integer seconds: Retry-After: 120 (wait 120 seconds)
• HTTP-date: Retry-After: Wed, 21 Oct 2015 07:28:00 GMT (wait until this time)

Note: If jitter_factor is configured, jitter is still applied to server-specified Retry-After values to prevent thundering herd issues when many clients receive the same retry delay from a server.

API Reference

get_with_automatic_retry()

Performs an HTTP GET request with automatic retry logic.

Parameters:

• url (str): The URL to send the request to
• client (httpx.Client | None): Optional httpx client to use
• timeout (float | httpx.Timeout): Request timeout in seconds
• max_retries (int): Maximum number of retry attempts
• backoff_factor (float): Exponential backoff factor
• status_forcelist (tuple[int, ...]): HTTP status codes that trigger a retry
• **kwargs: Additional arguments passed to httpx.Client.get()

Returns: httpx.Response

Raises:

• HttpRequestError: If the request fails after all retries
• ValueError: If parameters are invalid

post_with_automatic_retry()

Performs an HTTP POST request with automatic retry logic.

Parameters:

• url (str): The URL to send the request to
• client (httpx.Client | None): Optional httpx client to use
• timeout (float | httpx.Timeout): Request timeout in seconds
• max_retries (int): Maximum number of retry attempts
• backoff_factor (float): Exponential backoff factor
• status_forcelist (tuple[int, ...]): HTTP status codes that trigger a retry
• **kwargs: Additional arguments passed to httpx.Client.post()

Returns: httpx.Response

Raises:

• HttpRequestError: If the request fails after all retries
• ValueError: If parameters are invalid

put_with_automatic_retry()

Performs an HTTP PUT request with automatic retry logic.

Parameters:

• url (str): The URL to send the request to
• client (httpx.Client | None): Optional httpx client to use
• timeout (float | httpx.Timeout): Request timeout in seconds
• max_retries (int): Maximum number of retry attempts
• backoff_factor (float): Exponential backoff factor
• status_forcelist (tuple[int, ...]): HTTP status codes that trigger a retry
• **kwargs: Additional arguments passed to httpx.Client.put()

Returns: httpx.Response

Raises:

• HttpRequestError: If the request fails after all retries
• ValueError: If parameters are invalid

delete_with_automatic_retry()

Performs an HTTP DELETE request with automatic retry logic.

Parameters:

• url (str): The URL to send the request to
• client (httpx.Client | None): Optional httpx client to use
• timeout (float | httpx.Timeout): Request timeout in seconds
• max_retries (int): Maximum number of retry attempts
• backoff_factor (float): Exponential backoff factor
• status_forcelist (tuple[int, ...]): HTTP status codes that trigger a retry
• **kwargs: Additional arguments passed to httpx.Client.delete()

Returns: httpx.Response

Raises:

• HttpRequestError: If the request fails after all retries
• ValueError: If parameters are invalid

patch_with_automatic_retry()

Performs an HTTP PATCH request with automatic retry logic.

Parameters:

• url (str): The URL to send the request to
• client (httpx.Client | None): Optional httpx client to use
• timeout (float | httpx.Timeout): Request timeout in seconds
• max_retries (int): Maximum number of retry attempts
• backoff_factor (float): Exponential backoff factor
• status_forcelist (tuple[int, ...]): HTTP status codes that trigger a retry
• **kwargs: Additional arguments passed to httpx.Client.patch()

Returns: httpx.Response

Raises:

• HttpRequestError: If the request fails after all retries
• ValueError: If parameters are invalid

head_with_automatic_retry()

Performs an HTTP HEAD request with automatic retry logic.

Parameters:

• url (str): The URL to send the request to
• client (httpx.Client | None): Optional httpx client to use
• timeout (float | httpx.Timeout): Request timeout in seconds
• max_retries (int): Maximum number of retry attempts
• backoff_factor (float): Exponential backoff factor
• status_forcelist (tuple[int, ...]): HTTP status codes that trigger a retry
• **kwargs: Additional arguments passed to httpx.Client.head()

Returns: httpx.Response

Raises:

• HttpRequestError: If the request fails after all retries
• ValueError: If parameters are invalid

Use cases: HEAD requests retrieve only headers without the response body, making them useful for checking resource existence, metadata (Content-Length, Last-Modified, ETag), and performing lightweight validation.

options_with_automatic_retry()

Performs an HTTP OPTIONS request with automatic retry logic.

Parameters:

• url (str): The URL to send the request to
• client (httpx.Client | None): Optional httpx client to use
• timeout (float | httpx.Timeout): Request timeout in seconds
• max_retries (int): Maximum number of retry attempts
• backoff_factor (float): Exponential backoff factor
• status_forcelist (tuple[int, ...]): HTTP status codes that trigger a retry
• **kwargs: Additional arguments passed to httpx.Client.options()

Returns: httpx.Response

Raises:

• HttpRequestError: If the request fails after all retries
• ValueError: If parameters are invalid

Use cases: OPTIONS requests are used for CORS preflight requests, discovering allowed HTTP methods via the Allow header, and querying server capabilities.

Async Versions

All synchronous functions have async counterparts with identical parameters:

• get_with_automatic_retry_async() - Async version of GET
• post_with_automatic_retry_async() - Async version of POST
• put_with_automatic_retry_async() - Async version of PUT
• delete_with_automatic_retry_async() - Async version of DELETE
• patch_with_automatic_retry_async() - Async version of PATCH
• head_with_automatic_retry_async() - Async version of HEAD
• options_with_automatic_retry_async() - Async version of OPTIONS

These functions work exactly like their synchronous counterparts but must be awaited and use httpx.AsyncClient instead of httpx.Client.

Low-Level Functions

For custom HTTP methods or advanced use cases:

• request_with_automatic_retry() - Generic synchronous request with retry logic
• request_with_automatic_retry_async() - Generic async request with retry logic

These functions allow you to specify any HTTP method (e.g., HEAD, OPTIONS) and provide your own request function from an httpx client.

HttpRequestError

Exception raised when an HTTP request fails.

Attributes:

• method (str): HTTP method used
• url (str): URL that was requested
• status_code (int | None): HTTP status code (if available)
• response (httpx.Response | None): Full response object (if available)

Contributing

Please check the instructions in CONTRIBUTING.md.

API stability

:warning: While aresilient is in development stage, no API is guaranteed to be stable from one release to the next. In fact, it is very likely that the API will change multiple times before a stable 1.0.0 release. In practice, this means that upgrading aresilient to a new version will possibly break any code that was using the old version of aresilient.

License

aresilient is licensed under BSD 3-Clause "New" or "Revised" license available in LICENSE file.