abatcher 0.2.0

Parallel HTTP request batcher with rate limiting, retries, connection pooling, and more.

abatcher

Simple parallel HTTP request batcher with rate limiting, retries, connection pooling, and more. The entire API is only 1 function.

🛠️ Usage

Using abatcher should be as simple as:

import abatcher

requests = [
    # Simple URL GET request
    "https://httpbin.org/anything",
    # Custom request
    {
        "url": "https://httpbin.org/post",
        "method": "POST",
        "params": {"name": "Test"},
        "headers": {"X-Custom": "value"},
    },
]

results = abatcher.run(requests)

print(f"Batch requests results: {results}")

If you need more control, you can also send a custom client.

import httpx
import abatcher

custom_client = httpx.AsyncClient(
    auth=("user", "pass"),
    timeout=httpx.Timeout(45.0),
    limits=httpx.Limits(max_connections=100, max_keepalive_connections=20),
)

results = abatcher.run(
    requests,
    client=custom_client,
    max_concurrent=20,
    max_per_second=10,
    cache=True,
    cache_dir="custom_cache",
    cache_ttl=3600,
)

print(f"Batch requests results: {results}")