httpr 0.4.6

Fast HTTP client for Python

httpr

Blazing fast http-client for Python in Rust 🦀 that can be used as drop-in replacement for httpx and requests in most cases.

• Fast: httpr is built on top of reqwests, which is a blazing fast http client in Rust. Check out the benchmark.
• Both async and sync: httpr provides both a sync and async client.
• Lightweight: httpr is a lightweight http client with zero python-dependencies.
• Async: first-class async support.
• Streaming: supports streaming responses for efficient memory usage with large payloads.
• http2: httpr supports HTTP/2.
• mTLS: httpr supports mTLS.

Not implemented yet

• Fine-grained error handling: Fine-grained error handling is not implemented yet.

Documentation

📖 Full documentation: thomasht86.github.io/httpr

🤖 LLM-friendly docs: llms.txt | llms-full.txt

Table of Contents

• httpr
  • Not implemented yet
  • Documentation
  • Table of Contents
  • Installation
    • Install with uv
    • Install from PyPI
  • Benchmark
  • Usage
    • I. Client
      • Client methods
      • Response object
      • Streaming responses
      • Examples
    • II. AsyncClient
  • Precompiled wheels
  • CI
  • Acknowledgements

Installation

Install with uv

uv add httpr

or

uv pip install httpr

Install from PyPI

pip install -U httpr

Benchmark

Usage

I. Client

class Client:
    """Initializes an HTTP client.

    Args:
        auth (tuple[str, str| None] | None): Username and password for basic authentication. Default is None.
        auth_bearer (str | None): Bearer token for authentication. Default is None.
        params (dict[str, str] | None): Default query parameters to include in all requests. Default is None.
        headers (dict[str, str] | None): Default headers to send with requests. 
        cookies (dict[str, str] | None): - Map of cookies to send with requests as the `Cookie` header.
        timeout (float | None): HTTP request timeout in seconds. Default is 30.
        cookie_store (bool | None): Enable a persistent cookie store. Received cookies will be preserved and included
            in additional requests. Default is True.
        referer (bool | None): Enable or disable automatic setting of the `Referer` header. Default is True.
        proxy (str | None): Proxy URL for HTTP requests. Example: "socks5://127.0.0.1:9150". Default is None.
        follow_redirects (bool | None): Whether to follow redirects. Default is True.
        max_redirects (int | None): Maximum redirects to follow. Default 20. Applies if `follow_redirects` is True.
        verify (bool | None): Verify SSL certificates. Default is True.
        ca_cert_file (str | None): Path to CA certificate store. Default is None.
        https_only` (bool | None): Restrict the Client to be used with HTTPS only requests. Default is `false`.
        http2_only` (bool | None): If true - use only HTTP/2; if false - use only HTTP/1. Default is `false`.

    """

Client methods

The Client class provides a set of methods for making HTTP requests: get, head, options, delete, post, put, patch, each of which internally utilizes the request() method for execution. The parameters for these methods closely resemble those in httpx.

def get(
    url: str,
    params: dict[str, str] | None = None,
    headers: dict[str, str] | None = None,
    cookies: dict[str, str] | None = None,
    auth: tuple[str, str| None] | None = None,
    auth_bearer: str | None = None,
    timeout: float | None = 30,
):
    """Performs a GET request to the specified URL.

    Args:
        url (str): The URL to which the request will be made.
        params (dict[str, str] | None): A map of query parameters to append to the URL. Default is None.
        headers (dict[str, str] | None): A map of HTTP headers to send with the request. Default is None.
        cookies (dict[str, str] | None): - An optional map of cookies to send with requests as the `Cookie` header.
        auth (tuple[str, str| None] | None): A tuple containing the username and an optional password
            for basic authentication. Default is None.
        auth_bearer (str | None): A string representing the bearer token for bearer token authentication. Default is None.
        timeout (float | None): The timeout for the request in seconds. Default is 30.

    """

def post(
    url: str,
    params: dict[str, str] | None = None,
    headers: dict[str, str] | None = None,
    cookies: dict[str, str] | None = None,
    content: bytes | None = None,
    data: dict[str, Any] | None = None,
    json: Any | None = None,
    files: dict[str, str] | None = None,
    auth: tuple[str, str| None] | None = None,
    auth_bearer: str | None = None,
    timeout: float | None = 30,
):
    """Performs a POST request to the specified URL.

    Args:
        url (str): The URL to which the request will be made.
        params (dict[str, str] | None): A map of query parameters to append to the URL. Default is None.
        headers (dict[str, str] | None): A map of HTTP headers to send with the request. Default is None.
        cookies (dict[str, str] | None): - An optional map of cookies to send with requests as the `Cookie` header.
        content (bytes | None): The content to send in the request body as bytes. Default is None.
        data (dict[str, Any] | None): The form data to send in the request body. Default is None.
        json (Any | None): A JSON serializable object to send in the request body. Default is None.
        files (dict[str, str] | None): A map of file fields to file paths to be sent as multipart/form-data. Default is None.
        auth (tuple[str, str| None] | None): A tuple containing the username and an optional password
            for basic authentication. Default is None.
        auth_bearer (str | None): A string representing the bearer token for bearer token authentication. Default is None.
        timeout (float | None): The timeout for the request in seconds. Default is 30.

    """

Response object

The Client class returns a Response object that contains the following attributes and methods:

resp.content
resp.cookies
resp.encoding
resp.headers
resp.json()
resp.status_code
resp.text
resp.text_markdown  # html is converted to markdown text using html2text-rs
resp.text_plain  # html is converted to plain text
resp.text_rich  # html is converted to rich text
resp.url

Streaming responses

The Client class supports streaming responses for efficient memory usage when handling large payloads. Use the stream() context manager to iterate over response data without buffering the entire response in memory.

# Stream bytes chunks
with client.stream("GET", "https://example.com/large-file") as response:
    print(f"Status: {response.status_code}")
    for chunk in response.iter_bytes():
        process(chunk)

# Stream text chunks
with client.stream("GET", "https://example.com/text") as response:
    for text in response.iter_text():
        print(text, end="")

# Stream line by line (useful for Server-Sent Events)
with client.stream("GET", "https://example.com/events") as response:
    for line in response.iter_lines():
        print(line.strip())

# Read entire response (if needed after checking headers)
with client.stream("GET", url) as response:
    if response.status_code == 200:
        content = response.read()

StreamingResponse attributes:

• status_code - HTTP status code
• headers - Response headers (case-insensitive)
• cookies - Response cookies
• url - Final URL after redirects
• is_closed - Whether the stream has been closed
• is_consumed - Whether the stream has been fully consumed

StreamingResponse methods:

• iter_bytes() - Iterate over response as bytes chunks
• iter_text() - Iterate over response as text chunks (decoded using response encoding)
• iter_lines() - Iterate over response line by line
• read() - Read entire remaining response body into memory
• close() - Close the stream and release resources

Important notes:

• Streaming must be used as a context manager (with statement)
• Headers, cookies, and status code are available immediately before reading the body
• The response body is only read when you iterate over it or call read()
• Once consumed, the stream cannot be read again
• Streaming is supported for all HTTP methods (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS)

Examples

import httpr

# Initialize the client
client = httpr.Client() 

# GET request
resp = client.get("https://tls.peet.ws/api/all")
print(resp.json())

# GET request with passing params and setting timeout
params = {"param1": "value1", "param2": "value2"}
resp = client.post(url="https://httpbin.org/anything", params=params, timeout=10)
print(r.text)

# POST Binary Request Data
content = b"some_data"
resp = client.post(url="https://httpbin.org/anything", content=content)
print(r.text)

# POST Form Encoded Data
data = {"key1": "value1", "key2": "value2"}
resp = client.post(url="https://httpbin.org/anything", data=data)
print(r.text)

# POST JSON Encoded Data
json = {"key1": "value1", "key2": "value2"}
resp = client.post(url="https://httpbin.org/anything", json=json)
print(r.text)

# POST Multipart-Encoded Files
files = {'file1': '/home/root/file1.txt', 'file2': 'home/root/file2.txt'}
r = client.post("https://httpbin.org/post", files=files)
print(r.text)

# Authentication using user/password
auth = ("user", "password")
resp = client.post(url="https://httpbin.org/anything", auth=auth)
print(r.text)

# Authentication using auth bearer
auth_bearer = "bearerXXXXXXXXXXXXXXXXXXXX"
resp = client.post(url="https://httpbin.org/anything", auth_bearer=auth_bearer)
print(r.text)

# Using proxy or env var HTTPR_PROXY
resp = httpr.Client(proxy="http://127.0.0.1:8080").get("https://tls.peet.ws/api/all")
print(resp.json())
export HTTPR_PROXY="socks5://127.0.0.1:1080"
resp = httpr.Client().get("https://tls.peet.ws/api/all")
print(resp.json())

# Using custom CA certificate store: env var HTTPR_CA_BUNDLE
resp = httpr.Client(ca_cert_file="/cert/cacert.pem").get("https://tls.peet.ws/api/all")
print(resp.json())
resp = httpr.Client(ca_cert_file=certifi.where()).get("https://tls.peet.ws/api/all")
print(resp.json())
export HTTPR_CA_BUNDLE="/home/user/Downloads/cert.pem"
resp = httpr.Client().get("https://tls.peet.ws/api/all")
print(resp.json())

# You can also use convenience functions that use a default Client instance under the hood:
# httpr.get() | httpr.head() | httpr.options() | httpr.delete() | httpr.post() | httpr.patch() | httpr.put()
resp = httpr.get("https://httpbin.org/anything")
print(r.text)

II. AsyncClient

httpr.AsyncClient() is an asynchronous wrapper around the httpr.Client class, offering the same functions, behavior, and input arguments.

import asyncio
import logging

import httpr

async def aget_text(url):
    async with httpr.AsyncClient() as client:
        resp = await client.get(url)
        return resp.text

async def main():
    urls = ["https://nytimes.com/", "https://cnn.com/", "https://abcnews.go.com/"]
    tasks = [aget_text(u) for u in urls]
    results = await asyncio.gather(*tasks)

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    asyncio.run(main())

Streaming with AsyncClient:

The AsyncClient also supports streaming responses with the same API:

async with httpr.AsyncClient() as client:
    async with client.stream("GET", "https://example.com/large-file") as response:
        for chunk in response.iter_bytes():
            process(chunk)

Note: While the context manager is async, the iteration over chunks (iter_bytes(), iter_text(), iter_lines()) is synchronous.

Precompiled wheels

Provides precompiled wheels for the following platforms:

• 🐧 linux: amd64, aarch64, armv7 (aarch64 and armv7 builds are manylinux_2_34 compatible. ubuntu>=22.04, debian>=12)
• 🐧 musllinux: amd64, aarch64
• 🪟 windows: amd64
• 🍏 macos: amd64, aarch64.

Development

This project uses Taskfile for development workflows.

Setup

# Install dependencies
uv sync --extra dev

# Build Rust extension (required after any .rs changes)
uv run maturin develop

# Add hosts entry for e2e tests (one-time setup)
echo '127.0.0.1 httpbun.local' | sudo tee -a /etc/hosts

Running Tests

# List all available tasks
task --list

# Run unit tests only
task test:unit

# Run e2e tests (full workflow: start httpbun → test → stop)
task e2e

# Run e2e tests iteratively (keep httpbun running)
task e2e:local
task test:e2e  # run tests against running container

# Run all tests
task test

Other Tasks

task dev           # Build Rust extension
task check         # Run all checks (lint + test) - use before committing
task lint          # Run Python linters (ruff + mypy)
task lint:rust     # Run Rust linters (fmt + clippy)
task lint:all      # Run all linters (Python + Rust)
task fmt           # Format Python code with ruff
task fmt:rust      # Format Rust code
task fmt:all       # Format all code (Python + Rust)
task certs         # Generate SSL certificates for e2e tests
task httpbun:start # Start httpbun container
task httpbun:stop  # Stop httpbun container
task httpbun:logs  # Show container logs

Test Structure

• tests/unit/ - Unit tests using pytest-httpbin (fast, no Docker required)
• tests/e2e/ - E2E tests using httpbun Docker container with SSL

CI

Job	PRs	Push to main	Tags (Release)	Manual
lint	✓
test (Python 3.10-3.14)	✓	✓	✓
docs (build)	✓
linux, musllinux, windows, macos, sdist			✓	✓
release (PyPI publish)			✓	✓
benchmark			✓	✓

• PRs: Run lint, tests across Python 3.10-3.14 matrix, and verify docs build
• Push to main: Run tests only
• Tags: Run tests, build wheels, publish stable release to PyPI, run benchmarks
• Manual: Full multi-platform wheel builds with release and benchmarks

Acknowledgements

• uv: The package manager used, and for leading the way in the "Rust for python tools"-sphere.
• primp: A lot of code is borrowed from primp, that wraps rust library rquest for python in a similar way. If primp supported mTLS, I would have used it instead.
• reqwests: The rust library that powers httpr.
• pyo3
• maturin