rex-tls

rex-tls is a Python HTTP client backed by a native Rust TLS and HTTP core. It provides a requests-style API with transport profiles for Android Chrome and OkHttp, persistent sessions, cookies, proxies, streaming I/O, asynchronous requests, and bounded connection pools.

What's new in 2.3.1

• Every response now exposes its own cookies container. It contains only the cookies received in that response; the Session cookie jar remains the cumulative state used by later requests.
• Response Domain cookies use the same native Public Suffix List guard as the Session jar, so invalid public-suffix or cross-domain cookies are excluded.
• The Response object now covers the public Requests response surface, including elapsed, request, encoding, apparent_encoding, links, redirect helpers, next, iteration, context management, and response cookies.
• Response encoding can be overridden by assigning response.encoding, matching the common Requests workflow.
• Prepared request metadata is attached to network responses, so callers can inspect the method, URL, headers, body, and path_url that produced a response.
• Documentation now separates practical conclusions from benchmark details and gives response and cookie behavior concrete examples.

These Python API changes do not alter the TLS, HTTP/1.1, HTTP/2, or HTTP/3 wire profiles.

Installation

python -m pip install rex-tls

Python 3.9 or newer is required. Supported platforms are:

• Windows x86-64
• Linux x86-64 with manylinux 2.28 or newer

Transport profiles

Profile aliases:

• chrome_android selects chrome_android_150.
• chrome_android_latest selects chrome_android_150.
• okhttp selects okhttp_5.4.
• okhttp_latest selects okhttp_5.4.

Use the full versioned profile name when reproducibility matters.

Quick start

import rex_tls

response = rex_tls.get(
    "https://example.com/",
    profile="chrome_android_150",
    params={"page": 1},
    timeout=20,
)

response.raise_for_status()
print(response.status_code)
print(response.http_version)
print(response.text)

JSON and form requests use familiar keyword arguments:

response = rex_tls.post(
    "https://api.example.com/items",
    profile="okhttp_5.4",
    json={"name": "example", "enabled": True},
)

response = rex_tls.post(
    "https://api.example.com/form",
    profile="okhttp_4.12",
    data={"name": "example"},
)

Persistent sessions

A Session keeps cookies, connections, and TLS session state across requests.

from rex_tls import Session

with Session(
    profile="chrome_android_150",
    timeout=20,
) as session:
    session.headers.update({"accept": "application/json"})
    session.cookies.set("locale", "en-US")

    first = session.get("https://example.com/api/profile")
    second = session.get("https://example.com/api/settings")

    print(second.connection_reused)
    print(session.cookies.get_dict())

HTTP/1.1 concurrency can be bounded per origin and proxy route:

with Session(
    profile="okhttp_5.4",
    max_connections_per_route=4,
) as session:
    ...

The default is 1. HTTP/2 continues to multiplex streams on one connection; additional connections are opened only after the route has negotiated HTTP/1.1 and every existing route connection is busy.

Per-request cookies are supported and do not modify the session jar:

response = session.get(
    "https://example.com/api/items",
    params={"page": 2},
    cookies={"request-only": "value"},
    headers={"accept": "application/json"},
)

Cookies

Session.cookies provides the commonly used Requests-style CookieJar methods:

session.cookies.update({"theme": "dark"})
session.cookies.set(
    "api-token",
    "value",
    domain="api.example.com",
    path="/v1",
    secure=True,
)

print(session.cookies.get("theme"))
print(session.cookies.get_dict())
print(session.cookies.items())

session.cookies.clear(domain="api.example.com", path="/v1", name="api-token")

Available CookieJar methods:

• set(name, value, ...): create or replace a cookie.
• set_cookie(cookie): add a Cookie-compatible object.
• get(name, ...): read one cookie.
• get_dict(...): return matching cookies as a dictionary.
• update(values): merge a mapping or another cookie jar.
• clear(...): remove one cookie, one scope, or the complete jar.
• keys(), values(), and items(): inspect stored cookies.
• list_domains() and list_paths(): inspect cookie scopes.

Cookies received from a response are checked before storage. The checks cover domain, path, Secure, expiry, IP-address, and public-suffix rules.

Response and Session cookie containers have different scopes:

with Session("okhttp_5.4") as session:
    response = session.get("https://example.com/login")

    # Cookies set by this response only
    print(response.cookies.get_dict())

    # All cookies currently retained by the session
    print(session.cookies.get_dict())

Both containers support the same common methods shown above. Mutating response.cookies does not change session.cookies. This matches the usual Requests distinction between response cookies and the persistent Session jar.

Proxies

Use the proxy argument as a single proxy for all supported routes:

proxy = "http://username:password@proxy.example:8080"

with Session("okhttp_5.4", proxy=proxy) as session:
    response = session.get("https://example.com/")

Or use a requests-style mapping:

proxies = {
    "http": "http://proxy.example:8080",
    "https": "http://proxy.example:8080",
    "all": "http://fallback.example:8080",
    "no_proxy": ".internal.example,localhost,127.0.0.1",
}

with Session("chrome_android_149", proxies=proxies) as session:
    response = session.get("https://example.com/")

Session.proxies is mutable. A per-request proxies mapping overrides the session mapping. Set trust_env=True to read HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, and NO_PROXY.

Proxy URLs must use the http:// scheme. HTTPS destinations use HTTP CONNECT. Basic proxy credentials are supported; percent-encode reserved characters in usernames and passwords. HTTPS proxies, SOCKS, PAC, NTLM/Digest, and MASQUE are not supported.

HTTP version selection

The negotiated protocol is available as response.http_version.

Require an actual HTTP/2 connection with http2=True:

with Session("chrome_android_150", http2=True) as session:
    response = session.get("https://example.com/")
    assert response.http_version == "HTTP/2"

http2=True preserves the profile ALPN list and rejects the response if the server does not negotiate HTTP/2. It supports HTTPS only and is mutually exclusive with HTTP/3 modes.

Chrome profiles support the following http3 values:

with Session("chrome_android_150", http3="auto") as session:
    response = session.get("https://example.com/")

with Session("chrome_android_149", http3="only") as session:
    response = session.get("https://example.com/")
    assert response.http_version == "HTTP/3"

HTTP/3 requires HTTPS and a versioned Chrome profile. OkHttp profiles do not enable HTTP/3. Conventional HTTP proxies cannot carry the QUIC path, so http3="only" is rejected when such a proxy is selected.

Request headers

Headers may be supplied as a mapping or as ordered name/value pairs. The native core applies the selected profile's protocol-specific ordering and casing rules before transmission.

If both session and request headers are empty, the profile's default request headers are used. A non-empty caller header set is sent without adding unrelated profile defaults. Protocol-required fields and fields derived from cookies or the request body may still be generated.

HTTP/1.1 preserves the profile's observable field-name casing. HTTP/2 and HTTP/3 field names are lowercase as required by those protocols. Hop-by-hop HTTP/1.1 fields such as Connection are rejected for HTTP/2 and HTTP/3 instead of being silently removed.

Streaming downloads

Use a context manager so the connection is released when the body reaches EOF or the response is closed:

from rex_tls import Session

with Session("chrome_android_150") as session:
    with session.get("https://example.com/large.bin", stream=True) as response:
        response.raise_for_status()
        with open("large.bin", "wb") as output:
            for chunk in response.iter_content(64 * 1024):
                output.write(chunk)

iter_lines(), raw.read(), and raw.readinto() are also available. Set decode_content=False to receive the compressed response body without automatic content decoding.

Streaming uploads and multipart files

File objects and byte iterables are uploaded incrementally:

with Session("okhttp_5.4") as session:
    with open("large.bin", "rb") as source:
        response = session.post(
            "https://example.com/upload",
            content=source,
        )

    with open("image.png", "rb") as source:
        response = session.post(
            "https://example.com/form",
            data={"title": "example"},
            files={"file": ("image.png", source, "image/png")},
        )

Seekable upload sources can be replayed across 307/308 redirects. A one-shot source raises UnrewindableBodyError if a redirect requires replay.

Async API

AsyncSession provides an asyncio interface without blocking the event loop:

import asyncio
from rex_tls import AsyncSession

async def main() -> None:
    async with AsyncSession(
        "okhttp_5.4",
        max_concurrency=8,
        http2=True,
    ) as session:
        urls = [f"https://example.com/items/{item}" for item in range(10)]
        responses = await asyncio.gather(*(session.get(url) for url in urls))
        print([response.status_code for response in responses])

asyncio.run(main())

Async streaming uses these methods:

• aiter_content() yields body chunks.
• aiter_lines() yields decoded or byte lines.

from rex_tls import AsyncSession

async def stream_events(session: AsyncSession) -> None:
    response = await session.get("https://example.com/events", stream=True)
    async with response:
        async for line in response.aiter_lines():
            print(line)

Bounded session pools

SessionPool and AsyncSessionPool create multiple independent native sessions with a fixed concurrency limit.

from concurrent.futures import ThreadPoolExecutor
from rex_tls import SessionPool

urls = [f"https://example.com/items/{item}" for item in range(20)]

with SessionPool(
    "okhttp_4.12",
    max_connections=8,
    session_mode="shared",
) as pool:
    pool.headers["accept"] = "application/json"
    with ThreadPoolExecutor(max_workers=16) as executor:
        responses = list(executor.map(pool.get, urls))

Pool session modes:

• shared: members share headers, proxy configuration, and one thread-safe CookieJar. Connections remain independent.
• isolated: each member owns independent headers, proxy configuration, cookies, connections, and TLS state.

Lease a specific member when multiple requests must keep the same isolated state:

from rex_tls import SessionPool

with SessionPool(
    "okhttp_5.4",
    max_connections=4,
    session_mode="isolated",
) as pool:
    with pool.acquire(timeout=1) as account:
        account.headers["authorization"] = "Bearer example-token"
        account.cookies.set("account", "one")
        account.get("https://example.com/step-1")
        account.get("https://example.com/step-2")

Use pool_timeout to limit how long a request waits for a pool member. The ordinary timeout argument continues to control the network request.

Response API

Response follows the public Requests response interface and adds transport details specific to rex-tls.

Core data:

• status_code — numeric HTTP status.
• reason — HTTP reason phrase.
• url — final response URL.
• headers — case-insensitive response headers; get_all() returns duplicates and raw preserves field order.
• content — response body as bytes.
• text — decoded response text.
• encoding — selected text encoding; callers may assign a different encoding.
• apparent_encoding — detected fallback encoding.
• cookies — cookies received in this response only.
• elapsed — request duration as datetime.timedelta.
• history — redirect responses from oldest to newest.
• request — prepared request metadata for this response.
• connection — the Session transport owner attached for compatibility.
• raw — file-like body reader when stream=True.

Requests-compatible helpers:

• json() parses a JSON response.
• ok and boolean conversion report whether raise_for_status() would succeed.
• raise_for_status() raises HTTPError for 4xx and 5xx responses.
• is_redirect and is_permanent_redirect describe redirect responses.
• next contains the prepared follow-up request when redirects are disabled.
• links parses the Link response header.
• iter_content() and iter_lines() stream or iterate over cached content.
• Using close() or a context manager releases response resources.

Additional transport details:

• http_version — HTTP/1.1, HTTP/2, or HTTP/3.
• elapsed_seconds — request duration as a floating-point number of seconds.
• local_address and remote_address — transport endpoints when available.
• connection_reused — whether an existing connection carried this request.
• tls_session_reused — whether the TLS handshake resumed a previous session.
• closed and consumed — response resource and body-consumption state.
• content_decoded — whether streaming content decoding happened in the native response path.
• aiter_content(), aiter_lines(), and aclose() — asynchronous streaming helpers.

response = session.get("https://example.com/account")

print(response.status_code, response.elapsed)
print(response.request.method, response.request.path_url)
print(response.cookies.get_dict())
print(response.links)

Requests compatibility

The API directly supports common Requests-style method helpers, Session state, query parameters, headers, form data, JSON, files, cookies, timeouts, certificate verification, proxies, redirects, streaming, and response helpers.

It is not a drop-in replacement for every Requests extension point. Request authentication handlers, response hooks, transport adapters, adapter mounting, and custom RequestsCookieJar policy objects are not implemented.

TLS verification

Certificate verification is enabled by default and uses the installed certifi CA bundle.

# Default CA bundle
Session("chrome_android_150", verify=True)

# Custom CA file
Session("chrome_android_150", verify="/path/to/private-ca.pem")

# Disable verification explicitly
Session("chrome_android_150", verify=False)

Disabling verification removes server identity protection and should only be used in controlled test environments.

Error handling

import rex_tls

try:
    response = rex_tls.get(
        "https://example.com/",
        profile="chrome_android_150",
        timeout=10,
    )
    response.raise_for_status()
except rex_tls.HTTPError as exc:
    print(f"HTTP error: {exc}")
except rex_tls.RequestError as exc:
    print(f"Request failed: {exc}")
except rex_tls.MobileTLSError as exc:
    print(f"Native client error: {exc}")
except (TypeError, ValueError) as exc:
    print(f"Invalid configuration: {exc}")

Additional exception types:

• CookieConflictError: more than one stored cookie matches an ambiguous lookup.
• ContentDecodingError: a compressed response is malformed or exceeds limits.
• InvalidHeader: a header name or value is invalid for the selected protocol.
• InvalidURL: the URL cannot be parsed or is unsupported.
• SessionClosedError: an operation used a closed session.
• StreamClosedError: the response stream is already closed.
• StreamConsumedError: the response stream has already been consumed.
• UnrewindableBodyError: a redirect requires replaying a one-shot upload body.

Performance summary

rex-tls was compared on the same Windows host with curl_cffi, never_primp, and httpcloak. The tests use controlled local servers so they can reveal client-side overhead and concurrency regressions without public-network noise.

The practical conclusions are:

• For one request at a time, all four clients are close enough that real network latency will usually matter more than the local Python overhead.
• Under HTTP/1.1 concurrency, rex-tls scales well. In this test it outperformed curl_cffi and httpcloak at higher concurrency, while never_primp remained competitive and was faster in some cases.
• Under HTTP/2 concurrency, rex-tls correctly multiplexes requests on one connection and remains stable. curl_cffi and never_primp achieved higher peak throughput in several cases, so rex-tls is not presented as universally the fastest client.
• TLS session resumption worked for rex-tls in the controlled reconnect test, reducing the cost of repeated TLS handshakes.
• Both Chrome Android profiles completed the HTTP/3 concurrency matrix on one QUIC connection without functional failures.

How to read benchmark terms:

• RPS means completed requests per second; higher is better.
• P95 means 95 percent of requests finished within that time; lower is better.
• Concurrency is the maximum number of requests allowed to be in progress.
• Local results are useful for comparing implementation overhead, but they do not predict every public website, proxy, server, or network path.

For normal applications, reuse a Session. Use AsyncSession for asyncio code. Use SessionPool or AsyncSessionPool only when the workload needs several independent sessions, separate cookie jars, or a fixed pool-wide concurrency limit.

The full benchmark method and reproducible commands are documented in the performance guide.

Runtime information

import rex_tls

print(rex_tls.__version__)
print(rex_tls.profiles())
print(dict(rex_tls.native_versions()))
print(rex_tls.profile_info("chrome_android_150"))

License

MIT