hpke-http 1.5.0

End-to-end encryption for HTTP APIs using RFC 9180 HPKE. Drop-in middleware for FastAPI, aiohttp, and httpx.

hpke-http

End-to-end encryption for HTTP APIs using RFC 9180 HPKE (Hybrid Public Key Encryption). Drop-in middleware for FastAPI, aiohttp, and httpx.

Highlights

• Transparent - Drop-in middleware, no application code changes
• End-to-end encryption - Protects data even when TLS terminates at CDN or load balancer
• PSK binding - Each request cryptographically bound to pre-shared key (API key)
• Replay protection - Counter-based nonces prevent replay attacks
• RFC 9180 compliant - Auditable, interoperable standard
• Memory-efficient - Streams large file uploads with O(chunk_size) memory

Installation

uv add "hpke-http[fastapi]"       # Server
uv add "hpke-http[aiohttp]"       # Client (aiohttp)
uv add "hpke-http[httpx]"         # Client (httpx)
uv add "hpke-http[fastapi,zstd]"  # + zstd compression (gzip fallback included)

Quick Start

Standard JSON requests, SSE (Server-Sent Events) streaming, and file uploads are transparently encrypted.

Server (FastAPI)

from fastapi import FastAPI, Request
from fastapi.responses import StreamingResponse
from hpke_http.middleware.fastapi import HPKEMiddleware
from hpke_http.constants import KemId

app = FastAPI()

async def resolve_psk(scope: dict) -> tuple[bytes, bytes]:
    # Get derived PSK ID from X-HPKE-PSK-ID header (already decoded)
    psk_id = scope.get("hpke_psk_id")
    # Look up API key by its derived ID (see "PSK Authentication" section)
    record = await db.lookup_by_derived_id(psk_id)  # Returns {psk, tenant_id}
    scope["tenant_id"] = record["tenant_id"]  # For authorization
    return (record["psk"], psk_id)

app.add_middleware(
    HPKEMiddleware,
    private_keys={KemId.DHKEM_X25519_HKDF_SHA256: private_key},
    psk_resolver=resolve_psk,
)

@app.post("/users")
async def create_user(request: Request):
    data = await request.json()  # Decrypted by middleware
    return {"id": 123, "name": data["name"]}  # Encrypted by middleware

@app.get("/users/{user_id}")
async def get_user(request: Request):
    return {"id": 123, "name": "Alice"}  # Encrypted by middleware

@app.post("/chat")
async def chat(request: Request):
    data = await request.json()

    async def generate():
        yield b"event: progress\ndata: {\"step\": 1}\n\n"
        yield b"event: complete\ndata: {\"result\": \"done\"}\n\n"

    return StreamingResponse(generate(), media_type="text/event-stream")

Client (aiohttp)

import hashlib
import aiohttp
from hpke_http.middleware.aiohttp import HPKEClientSession

# Derive PSK ID from API key (see "PSK Authentication" section)
psk_id = hashlib.sha256(api_key).digest()

async with HPKEClientSession(
    base_url="https://api.example.com",
    psk=api_key,        # >= 32 bytes
    psk_id=psk_id,      # Derived from key, not tenant ID
    # compress=True,           # Compression (zstd preferred, gzip fallback)
    # require_encryption=True, # Raise if server responds unencrypted
    # release_encrypted=True,  # Free encrypted bytes after decryption (saves memory)
) as session:
    # POST with JSON body
    async with session.post("/users", json={"name": "Alice"}) as resp:
        user = await resp.json()

    # SSE streaming
    async with session.post("/chat", json={"prompt": "Hello"}) as resp:
        async for chunk in session.iter_sse(resp):
            print(chunk)  # b"event: progress\ndata: {...}\n\n"

    # GET (bodyless) - response is still encrypted
    async with session.get("/users/123") as resp:
        user = await resp.json()

    # File upload - streams with O(chunk_size) memory
    form = aiohttp.FormData()
    form.add_field("file", open("large.pdf", "rb"), filename="large.pdf")
    async with session.post("/upload", data=form) as resp:
        result = await resp.json()

Client (httpx)

import hashlib
from hpke_http.middleware.httpx import HPKEAsyncClient

# Derive PSK ID from API key (see "PSK Authentication" section)
psk_id = hashlib.sha256(api_key).digest()

async with HPKEAsyncClient(
    base_url="https://api.example.com",
    psk=api_key,        # >= 32 bytes
    psk_id=psk_id,      # Derived from key, not tenant ID
    # compress=True,           # Compression (zstd preferred, gzip fallback)
    # require_encryption=True, # Raise if server responds unencrypted
    # release_encrypted=True,  # Free encrypted bytes after decryption (saves memory)
) as client:
    # POST with JSON body
    resp = await client.post("/users", json={"name": "Alice"})
    user = resp.json()

    # SSE streaming
    resp = await client.post("/chat", json={"prompt": "Hello"})
    async for chunk in client.iter_sse(resp):
        print(chunk)  # b"event: progress\ndata: {...}\n\n"

    # GET (bodyless) - response is still encrypted
    resp = await client.get("/users/123")
    user = resp.json()

    # File upload - streams with O(chunk_size) memory
    resp = await client.post("/upload", files={"file": open("large.pdf", "rb")})
    result = resp.json()

Documentation

• RFC 9180 - HPKE
• RFC 7748 - X25519
• RFC 5869 - HKDF
• RFC 8439 - ChaCha20-Poly1305
• RFC 8878 - Zstandard (preferred compression)
• RFC 1952 - Gzip (fallback compression, always available)
• RFC 9110 - HTTP Semantics (Accept-Encoding negotiation)

Cipher Suite

Component	Algorithm	ID
KEM (Key Encapsulation)	DHKEM(X25519, HKDF-SHA256)	0x0020
KDF (Key Derivation)	HKDF-SHA256	0x0001
AEAD (Authenticated Encryption)	ChaCha20-Poly1305	0x0003
Mode	PSK (Pre-Shared Key)	0x01

PSK Authentication

HPKE PSK mode binds each request to a pre-shared key. This requires two values:

Value	What it is	Example
PSK	The secret key material	API key bytes, b"sk_live_7f3a9c..."
PSK ID	Identifies which PSK to use	SHA256(api_key) — 32 bytes recommended, min 1 byte

Data model: One tenant typically has many API keys (dev/prod, per-service, per-team-member). The PSK ID identifies the specific key, not the tenant.

Security Considerations

RFC 9180 §9.4 warns that psk_id "might be considered sensitive, since, in a given application context, [it] might identify the sender."

The X-HPKE-PSK-ID header is sent in plaintext (only base64url-encoded, not encrypted). RFC 9257 documents the risks:

Risk	Description
Passive linkability	Observers correlate connections using the same PSK ID
Traffic analysis	Identify specific API keys/users by their identifier
Active suppression	Targeted blocking based on observed identifiers

Mitigation: Derive PSK ID from the Key

Derive psk_id from the PSK itself (RFC 9180 §9.4):

sequenceDiagram
    participant C as Client
    participant S as Server

    Note over C: psk_id = SHA256(psk)
    C->>C: Encrypt body with (psk, psk_id)
    C->>S: POST /api<br/>X-HPKE-PSK-ID: <derived_id>
    S->>S: Lookup PSK by derived_id
    S->>S: Decrypt with (psk, psk_id)
    S-->>C: Encrypted response

Implementation

Client — derive PSK ID from key:

import hashlib

api_key = b"sk_live_7f3a9c..."  # Your API key (>= 32 bytes)
# Derive PSK ID from the key itself
psk_id = hashlib.sha256(api_key).digest()

async with HPKEClientSession(
    base_url="https://api.example.com",
    psk=api_key,
    psk_id=psk_id,
) as client:
    await client.post("/api", json=data)

Server — store derived ID when key created, lookup on request:

import hashlib

# Key creation: store derived_id → {psk, tenant_id}
derived_id = hashlib.sha256(api_key).digest()
db.store(derived_id, {"psk": api_key, "tenant_id": tenant_id})

# psk_resolver: lookup by derived_id from header
async def resolve_psk(scope: dict) -> tuple[bytes, bytes]:
    derived_id = scope.get("hpke_psk_id")
    record = await db.lookup(derived_id)
    scope["tenant_id"] = record["tenant_id"]
    return (record["psk"], derived_id)

Wire Format

Request/Response (Chunked Binary)

See Header Modifications for when headers are added.

Headers:
  X-HPKE-Enc: <base64url(32B ephemeral key)>
  X-HPKE-Stream: <base64url(4B session salt)>
  X-HPKE-PSK-ID: <base64url(derived key ID, 32B recommended)>

Body (repeating chunks):
┌───────────┬────────────┬─────────────────────────────────┐
│ Length(4B)│ Counter(4B)│ Ciphertext (N + 16B tag)        │
│ big-endian│ big-endian │ encrypted: encoding_id || data  │
└───────────┴────────────┴─────────────────────────────────┘
Overhead: 24B/chunk (4B length + 4B counter + 16B tag)

SSE Event

event: enc
data: <base64(counter_be32 || ciphertext)>
Decrypted: raw SSE chunk (e.g., "event: progress\ndata: {...}\n\n")

Uses standard base64 (not base64url) - SSE data fields allow +/= characters.

Compression (Optional)

Zstd reduces bandwidth by 40-95% for JSON/text. Enable with compress=True on both client and server. Payloads < 64 bytes skip compression. See Compression table for algorithm priority.

Pitfalls

# PSK too short
HPKEClientSession(psk=b"short", psk_id=...)     # InvalidPSKError
HPKEClientSession(psk=secrets.token_bytes(32), psk_id=...)  # >= 32 bytes

# PSK ID must be derived from the key (see "PSK Authentication" section)
psk_id = hashlib.sha256(api_key).digest()
HPKEClientSession(psk=api_key, psk_id=psk_id)   # Correct

# SSE missing content-type (won't use SSE format)
return StreamingResponse(gen())                                  # Binary format (wrong for SSE)
return StreamingResponse(gen(), media_type="text/event-stream")  # SSE format (correct)

# Standard responses work automatically - no special handling needed
return {"data": "value"}  # Auto-encrypted as binary chunks

Limits

Resource	Limit	Applies to
HPKE messages/context	2^96-1	All
Chunks/session	2^32-1	All
PSK minimum	32 bytes	All
PSK ID minimum	1 byte	All
Chunk size	64KB	All
Binary chunk overhead	24B (length + counter + tag)	Requests & standard responses
SSE event buffer	64MB (configurable)	SSE only

Note: SSE is text-only (UTF-8). Binary data must be base64-encoded (+33% overhead).

HTTP Compatibility

Protocol Support

Feature	Supported	Notes
HTTP/1.1	Yes	Chunked transfer encoding for streaming
HTTP/2	Yes	Native framing (chunked encoding forbidden by spec)
HTTP/3	Yes	QUIC streams, same semantics as HTTP/2
WebSockets	No	Different protocol, not applicable

HTTP Methods

HPKE key exchange happens on every request, including bodyless methods like GET and DELETE.

Method	Typical Use	Request Body	Response
POST	Create	Encrypted	Encrypted
PUT	Replace	Encrypted	Encrypted
PATCH	Update	Encrypted	Encrypted
DELETE	Remove	Encrypted (if body)	Encrypted
GET	Read	No body	Encrypted
HEAD	Metadata	No body	Headers only (no body per HTTP spec)
OPTIONS	Preflight	No body	Encrypted

Response Encryption (Server)

Content-Type	Wire Format	Memory
Any non-SSE	Length-prefixed 64KB chunks	O(64KB) buffer
text/event-stream	Base64 SSE events	O(event size)

Response Decryption (Client)

Content-Type	API	Memory	Delivery
Any non-SSE	resp.json(), resp.content	O(response size)	After full download
text/event-stream	async for chunk in iter_sse(resp)	O(event size)	As events arrive

Use release_encrypted=True to free encrypted buffer after decryption (reduces peak memory).

Compression

Algorithm	Request	Response	Priority
Zstd (RFC 8878)	Yes	Yes	1 (preferred)
Gzip (RFC 1952)	Yes	Yes	2 (fallback)
Identity	Yes	Yes	3 (no compression)

Auto-negotiated via Accept-Encoding header on discovery endpoint (/.well-known/hpke-keys).

Why HTTP-Level Compression Doesn't Help

Disable gzip/brotli on CDN/LB for HPKE endpoints. Ciphertext is incompressible—HTTP compression wastes CPU. Use compress=True on the client instead (compresses before encryption).

Encryption Scope

What IS Encrypted

Component	Encrypted	Format
Request body	Yes	Binary chunks
Response body	Yes	Binary chunks or SSE events

What is NOT Encrypted

Component	Visible to	Reason
URL path	Network	Routing requires plaintext
Query parameters	Network	Part of URL
HTTP method	Network	Protocol requirement
HTTP headers	Network	Routing, caching, auth
Status code	Network	Protocol requirement
TLS metadata	Network	Transport layer

Header Modifications

Header	Request	Response	Reason
Content-Type	Set to application/octet-stream (if body)	Preserved	Encrypted body is binary
Content-Length	Auto (chunked, if body)	Removed	Size changes after encryption
X-HPKE-Enc	Always	-	Ephemeral public key
X-HPKE-Stream	Always	Added	Session salt for nonces
X-HPKE-PSK-ID	Always	-	Derived PSK identifier (see PSK Authentication)
X-HPKE-Encoding	Added (if compressed)	-	Compression algorithm
X-HPKE-Content-Type	Added (if body)	-	Original Content-Type for server parsing

Security Boundary

┌─────────────────────────────────────────────────────────────┐
│ TLS Encrypted (transport)                                   │
│  ┌───────────────────────────────────────────────────────┐  │
│  │ HTTP Layer (visible to CDN/LB/proxies)                │  │
│  │  • Method: POST                                       │  │
│  │  • URL: /api/chat                                     │  │
│  │  • Headers: Authorization, X-HPKE-*, Content-Type     │  │
│  │  ┌─────────────────────────────────────────────────┐  │  │
│  │  │ HPKE Encrypted (end-to-end)                     │  │  │
│  │  │  • Request body: {"prompt": "Hello"}            │  │  │
│  │  │  • Response body: {"response": "Hi!"}           │  │  │
│  │  │  • SSE events: event: done\ndata: {...}\n\n     │  │  │
│  │  └─────────────────────────────────────────────────┘  │  │
│  └───────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────┘

Low-Level API

Direct access to HPKE seal/open operations:

from hpke_http.hpke import seal_psk, open_psk

# pk_r: recipient public key, sk_r: recipient secret key
# psk/psk_id: pre-shared key and identifier, aad: additional authenticated data
enc, ct = seal_psk(pk_r, b"info", psk, psk_id, b"aad", b"plaintext")
pt = open_psk(enc, sk_r, b"info", psk, psk_id, b"aad", ct)

Security

Uses OpenSSL constant-time implementations via cryptography library.

• Security Policy - Vulnerability reporting
• SBOM - Software Bill of Materials (CycloneDX format) attached to releases

Contributing

Contributions welcome! Please open an issue first to discuss changes.

make install      # Setup venv
make test         # Run tests
make lint         # Format and lint

License

Apache-2.0