pyhaul

Resumable HTTP downloads for Python. Bring your own client: pyhaul borrows your existing session and handles byte-range negotiation, crash-safe checkpointing, and validation.

Install pyhaul from PyPI — pick the extra matching your HTTP client:

pip install pyhaul[httpx]   # or: niquests, requests, urllib3, aiohttp, wreq

import httpx
from pathlib import Path
from pyhaul import haul, PartialHaulError

dest = Path("big.zip")
with httpx.Client() as client:
    for _ in range(10):
        try:
            result = haul("https://example.com/big.zip", client, dest=dest)
            break
        except PartialHaulError:
            pass  # only retryable error; others propagate

print(f"done: {dest.stat().st_size:,} bytes")

What is it?

A small, pure-Python library that makes HTTP downloads resumable. Call haul() with a URL, your existing HTTP client, and a destination path. pyhaul handles byte-range negotiation, ETag validation, crash-safe checkpointing, and atomic file completion. Supports both sync and async across multiple HTTP client libraries.

Each call to haul() upholds these guarantees:

• One haul() makes one request. You are responsible for retry loops, but retry just means call haul() again.
• The destination file will not exist until download is complete. Incomplete data lives in a temporary .part file; on completion it is atomically moved into place.
• Interrupted downloads resume when possible. Checkpoint state lives on disk, not in memory. Kill the process, lose the network, get a 503 — the next haul() picks up from the last durable byte.
• If the remote resource changes, retry will not corrupt. ETag mismatch detection prevents gluing mismatched halves together.
• Your HTTP client is borrowed, not owned. pyhaul sets per-request headers and returns your session untouched.
• Transport errors pass through unwrapped. httpx.ReadTimeout stays httpx.ReadTimeout. You catch the types you already know.

No hard dependency on any HTTP library. Pick one (or several) as extras.

Documentation

Full documentation — Quick start, guides, and API reference.

• Quick Start — install, first download, async usage
• Why pyhaul Exists — silent failure modes in HTTP resume, comparison with curl/wget/aria2c
• Design & Architecture — transport adapters, checkpoint state, download lifecycle
• API Reference — haul(), haul_async(), HaulState, exceptions
• Control File Spec — checkpoint format for implementers