pyhaul 0.6.1

Resumable, cursor-based, CDN-safe HTTP downloads for Python

pyhaul

Resumable HTTP downloads for Python. Bring your own client: pyhaul borrows your existing session and handles byte-range negotiation, crash-safe checkpointing, and validation.

pip install pyhaul[httpx]   # or: niquests, requests, urllib3, aiohttp

import httpx
from pathlib import Path
from pyhaul import haul, PartialHaulError

dest = Path("big.zip")
with httpx.Client() as client:
    for _ in range(10):
        try:
            result = haul("https://example.com/big.zip", client, dest=dest)
            break
        except PartialHaulError:
            pass  # only retryable error; others propagate

print(f"done: {dest.stat().st_size:,} bytes")

---

What is it?

A small, pure-Python library that makes HTTP downloads resumable. To download a file, call haul() with a URL, your existing HTTP client, and a destination path. pyhaul handles byte-range negotiation for resume, ETag validation, crash-safe checkpointing, and atomic file completion. Supports both sync and async across multiple HTTP client libraries.

Each call to haul() upholds these guarantees:

• One haul() makes one request. You are responsible for retry loops, but retry just means call haul() again.
• The destination file will not exist until download is complete. There is no state where a partially-written file sits at the final path. Incomplete data lives in a temporary .part file; on completion it is atomically moved into place.
• Interrupted downloads resume when possible. Checkpoint state lives on disk, not in memory. Kill the process, lose the network, get a 503 — the next haul() picks up from the last durable byte. Zero re-downloaded data if the resource hasn't changed.
• If the remote resource changes, retry will not corrupt. If the remote file changes between attempts, pyhaul detects the mismatch via ETag (a server-side fingerprint) and starts over cleanly instead of gluing mismatched halves together.
• Your HTTP client is borrowed, not owned. pyhaul sets per-request headers and returns your session untouched. It never creates, configures, or closes sessions.
• Transport errors pass through unwrapped. httpx.ReadTimeout stays httpx.ReadTimeout. You catch the types you already know.

How it fits into your code

One haul() = one HTTP request. It either succeeds and returns CompleteHaul, or it throws — possibly after saving progress to a .part file that allows the next call to resume. pyhaul never creates sessions, connections, or clients. Your HTTP library's native exceptions propagate through unwrapped, so you can drop haul() into existing code without changing your error handling. Retries are your call — a for-loop, tenacity, or nothing. Concurrency limiting (e.g. asyncio.Semaphore) is also yours — pyhaul downloads one file per call and doesn't manage parallelism.

def haul(url, client, *, dest) -> CompleteHaul: ...
async def haul_async(url, client, *, dest) -> CompleteHaul: ...

Optional HaulState (progress bag, updated in-place) and other keyword-only options (extra headers, progress hooks, buffer sizing) are documented on the site. See docs/DESIGN.md for the exception hierarchy, transport adapters, and download lifecycle.

Documentation

Full documentation →

• docs/DESIGN.md — Transport adapters, checkpoint state, and the download lifecycle.
• docs/WHY.md — Silent failure modes in HTTP range/resume, and how pyhaul compares to curl, wget, and aria2c.
• docs/SPEC.md — Control file and checkpoint format (implementers / compatible tools).