Skip to main content

HTTP library with thread-safe connection pooling and file post support

Project description


  • Re-use the same socket connection for multiple requests (HTTPConnectionPool)

  • File posting (encode_multipart_formdata)

  • Built-in redirection and retries (optional)

  • Thread-safe

What’s wrong with urllib and urllib2?

There are two critical features missing from the Python standard library: Connection re-using/pooling and file posting. It’s not terribly hard to implement these yourself, but it’s much easier to use a module that already did the work for you.

The Python standard libraries urllib and urllib2 have little to do with each other. They were designed to be independent and standalone, each solving a different scope of problems, and urllib3 follows in a similar vein.

Why do I want to reuse connections?

Performance. When you normally do a urllib call, a separate socket connection is created with each request. By reusing existing sockets (supported since HTTP 1.1), the requests will take up less resources on the server’s end, and also provide a faster response time at the client’s end. With some simple benchmarks (see test/ ), downloading 15 URLs from is about twice as fast when using HTTPConnectionPool (which uses 1 connection) than using plain urllib (which uses 15 connections).

This library is perfect for:

  • Talking to an API

  • Crawling a website

  • Any situation where being able to post files, handle redirection, and retrying is useful. It’s relatively lightweight, so it can be used for anything!


Go to the Examples wiki for more nice syntax-highlighted examples.

But, long story short:

from urllib3 import HTTPConnectionPool

API_URL = ''

http_pool = HTTPConnectionPool.from_url(API_URL)

fields = {'v': '1.0', 'q': 'urllib3'}
r = http_pool.get_url(API_URL, fields)

print r.status,

Project details

Release history Release notifications | RSS feed

This version


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

urllib3-0.3.tar.gz (10.1 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page