requests-toolbelt

requests toolbelt

This is just a collection of utilities for python-requests, but don’t really belong in requests proper. The minimum tested requests version is 2.1.0. In reality, the toolbelt should work with 2.0.1 as well, but some idiosyncracies prevent effective or sane testing on that version.

pip install requests-toolbelt to get started!

multipart/form-data Encoder

The main attraction is a streaming multipart form-data object, MultipartEncoder. Its API looks like this:

from requests_toolbelt import MultipartEncoder
import requests

m = MultipartEncoder(
    fields={'field0': 'value', 'field1': 'value',
            'field2': ('filename', open('file.py', 'rb'), 'text/plain')}
    )

r = requests.post('http://httpbin.org/post', data=m,
                  headers={'Content-Type': m.content_type})

You can also use multipart/form-data encoding for requests that don’t require files:

from requests_toolbelt import MultipartEncoder
import requests

m = MultipartEncoder(fields={'field0': 'value', 'field1': 'value'})

r = requests.post('http://httpbin.org/post', data=m,
                  headers={'Content-Type': m.content_type})

Or, you can just create the string and examine the data:

# Assuming `m` is one of the above
m.to_string()  # Always returns unicode

User-Agent constructor

You can easily construct a requests-style User-Agent string:

from requests_toolbelt import user_agent

headers = {
    'User-Agent': user_agent('my_package', '0.0.1')
    }

r = requests.get('https://api.github.com/users', headers=headers)

SSLAdapter

The SSLAdapter was originally published on Cory Benfield’s blog. This adapter allows the user to choose one of the SSL protocols made available in Python’s ssl module for outgoing HTTPS connections:

from requests_toolbelt import SSLAdapter
import requests
import ssl

s = requests.Session()
s.mount('https://', SSLAdapter(ssl.PROTOCOL_TLSv1))

cookies/ForgetfulCookieJar

The ForgetfulCookieJar prevents a particular requests session from storing cookies:

from requests-toolbelt.cookies import ForgetfulCookieJar

session = requests.Session()
session.cookies = ForgetfulCookieJar()

Known Issues

On Python 3.3.0 and 3.3.1, the standard library’s http module will fail when passing an instance of the MultipartEncoder. This is fixed in later minor releases of Python 3.3. Please consider upgrading to a later minor version or Python 3.4. There is absolutely nothing this library can do to work around that bug.

Contributing

Please read the suggested workflow for contributing to this project.

History

0.6.2 – 2016-05-10

Fixed Bugs

When passing a timeout via Requests, it was not appropriately translated to the timeout that the urllib3 code was expecting.

0.6.1 – 2016-05-05

Fixed Bugs

Remove assertion about request URLs in the AppEngineAdapter.
Prevent pip from installing requests 3.0.0 when that is released until we are ready to handle it.

0.6.0 – 2016-01-27

More information about this release can be found on the 0.6.0 milestone.

New Features

Add AppEngineAdapter to support developers using Google’s AppEngine platform with Requests.
Add GuessProxyAuth class to support guessing between Basic and Digest Authentication for proxies.

Fixed Bugs

Ensure that proxies use the correct TLS version when using the SSLAdapter.
Fix an AttributeError when using the HTTPProxyDigestAuth class.

Miscellaneous

Drop testing support for Python 3.2. virtualenv and pip have stopped supporting it meaning that it is harder to test for this with our CI infrastructure. Moving forward we will make a best-effort attempt to support 3.2 but will not test for it.

0.5.1 – 2015-12-16

More information about this release can be found on the 0.5.1 milestone.

Fixed Bugs

Now papers over the differences in requests’ super_len function from versions prior to 2.9.0 and versions 2.9.0 and later.

0.5.0 – 2015-11-24

More information about this release can be found on the milestone for 0.5.0.

New Features

The tee submodule was added to requests_toolbelt.downloadutils. It allows you to iterate over the bytes of a response while also writing them to a file. The tee.tee function, expects you to pass an open file object, while tee.tee_to_file will use the provided file name to open the file for you.
Added a new parameter to requests_toolbelt.utils.user_agent that allows the user to specify additional items.
Added nested form-data helper, requests_toolbelt.utils.formdata.urlencode.
Added the ForgetfulCookieJar to requests_toolbelt.cookies.
Added utilities for dumping the information about a request-response cycle in requests_toolbelt.utils.dump.
Implemented the API described in the requests_toolbelt.threaded module docstring, i.e., added requests_toolbelt.threaded.map as an available function.

Fixed Bugs

Now papers over the API differences in versions of requests installed from system packages versus versions of requests installed from PyPI.
Allow string types for SourceAddressAdapter.

0.4.0 – 2015-04-03

For more information about this release, please see milestone 0.4.0 on the project’s page.

New Features

A naive implemenation of a thread pool is now included in the toolbelt. See the docs in docs/threading.rst or on Read The Docs.
The StreamingIterator now accepts files (such as sys.stdin) without a specific length and will properly stream them.
The MultipartEncoder now accepts exactly the same format of fields as requests’ files parameter does. In other words, you can now also pass in extra headers to add to a part in the body. You can also now specify a custom Content-Type for a part.
An implementation of HTTP Digest Authentication for Proxies is now included.
A transport adapter that allows a user to specify a specific Certificate Fingerprint is now included in the toolbelt.
A transport adapter that simplifies how users specify socket options is now included.
A transport adapter that simplifies how users can specify TCP Keep-Alive options is now included in the toolbelt.
Deprecated functions from requests.utils are now included and maintained.
An authentication tool that allows users to specify how to authenticate to several different domains at once is now included.
A function to save streamed responses to disk by analyzing the Content-Disposition header is now included in the toolbelt.

Fixed Bugs

The MultipartEncoder will now allow users to upload files larger than 4GB on 32-bit systems.
The MultipartEncoder will now accept empty unicode strings for form values.

0.3.1 – 2014-06-23

Fix the fact that 0.3.0 bundle did not include the StreamingIterator

0.3.0 – 2014-05-21

Bug Fixes

Complete rewrite of MultipartEncoder fixes bug where bytes were lost in uploads

New Features

MultipartDecoder to accept multipart/form-data response bodies and parse them into an easy to use object.
SourceAddressAdapter to allow users to choose a local address to bind connections to.
GuessAuth which accepts a username and password and uses the WWW-Authenticate header to determine how to authenticate against a server.
MultipartEncoderMonitor wraps an instance of the MultipartEncoder and keeps track of how many bytes were read and will call the provided callback.
StreamingIterator will wrap an iterator and stream the upload instead of chunk it, provided you also provide the length of the content you wish to upload.

0.2.0 – 2014-02-24

Add ability to tell MultipartEncoder which encoding to use. By default it uses ‘utf-8’.
Fix #10 - allow users to install with pip
Fix #9 - Fix MultipartEncoder#to_string so that it properly handles file objects as fields

0.1.2 – 2014-01-19

At some point during development we broke how we handle normal file objects. Thanks to @konomae this is now fixed.

0.1.1 – 2014-01-19

Handle io.BytesIO-like objects better

0.1.0 – 2014-01-18

Add initial implementation of the streaming MultipartEncoder
Add initial implementation of the user_agent function
Add the SSLAdapter

Release files for requests-toolbelt 0.6.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for requests-toolbelt 0.6.2
File	Size	Uploaded
requests-toolbelt-0.6.2.tar.gz	190.1 kB	May 10, 2016	Details

Built distribution (wheel)

Table of built distributions (wheels) for requests-toolbelt 0.6.2
File	Interpreter	ABI	Platform	Reset
requests_toolbelt-0.6.2-py2.py3-none-any.whl 49.2 kB May 10, 2016	Python 2, Python 3	none	any	Details

Total release size: 239.2 kB

Release files / requests-toolbelt-0.6.2.tar.gz

Download URL	requests-toolbelt-0.6.2.tar.gz
Size	190.1 kB
Tags	Source
SHA-256 checksum How to use checksums	`263215241278aefeb03b5e84ac4574ea5af191dc3ffc3649a6d816745fb60997`
BLAKE2b-256 checksum How to use checksums	`e1a4a94c037bc72ad70441aff1403d3243510d2542ddca7759faaeffeb11aefe`
Upload date	May 10, 2016
Uploaded using Trusted Publishing? What is trusted publishing?	No

Release files / requests_toolbelt-0.6.2-py2.py3-none-any.whl

Download URL	requests_toolbelt-0.6.2-py2.py3-none-any.whl
Size	49.2 kB
Tags	Python 2 Python 3
SHA-256 checksum How to use checksums	`a252f66501663d655ab6c5be83a08c82afc3932931cc0af54b315a58e34d0a3c`
BLAKE2b-256 checksum How to use checksums	`c006d198c065f25815e43b6fb4acdeb88ffcffe7a383c888bb8da80edc546dbc`
Upload date	May 10, 2016
Uploaded using Trusted Publishing? What is trusted publishing?	No