requests-cache

Summary

requests-cache is a persistent HTTP cache that provides an easy way to get better performance with the python requests library.

Complete project documentation can be found at requests-cache.readthedocs.io.

Features

🍰 Ease of use: Keep using the requests library you're already familiar with. Add caching with a drop-in replacement for requests.Session, or install globally to add transparent caching to all requests functions.
🚀 Performance: Get sub-millisecond response times for cached responses. When they expire, you still save time with conditional requests.
💾 Persistence: Works with several storage backends including SQLite, Redis, MongoDB, and DynamoDB; or save responses as plain JSON files, YAML, and more
🕗 Expiration: Use Cache-Control and other standard HTTP headers, define your own expiration schedule, keep your cache clutter-free with backends that natively support TTL, or any combination of strategies
⚙️ Customization: Works out of the box with zero config, but with a robust set of features for configuring and extending the library to suit your needs
🧩 Compatibility: Can be combined with other popular libraries based on requests

Quickstart

First, install with pip:

pip install requests-cache

Then, use requests_cache.CachedSession to make your requests. It behaves like a normal requests.Session, but with caching behavior.

To illustrate, we'll call an endpoint that adds a delay of 1 second, simulating a slow or rate-limited website.

This takes 1 minute:

import requests

session = requests.Session()
for i in range(60):
    session.get('https://httpbin.org/delay/1')

This takes 1 second:

import requests_cache

session = requests_cache.CachedSession('demo_cache')
for i in range(60):
    session.get('https://httpbin.org/delay/1')

With caching, the response will be fetched once, saved to demo_cache.sqlite, and subsequent requests will return the cached response near-instantly.

Patching

If you don't want to manage a session object, or just want to quickly test it out in your application without modifying any code, requests-cache can also be installed globally, and all requests will be transparently cached:

import requests
import requests_cache

requests_cache.install_cache('demo_cache')
requests.get('https://httpbin.org/delay/1')

Headers and Expiration

By default, requests-cache will keep cached responses indefinitely. In most cases, you will want to use one of the two following strategies to balance cache freshness and performance:

Define exactly how long to keep responses:

Use the expire_after parameter to set a fixed expiration time for all responses:

from requests_cache import CachedSession
from datetime import timedelta

# Keep responses for 360 seconds
session = CachedSession('demo_cache', expire_after=360)

# Or use timedelta objects to specify other units of time
session = CachedSession('demo_cache', expire_after=timedelta(hours=1))

See Expiration for more features and settings.

Use Cache-Control headers:

Use the cache_control parameter to enable automatic expiration based on Cache-Control and other standard HTTP headers sent by the server:

from requests_cache import CachedSession

session = CachedSession('demo_cache', cache_control=True)

See Cache Headers for more details.

Settings

The default settings work well for most use cases, but there are plenty of ways to customize caching behavior when needed. Here is a quick example of some of the options available:

from datetime import timedelta
from requests_cache import CachedSession

session = CachedSession(
    'demo_cache',
    use_cache_dir=True,                # Save files in the default user cache dir
    cache_control=True,                # Use Cache-Control response headers for expiration, if available
    expire_after=timedelta(days=1),    # Otherwise expire responses after one day
    allowable_codes=[200, 400],        # Cache 400 responses as a solemn reminder of your failures
    allowable_methods=['GET', 'POST'], # Cache whatever HTTP methods you want
    ignored_parameters=['api_key'],    # Don't match this request param, and redact if from the cache
    match_headers=['Accept-Language'], # Cache a different response per language
    stale_if_error=True,               # In case of request errors, use stale cache data if possible
)

Next Steps

To find out more about what you can do with requests-cache, see:

Metadata

Release files for requests-cache 1.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for requests-cache 1.0.0
File	Size	Uploaded
requests_cache-1.0.0.tar.gz	2.9 MB	Mar 2, 2023	Details

Built distribution (wheel)

Table of built distributions (wheels) for requests-cache 1.0.0
File	Interpreter	ABI	Platform	Reset
requests_cache-1.0.0-py3-none-any.whl 58.6 kB Mar 2, 2023	Python 3	none	any	Details

Total release size: 2.9 MB

Release files / requests_cache-1.0.0.tar.gz

Download URL	requests_cache-1.0.0.tar.gz
Size	2.9 MB
Tags	Source
SHA-256 checksum How to use checksums	`162f591ed9ad0ea601cefd898770f499d473cb056466e85494299a1b0ca61cbf`
BLAKE2b-256 checksum How to use checksums	`ae362d19b9c2442cb73c4a9a3a9c2916cc1709077ed3f78d8c7270f9ae5273ca`
Upload date	Mar 2, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.4.0 CPython/3.11.2 Linux/5.15.0-1033-azure`

Release files / requests_cache-1.0.0-py3-none-any.whl

Download URL	requests_cache-1.0.0-py3-none-any.whl
Size	58.6 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`558e335bf1c0f42ecbcf1abb8e7c8e679845ee522d75d1c50b77d4fbf5434012`
BLAKE2b-256 checksum How to use checksums	`0074dc9df8e0914b6d1261dd87e19a162888da0e0197d2db321f16f869298522`
Upload date	Mar 2, 2023
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`poetry/1.4.0 CPython/3.11.2 Linux/5.15.0-1033-azure`