Skip to main content

Persistent cache for aiohttp requests

Project description

aiohttp-client-cache

Build status Documentation Status Coverage Status PyPI PyPI - Python Versions PyPI - Format

aiohttp-client-cache is an async persistent cache for aiohttp client requests.

See full documentation at https://aiohttp-client-cache.readthedocs.io

Features

  • Ease of use: Use as a drop-in replacement for aiohttp.ClientSession
  • Customization: Works out of the box with little to no config, but with plenty of options available for customizing cache expiration and other behavior
  • Persistence: Includes several storage backends: SQLite, DynamoDB, MongoDB, and Redis.

Development Status

This library is a work in progress!

Breaking changes should be expected until a 1.0 release, so version pinning is recommended.

I am developing this while also maintaining requests-cache, and my goal is to eventually have a similar (but not identical) feature set between the two libraries. If there is a feature you want, or if you've discovered a bug, of it you have other general feedback, please create an issue for it!

Quickstart

First, install with pip (python 3.7+ required):

pip install aiohttp-client-cache

Basic Usage

Next, use aiohttp_client_cache.CachedSession in place of aiohttp.ClientSession. To briefly demonstrate how to use it:

Replace this:

from aiohttp import ClientSession

async with ClientSession() as session:
    await session.get('http://httpbin.org/delay/1')                                                          

With this:

from aiohttp_client_cache import CachedSession, SQLiteBackend

async with CachedSession(cache=SQLiteBackend('demo_cache')) as session:
    await session.get('http://httpbin.org/delay/1')                                                          

The URL in this example adds a delay of 1 second, simulating a slow or rate-limited website. With caching, the response will be fetched once, saved to demo_cache.sqlite, and subsequent requests will return the cached response near-instantly.

Configuration

Several options are available to customize caching behavior. This example demonstrates a few of them:

from aiohttp_client_cache import SQLiteBackend

cache = SQLiteBackend(
    cache_name='~/.cache/aiohttp-requests.db',  # For SQLite, this will be used as the filename
    expire_after=60*60,                         # By default, cached responses expire in an hour
    urls_expire_after={'*.fillmurray.com': -1}, # Requests for any subdomain on this site will never expire
    allowed_codes=(200, 418),                   # Cache responses with these status codes
    allowed_methods=('GET', 'POST'),            # Cache requests with these HTTP methods
    include_headers=True,                       # Cache requests with different headers separately
    ignored_params=['auth_token'],              # Keep using the cached response even if this param changes
    timeout=2.5,                                # Connection timeout for SQLite backend 
)

More Info

To learn more, see:

History

0.3.0 (2021-04-TBD)

See all issues & PRs for v0.3

  • Add async implementation of DynamoDb backend
  • Add support for expiration for individual requests
  • Add support for expiration based on URL patterns
  • Add support for serializing/deserializing ClientSession.links
  • Add case-insensitive response headers for compatibility with aiohttp.ClientResponse.headers
  • Add optional integration with itsdangerous for safer serialization
  • Add CacheBackend.get_urls() to get all urls currently in the cache
  • Add some default attributes (from_cache, is_expired, etc.) to returned ClientResponse objects
  • Allow passing all backend-specific connection kwargs via CacheBackend
  • Add support for json request body
  • Convert all keys() and values() methods into async generators
  • Fix serialization of Content-Disposition
  • Fix filtering ignored parameters for request body (data and json)
  • Add user guide, more examples, and other project docs

0.2.0 (2021-02-28)

See all issues & PRs for v0.2

  • Refactor SQLite backend to use aiosqlite for async cache operations
  • Refactor MongoDB backend to use motor for async cache operations
  • Refactor Redis backend to use aiosqlite for async cache operations
  • Add integration tests and docker-compose for local test servers

0.1.0 (2020-11-14)

  • Initial fork from requests-cache
  • First pass at a general refactor and conversion from requests to aiohttp
  • Basic features are functional, but some backends do not actually operate asynchronously

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for aiohttp-client-cache, version 0.3.0
Filename, size File type Python version Upload date Hashes
Filename, size aiohttp_client_cache-0.3.0-py3-none-any.whl (35.9 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size aiohttp-client-cache-0.3.0.tar.gz (32.5 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page