Persistent cache for aiohttp requests
Project description
aiohttp-client-cache
aiohttp-client-cache is an async persistent cache for aiohttp client requests.
See full documentation at https://aiohttp-client-cache.readthedocs.io
Features
- Ease of use: Use as a drop-in replacement
for
aiohttp.ClientSession
- Customization: Works out of the box with little to no config, but with plenty of options available for customizing cache expiration and other behavior
- Persistence: Includes several storage backends: SQLite, DynamoDB, MongoDB, and Redis.
Development Status
This library is a work in progress!
Breaking changes should be expected until a 1.0
release, so version pinning is recommended.
I am developing this while also maintaining requests-cache, and my goal is to eventually have a similar (but not identical) feature set between the two libraries. If there is a feature you want, or if you've discovered a bug, of it you have other general feedback, please create an issue for it!
Quickstart
First, install with pip (python 3.7+ required):
pip install aiohttp-client-cache
Basic Usage
Next, use aiohttp_client_cache.CachedSession in place of aiohttp.ClientSession. To briefly demonstrate how to use it:
Replace this:
from aiohttp import ClientSession
async with ClientSession() as session:
await session.get('http://httpbin.org/delay/1')
With this:
from aiohttp_client_cache import CachedSession, SQLiteBackend
async with CachedSession(cache=SQLiteBackend('demo_cache')) as session:
await session.get('http://httpbin.org/delay/1')
The URL in this example adds a delay of 1 second, simulating a slow or rate-limited website.
With caching, the response will be fetched once, saved to demo_cache.sqlite
, and subsequent
requests will return the cached response near-instantly.
Configuration
Several options are available to customize caching behavior. This example demonstrates a few of them:
from aiohttp_client_cache import SQLiteBackend
cache = SQLiteBackend(
cache_name='~/.cache/aiohttp-requests.db', # For SQLite, this will be used as the filename
expire_after=60*60, # By default, cached responses expire in an hour
urls_expire_after={'*.fillmurray.com': -1}, # Requests for any subdomain on this site will never expire
allowed_codes=(200, 418), # Cache responses with these status codes
allowed_methods=('GET', 'POST'), # Cache requests with these HTTP methods
include_headers=True, # Cache requests with different headers separately
ignored_params=['auth_token'], # Keep using the cached response even if this param changes
timeout=2.5, # Connection timeout for SQLite backend
)
More Info
To learn more, see:
History
0.3.0 (2021-04-TBD)
- Add async implementation of DynamoDb backend
- Add support for expiration for individual requests
- Add support for expiration based on URL patterns
- Add support for serializing/deserializing
ClientSession.links
- Add case-insensitive response headers for compatibility with aiohttp.ClientResponse.headers
- Add optional integration with
itsdangerous
for safer serialization - Add
CacheBackend.get_urls()
to get all urls currently in the cache - Add some default attributes (
from_cache, is_expired
, etc.) to returned ClientResponse objects - Allow passing all backend-specific connection kwargs via CacheBackend
- Add support for
json
request body - Convert all
keys()
andvalues()
methods into async generators - Fix serialization of Content-Disposition
- Fix filtering ignored parameters for request body (
data
andjson
) - Add user guide, more examples, and other project docs
0.2.0 (2021-02-28)
- Refactor SQLite backend to use
aiosqlite
for async cache operations - Refactor MongoDB backend to use
motor
for async cache operations - Refactor Redis backend to use
aiosqlite
for async cache operations - Add integration tests and
docker-compose
for local test servers
0.1.0 (2020-11-14)
- Initial fork from
requests-cache
- First pass at a general refactor and conversion from
requests
toaiohttp
- Basic features are functional, but some backends do not actually operate asynchronously
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for aiohttp-client-cache-0.3.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d6feafb9924cfd40159f80fd4cd5f39794ff6b90f209bca190a5432d64892cab |
|
MD5 | 4d106b0cf2198891054887c025acea15 |
|
BLAKE2b-256 | 1afb481a11bd57a833ec16b9dafd1bc216b9c2bacb2bca00ad284aee46f5e85d |
Hashes for aiohttp_client_cache-0.3.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b6d1c09aeb5eaa753e488c2e5f98069c8d4f466b325b327af29f37ad181f67de |
|
MD5 | 302e83b08464cd1d26fa39775f5f4076 |
|
BLAKE2b-256 | 7e62ddcbab066dbb50d04cb01dfbc378f8396e731f0c915148b88d7319d40dd2 |