storing cache using dynamodb dax
Project description
dynamocacher
Description
A package which allows you to use dynamodb to cache output of a slow or expensive function, to reduce cost and increase the efficiency
Dax is supported using a Pynamodb fork, and its highly recommended for performance-critical application 50-100x performance difference is expected with dax
ToDos
- Dax support : Done
- Auto create table : Done
- policy template : in progress
- example readme : Done
Examples
SampleUsage
cache this slow function
def power(input:dict):
'''
This is a very bad and slow function
accept a dict containing these keys
base: float
power: int
and return the power embarrasingly slowly
response
result: float
'''
base = input['base']
power = input['power']
result = base
for i in range (power):
result *= base
return {
'result':result
}
def powerWithCaching(input:dict):
cache = Cache.getCache(input, timeout = 3600)
if cache: return cache
result = power(input)
Cache.addCache(input=input, output= result)
return result
performance
Example code for trying it yourself
sampleInput = {
'base': 1 + 1e-10,
'power': 86289369
}
%timeit power(sampleInput) # 3.72 s ± 35 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
%timeit powerWithCaching(sampleInput) # 238 ms ± 482 µs without dax
%timeit powerWithCaching(sampleInput) # 5.43 ms with dax
SetUp
from dynamocacher.cacher import Cacher
class Cache(Cacher):
class Meta:
table_name = 'dynamoCache'
region = 'us-east-1'
aws_access_key_id = USER
aws_secret_access_key = PW
billing_mode= 'PAY_PER_REQUEST'
dax_read_endpoints = ['....:8111'] # optional dax, note that dax will speed up dynamodb significantly
dax_write_endpoints = ['....:8111']# optional dax, note that dax will speed up dynamodb significantly
createTable
Cache.create_table()
basic
addCache
sampleInput = {
'query': 'testQuery',
'fruit': 'strawberry'
}
Cache.addCache(
input = sampleInput,
output = sampleReturn
)
getCache
output = Cache.getCache(sampleInput, timeout = 3600) # ignore cache older than 3600 seconds
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
dynamocacher-0.0.13.tar.gz
(3.5 kB
view hashes)
Built Distribution
Close
Hashes for dynamocacher-0.0.13-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f4ec7235e9899a10562f395e9836db553fef97a61918cf958a4ce9587c94cc84 |
|
MD5 | eaa6cd03da3c3acb9c89035c76ed7ef3 |
|
BLAKE2b-256 | 6494a15a110145a5cdf801511a3dfad3db6ffee3e3c05df4f7862c0021ea9929 |