Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Some of Hubspot's python cache utils, namely generational caching

Project Description
[![Build Status](https://travis-ci.org/HubSpot/hscacheutils.png?branch=master)](https://travis-ci.org/HubSpot/hscacheutils)

A python implementation of [generational caching](http://www.regexprn.com/2011/06/web-application-caching-strategies_05.html).

Three main interfaces are provided:

- Direct methods `gen_cache.get`, `gen_cache.set`, `gen_cache.invalidate`, and `gen_cache.delete`
- A `CustomUseGenCache` instance
- A function decorator `@gen_cache.wrap`


## Direct methods:

```python
from hscacheutils.generational_cache import gen_cache

html = gen_cache.get(('nav', 'nav_portal:user_id'), user_id=1)
gen_cache.set('<html>', ('nav', 'nav_portal:user_id'), user_id=1)
gen_cache.invalidate(('nav', 'nav_portal:user_id'), user_id=1)
```

## A `CustomUseGenCache` instance

The same as the direct methods, but creates an object so that you con't have to keep on passing in the generation names every single time.

```python
custom_cache = CustomUseGenCache([
'customgenz:user_id',
'customgenz:blog_id'
])

blog_id = 17
user_id = 123
key = random.randint(1, 20000000)

val = custom_cache.get(blog_id=blog_id, user_id=123, cache_key=key)
custom_cache.set(value=first_val, blog_id=blog_id, user_id=123, cache_key=key)
custom_cache.invalidate(user_id=123)
custom_cache.delete(blog_id=blog_id, user_id=123, cache_key=key)
```


## The `@gen_cache.wrap` decorator

It can be applied to function, method or classmethod. It is mostly similar to gen_cache.get, but with some additional magic to make your life easier.


### Magic #1: (true for all 3 interfaces)

The contents of "value-based" generations are automatically pulled from the arguments in wrapped function. Eg.

```python
@gen_cache.wrap('project_name', 'foo_per_user_id:user_id')
def foobar(user_id):
...

foobar(53) # Uses 'project_name' and 'foo_per_user_id:53' as generations
foobar(999) # Uses 'project_name' and 'foo_per_user_id:999' as generations

# So when invalidating like so...
gen_cache.invalidate("for_per_user_id:user_id", user_id=999)

foobar(53) # This is still cached
foobar(999) # This has been invalidated
```

So the when foobar is called the ':user_id' part of the value-based generation looks for any
argument named "user_id", then takes its value to create a generation such as "for_per_user_id:53".
This means that the "for_per_user_id" generation is only invalidated on a per-portal basis


### Magic #2: (true for all 3 interfaces)

All of the arguments (not used in value-based generations described above) are automatically appended to the cache key. Eg.

```python
@gen_cache.wrap('whatever')
def foobar(something, another=False):
...

# XXX represents the current counter value of the 'whatever' generation

foobar(1) # Uses a cache key roughly like: "whatever:XXX [1]{another=False}"
foobar(2) # Uses a cache key roughly like: "whatever:XXX [2]{another=False}"
foobar(2, another=True) # Uses a cache key roughly like: "whatever:XXX [2]{another=True}"

# So when invalidating like so...
gen_cache.invalidate("whatever")

foobar(1) # This has been invalidated
foobar(2) # This has been invalidated
foobar(2, another=True) # This has been invalidated
```

If you don't what this behavior for one or more arguments, make sure to put the name of that
argument(s) in the "exclude" option (see below).


### Magic #3: (only true for the decorator)

The cache key will automatically include the current module name, function name, and
line number. So when this function moves to a different file, is renamed, or moves up or down a
few lines, the cache will automatically be invalidated.

(Note, I'm not sure this file/function name magic is worth keeping)

### EXTRAS

The wrapped callable gets `invalidate` methods. Call `invalidate` with
same arguments as function and the result for these arguments will be
invalidated.

### KEYWORD OPTIONS

timeout=3600 (defaults to None) is the number of seconds before this cache should expire

exclude=[...] (defaults to empty list) is all the arguments you do not want to automaticaaly be
a included in the cache key.

log_misses=True (False by default) will print out some debugging into on every cache miss

ignore_locally=True (False by default) will disable this caching when ENV == 'local'

### REAL `gen_cache.wraps` cache key example

[cached]hsdjango.test.test_generational_cache.func_with_lots_of_args:369(['one','two']{'project':1336056824437339,'foobar':'NOThello','user_id':42})
^ ^ ^ ^ ^ ^ ^
| | | | | | |
prefix module name func name line # | generation & current counter value |
| |
non-excluded positional args non-excluded keyword args




Gotcha #1: Be careful to use either "self" or "cls" as the first argument name when wrapping
methods and classmethods. This code relies on those names (see _func_type) to automatically
chop off the first argument from the cache key.



_Note: based on (and built re-using) django-cache-utils._
Release History

Release History

This version
History Node

0.1.8

History Node

0.1.6

History Node

0.1.5

History Node

0.1.4

History Node

0.1.3

History Node

0.1.2

History Node

0.1.1

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
hscacheutils-0.1.8.tar.gz (10.1 kB) Copy SHA256 Checksum SHA256 Source May 16, 2014

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting