Skip to main content

Some of Hubspot's python cache utils, namely generational caching

Project description

[![Build Status](](

A python implementation of [generational caching](

Three main interfaces are provided:

- Direct methods `gen_cache.get`, `gen_cache.set`, `gen_cache.invalidate`, and `gen_cache.delete`
- A `CustomUseGenCache` instance
- A function decorator `@gen_cache.wrap`

## Direct methods:

from hscacheutils.generational_cache import gen_cache

html = gen_cache.get(('nav', 'nav_portal:user_id'), user_id=1)
gen_cache.set('<html>', ('nav', 'nav_portal:user_id'), user_id=1)
gen_cache.invalidate(('nav', 'nav_portal:user_id'), user_id=1)

## A `CustomUseGenCache` instance

The same as the direct methods, but creates an object so that you con't have to keep on passing in the generation names every single time.

custom_cache = CustomUseGenCache([

blog_id = 17
user_id = 123
key = random.randint(1, 20000000)

val = custom_cache.get(blog_id=blog_id, user_id=123, cache_key=key)
custom_cache.set(value=first_val, blog_id=blog_id, user_id=123, cache_key=key)
custom_cache.delete(blog_id=blog_id, user_id=123, cache_key=key)

## The `@gen_cache.wrap` decorator

It can be applied to function, method or classmethod. It is mostly similar to gen_cache.get, but with some additional magic to make your life easier.

### Magic #1: (true for all 3 interfaces)

The contents of "value-based" generations are automatically pulled from the arguments in wrapped function. Eg.

@gen_cache.wrap('project_name', 'foo_per_user_id:user_id')
def foobar(user_id):

foobar(53) # Uses 'project_name' and 'foo_per_user_id:53' as generations
foobar(999) # Uses 'project_name' and 'foo_per_user_id:999' as generations

# So when invalidating like so...
gen_cache.invalidate("for_per_user_id:user_id", user_id=999)

foobar(53) # This is still cached
foobar(999) # This has been invalidated

So the when foobar is called the ':user_id' part of the value-based generation looks for any
argument named "user_id", then takes its value to create a generation such as "for_per_user_id:53".
This means that the "for_per_user_id" generation is only invalidated on a per-portal basis

### Magic #2: (true for all 3 interfaces)

All of the arguments (not used in value-based generations described above) are automatically appended to the cache key. Eg.

def foobar(something, another=False):

# XXX represents the current counter value of the 'whatever' generation

foobar(1) # Uses a cache key roughly like: "whatever:XXX [1]{another=False}"
foobar(2) # Uses a cache key roughly like: "whatever:XXX [2]{another=False}"
foobar(2, another=True) # Uses a cache key roughly like: "whatever:XXX [2]{another=True}"

# So when invalidating like so...

foobar(1) # This has been invalidated
foobar(2) # This has been invalidated
foobar(2, another=True) # This has been invalidated

If you don't what this behavior for one or more arguments, make sure to put the name of that
argument(s) in the "exclude" option (see below).

### Magic #3: (only true for the decorator)

The cache key will automatically include the current module name, function name, and
line number. So when this function moves to a different file, is renamed, or moves up or down a
few lines, the cache will automatically be invalidated.

(Note, I'm not sure this file/function name magic is worth keeping)


The wrapped callable gets `invalidate` methods. Call `invalidate` with
same arguments as function and the result for these arguments will be


timeout=3600 (defaults to None) is the number of seconds before this cache should expire

exclude=[...] (defaults to empty list) is all the arguments you do not want to automaticaaly be
a included in the cache key.

log_misses=True (False by default) will print out some debugging into on every cache miss

ignore_locally=True (False by default) will disable this caching when ENV == 'local'

### REAL `gen_cache.wraps` cache key example

^ ^ ^ ^ ^ ^ ^
| | | | | | |
prefix module name func name line # | generation & current counter value |
| |
non-excluded positional args non-excluded keyword args

Gotcha #1: Be careful to use either "self" or "cls" as the first argument name when wrapping
methods and classmethods. This code relies on those names (see _func_type) to automatically
chop off the first argument from the cache key.

_Note: based on (and built re-using) django-cache-utils._

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

hscacheutils-0.1.8.tar.gz (10.1 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page