django-cachalot 0.7.0

Caches your Django ORM queries and automatically invalidates them.

Caches your Django ORM queries and automatically invalidates them.

Project status:

Currently in beta, do not use in production

Quick start

Requirements

• Django 1.6 or 1.7

• Python 2.6, 2.7, 3.2, 3.3, or 3.4

• django-redis, memcached (or locmem, but it’s not shared between processes, so don’t use it with RQ or Celery)

• PostgreSQL, MySQL or SQLite

Usage

1. pip install django-cachalot

2. Add 'cachalot', to your INSTALLED_APPS

3. Enjoy!

Settings

Setting	Default value	Description
CACHALOT_ENABLED	True	If set to False, disables SQL caching but keeps invalidating to avoid stale cache
CACHALOT_CACHE	'default'	Name of the cache from CACHES used by django-cachalot

These settings can be changed whenever you want. You have to use cachalot_settings as a context manager, a decorator, or simply by changing its attributes:

from cachalot import cachalot_settings

with cachalot_settings(CACHALOT_ENABLED=False):
    # SQL queries are not cached in this block

@cachalot_settings(CACHALOT_CACHE='another_cache')
def your_function():
    # What’s in this function uses another cache

# Globally disables SQL caching until you set it back to True
cachalot_settings.CACHALOT_ENABLED = False

In tests, you can use Django’s testing tools as well as cachalot_settings. The only difference is that you can’t use cachalot_settings to decorate a class.

Limits

Locmem

Locmem is a just a dict stored in a single Python process. It’s not shared between processes, so don’t use django-cachalot in a multi-processes project, if you use RQ or Celery for instance.

QuerySet.extra

Django-cachalot doesn’t cache queries it can’t reliably invalidate. If a SQL query or a part of it is written in pure SQL, it won’t be cached. That’s why QuerySet.extra with select or where arguments, Model.objects.raw(…), & cursor.execute(…) queries are not cached.

Bug reports, questions, discussion, new features

• If you spotted a bug, please file a precise bug report on GitHub

• If you have a question on how django-cachalot works or to simply discuss, go to our Google group.

• If you want to add a feature:

  • if you have an idea on how to implement it, you can fork the project and send a pull request, but please open an issue first, because someone else could already be working on it

  • if you’re sure that it’s a must-have feature, open an issue

  • if it’s just a vague idea, please ask on google groups before

How django-cachalot works

(If you don’t care/understand, just pretend it’s magic)

Reverse engineering

It’s a lot of Django reverse engineering combined with a strong test suite. Such a test suite is crucial for a reverse engineering project. If some important part of Django changes and breaks the expected behaviour, you can be sure that the test suite will fail.

Monkey patching

django-cachalot modifies Django in place during execution to add a caching tool just before SQL queries are executed. We detect which cache keys must be removed when some data is created/changed/deleted on the database.

What still needs to be done

For version 1.0

• Handle multiple databases

• Write tests for multi-table inheritance

• Add invalidation on migrations in Django 1.7 (& South?)

In a more distant future

• Add a setting to choose if we cache QuerySet.order_by('?')

• Use connection.introspection.table_names() to detect which tables are implied in a QuerySet.extra

Legacy

This work is highly inspired of johnny-cache, another easy-to-use ORM caching tool! It’s working with Django <= 1.5. I used it in production during 3 years, it’s an excellent module!

Unfortunately, we failed to make it migrate to Django 1.6 (I was involved). It was mostly because of the transaction system that was entirely refactored.

I also noticed a few advanced invalidation issues when using QuerySet.extra and some complex cases implying multi-table inheritance and related ManyToManyField.