django-zeal 1.1.0

Detect N+1s in your Django app

django-zeal

Catch N+1 queries in your Django project.

🔥 Battle-tested at Cinder

Features

• Detects N+1s from missing prefetches and from use of .defer()/.only()
• Friendly error messages like N+1 detected on User.followers at myapp/views.py:25 in get_user
• Configurable thresholds
• Allow-list
• Well-tested
• No dependencies

Acknowledgements

This library draws heavily from jmcarp's nplusone. It's not a fork, but a lot of the central concepts and initial code came from nplusone.

Installation

First:

pip install django-zeal

Then, add zeal to your INSTALLED_APPS and MIDDLEWARE. You probably don't want to run it in production: I haven't profiled it but it will have a performance impact.

if DEBUG:
    INSTALLED_APPS.append("zeal")
    MIDDLEWARE.append("zeal.middleware.zeal_middleware")

This will detect N+1s that happen in web requests. To catch N+1s in more places, read on!

Celery

If you use Celery, you can configure this using signals:

from celery.signals import task_prerun, task_postrun
from zeal import setup, teardown
from django.conf import settings

@task_prerun.connect()
def setup_zeal(*args, **kwargs):
    setup()

@task_postrun.connect()
def teardown_zeal(*args, **kwargs):
    teardown()

Tests

Django runs tests with DEBUG=False, so to run zeal in your tests, you'll first need to ensure it's added to your INSTALLED_APPS and MIDDLEWARE. You could do something like:

import sys

TEST = "test" in sys.argv
if DEBUG or TEST:
    INSTALLED_APPS.append("zeal")
    MIDDLEWARE.append("zeal.middleware.zeal_middleware")

This will enable zeal in any tests that go through your middleware. If you want to enable it in all tests, you need to do a bit more work.

If you use pytest, use a fixture in your conftest.py:

import pytest
from zeal import zeal_context

@pytest.fixture(scope="function", autouse=True)
def use_zeal():
    with zeal_context():
        yield

If you use unittest, add a custom test runner:

# In e.g. `myapp/testing/test_runners.py`
from zeal import setup as zeal_setup, teardown as zeal_teardown
from django.test.runner import DiscoverRunner
from unittest.runner import TextTestResult

class ZealTestResult(TextTestResult):
    def startTest(self, test):
        zeal_setup()
        return super().startTest(test)

    def addError(self, test, err) -> None:
        zeal_teardown()
        return super().addError(test, err)

    def addFailure(self, test, err) -> None:
        zeal_teardown()
        return super().addFailure(test, err)

    def addSuccess(self, test):
        zeal_teardown()
        return super().addSuccess(test)

class ZealTestRunner(DiscoverRunner):
    def get_resultclass(self):
        return ZealTestResult


# And in your settings:
TEST_RUNNER = (
    "myapp.testing.test_runners.ZealTestRunner"
)

Generic setup

If you also want to detect N+1s in other places not covered here, you can use the setup and teardown functions, or the zeal_context context manager:

from zeal import setup, teardown, zeal_context


def foo():
    setup()
    try:
        # your code goes here
    finally:
        teardown()


@zeal_context()
def bar():
    # your code goes here


def baz():
    with zeal_context():
        # your code goes here

Configuration

By default, any issues detected by zeal will raise a ZealError. If you'd rather log any detected N+1s, you can set:

ZEAL_RAISE = False

N+1s will be reported when the same query is executed twice. To configure this threshold, set the following in your Django settings.

ZEAL_NPLUSONE_THRESHOLD = 3

To handle false positives, you can temporarily disable zeal in parts of your code using a context manager:

from zeal import zeal_ignore

with zeal_ignore():
    # code in this block will not log/raise zeal errors

If you only want to ignore a specific N+1, you can pass in a list of models/fields to ignore:

with zeal_ignore([{"model": "polls.Question", "field": "options"}]):
    # code in this block will ignore N+1s on Question.options

Finally, if you want to ignore N+1 alerts from a specific model/field globally, you can add it to your settings:

ZEAL_ALLOWLIST = [
    {"model": "polls.Question", "field": "options"},

    # you can use fnmatch syntax in the model/field, too
    {"model": "polls.*", "field": "options"},

    # if you don't pass in a field, all N+1s arising from the model will be ignored
    {"model": "polls.Question"},
]

Comparison to nplusone

zeal borrows heavily from nplusone, but has some differences:

• zeal also detects N+1 caused by using .only() and .defer()
• it lets you configure your own threshold for what constitutes an N+1
• it has slightly more helpful error messages that tell you where the N+1 occurred
• nplusone patches the Django ORM even in production when it's not enabled. zeal does not!
• nplusone appears to be abandoned at this point.
• however, zeal only works with Django, whereas nplusone can also be used with SQLAlchemy.

Contributing

1. First, install uv.
2. Create a virtual env using uv venv and activate it with source .venv/bin/activate.
3. Run make install to install dev dependencies.
4. To run tests, run make test.