django-query-contract

A query-capture engine for Django, and a pytest plugin over it.

Every statement is recorded with a normalised SQL fingerprint and the call stack that emitted it, through connection.execute_wrapper(). When a query-count assertion fails, the failure gains a paragraph saying which shape repeated and where it came from. When a block runs past the ceiling on Django's own query log, that is reported instead of papered over.

Install

pip install django-query-contract

The pytest plugin loads itself. There is nothing to add to INSTALLED_APPS, and no fixture to request.

It ships no count assertion, on purpose

django_assert_num_queries is the assertion. It is typed, it handles connection= / using= and a custom failure note, and it yields the captured queries. Keep writing it:

def test_listing_is_flat(django_assert_num_queries, authors):
    with django_assert_num_queries(2):
        render_author_list()

When it fails, the failure now carries a diagnosis underneath it:

------------------------------ django-query-contract ------------------------------
13 statements captured: 13 on 'default'.

N+1 -- one statement shape, executed more than once from one call path:
  11 x  from shop/views.py:16 in render_author_list
       SELECT "shop_book"."id", "shop_book"."author_id", "shop_book"."title" FROM "shop_book" WHERE "shop_book"."author_id" = %s
       queries #1, #2, #3, #4, #5, #6, #7, #8, ...
  2 statement(s) were not repeated from any one call path.

N+1 by construction, with nothing to tune

More than one execution with the same normalised SQL and the same call stack is an N+1. No threshold, no confidence score, no rule about lazy loads: the same line ran the same statement again with different data instead of asking once.

Four Python N+1 detectors are dead on PyPI -- nplusone (2018, 1068 stars, and still what the blog posts recommend), django-query-capture, django-nplusone and django-explain. The probable reason is that they classified by rule, so they cried wolf and were removed. A detector nobody disables is a different package from a detector that finds more.

The identity is the whole call stack, not the call site. Two callers of one get_books(author) helper are two defects with two fixes, and a report grouped by call site would name the one line that is fine. The identity does not include the connection, so a loop that queries two databases stays one finding. A statement with no call stack -- everything in a capture rebuilt from a CaptureQueriesContext -- is not grouped at all, because guessing there would manufacture a finding out of a gap in the input.

Nothing fails on a finding. A batched bulk_create is one shape run a hundred times from one line, structurally identical to the defect, so it is reported like any other repetition and costs nobody a red build. That is what keeps this from crying wolf, rather than an exemption list -- which would be the first tunable.

pytest --n-plus-one

lists every finding in the run, worst first, and changes no outcome:

=============================== django-query-contract ===============================
2 N+1 finding(s), most repeated first:
  40 x  from shop/views.py:16 in render_author_list
       in tests/test_views.py::test_author_listing
       SELECT "shop_book"."id", "shop_book"."author_id" FROM "shop_book" WHERE ...
       queries #3, #4, #5, #6, #7, #8, #9, #10, ...

Or read them yourself:

from django_query_contract import QueryCapture, find_n_plus_one

with QueryCapture(using="default") as capture:
    view(request)

for finding in find_n_plus_one(capture):
    print(finding.count, finding.call_site, finding.fingerprint)

The ceiling nobody mentions

assertNumQueries and django_assert_num_queries both count through CaptureQueriesContext, which slices connection.queries between two absolute indices. That log is a deque(maxlen=connection.queries_limit) -- 9000 by default -- so once it rotates the indices no longer point at what they did. Measured against Django 6.1:

The last row is a passing django_assert_max_num_queries(1) around five real queries, and the regime it happens in -- thousands of statements in one block -- is exactly the N+1-at-scale case worth catching.

Capture here rides on execute_wrapper, which has no bound, so this package raises a QueryLogCeilingWarning naming the test, the real count and the number the assertion was handed.

Reading the capture directly

from django_query_contract import QueryCapture

with QueryCapture() as capture:
    render_author_list()

for fingerprint, records in capture.by_fingerprint().items():
    print(len(records), fingerprint)

A QueryRecord carries the statement, its fingerprint, the connection alias and vendor, the parameter count and the call stack. It carries no parameters -- a bulk_create is one execution and ten thousand values, and a runtime reader of this capture has no business holding them -- and no duration, because a performance assertion that mentions milliseconds is a flaky test with extra steps.

QueryCapture.from_capture_context(...) builds one from the CaptureQueriesContext that django_assert_num_queries yields. It is honestly degraded: no call stacks, no parameter counts, and no ceiling, because a count taken from a rotated deque cannot report what it lost.

Where the capture stops

execute_wrapper wraps Django's cursor wrapper, so it sees execute and executemany and nothing else. A statement issued on the raw driver connection is invisible, and so is a driver API that is neither -- psycopg 3's cursor.copy(), for instance. Django's own query log has the same blind spot, so the two agree, and there is a test that pins it rather than a note that assumes it.

Turning it off

[pytest]
query_contract = false
query_contract_stack_depth = 25

or --no-query-contract for one run. query_contract_stack_depth is also what widens the window the N+1 identity is formed from -- see N+1 detection.

Status

Early. The capture engine, the pytest diagnosis, and N+1 by (call stack, fingerprint). The growth assertion, call-site attribution as a public surface, and plan capture come next.

Full documentation: https://artui.github.io/django-query-contract/

License

MIT