drf-serializer-prefetch 1.0.4

An automatic prefetcher for django-rest-framework.

drf-serializer-prefetch

An automatic prefetcher that looks at the serializer fields and determines what needs to be prefetched accordingly.

Installation

To install, call pip install drf-serializer-prefetch.

Usage

In its most simple form, the serializer prefetch can be used by simply adding PrefetchingSerializerMixin to the class definition of the model serializer you want to allow automatic prefetching for like so:

from rest_framework import serializers
from serializer_prefetch import PrefetchingSerializerMixin


class SomeSerializer(PrefetchingSerializerMixin, serializer.ModelSerializer):
    class Meta:
        model = SomeModel
        fields = (
            'some',
            'fields',
            'are',
            'defined',
        )

Using it like this, the Prefetching Serializer will be able to see what fields will be returned and prefetch them accordingly.

However, SerializerMethodFields are an issue for the Prefetching Serializer, as those can make calls that are only done at run time. The Prefetching Serializer does not make any kind of file analysis, and as such cannot know what will happen in those methods. If you are calling a related model in that function, you can define either select_related or prefetch_related on the model. The fields defined here will also be taken into account. As for usual with Django, you can go as deep as you need here. For example, you could add user, or you could add user__pizza_set, if you were to loop over the user's pizzas.

For example:

from rest_framework import serializers
from serializer_prefetch import PrefetchingSerializerMixin


class SomeSerializer(PrefetchingSerializerMixin, serializer.ModelSerializer):
    select_related = ('other_model',)

    other_field = serializers.SerializerMethodField()

    def get_other_field(self, obj):
        return obj.other_model.other_field

    class Meta:
        model = SomeModel
        fields = (
            'some',
            'fields',
            'are',
            'defined',
        )

If you need to add some logic to when the prefetching should happen, you can also define get_select_related or get_prefetch_related:

from rest_framework import serializers
from serializer_prefetch import PrefetchingSerializerMixin


class SomeSerializer(PrefetchingSerializerMixin, serializer.ModelSerializer):
    def get_select_related(self):
        return ('other_model',)

    other_field = serializers.SerializerMethodField()

    def get_other_field(self, obj):
        return obj.other_model.other_field

    class Meta:
        model = SomeModel
        fields = (
            'some',
            'fields',
            'are',
            'defined',
        )

Note that because this is run before the queryset has been split into individual models, we cannot use object logic here. If you need a field for some models only, you can either fetch it for all of them, or fetch it for none of them.

Sometimes, other serializers can be called in a SerializerMethodField. For this, you can either define additional_serializers like so:

from rest_framework import serializers
from serializer_prefetch import PrefetchingSerializerMixin


class SomeSerializer(PrefetchingSerializerMixin, serializer.ModelSerializer):
    additional_serializers = (
        {
            'relation_and_field': 'other_model',
            'serializer': OtherModelSerializer(),
        },
    )

    other_field = serializers.SerializerMethodField()

    def get_other_field(self, obj):
        return OtherModelSerializer(obj.other_model, auto_prefetch=False)

    class Meta:
        model = SomeModel
        fields = (
            'some',
            'fields',
            'are',
            'defined',
        )

Notice the part auto_prefetch=False in the OtherModelSerializer call. This is because drf-auto-prefetch uses parents to know if the serializer is at the higher-most level, so that the fetching is only done once. In this case however, the serializer has no parent, but the prefetching is already done by SomeSerializer, so we do not want to do it again. We can tell it to not prefetch by setting auto_prefetch to False.

As for select_related and prefetch_related, you can define get_additional_serializers instead:

from rest_framework import serializers
from serializer_prefetch import PrefetchingSerializerMixin


class SomeSerializer(PrefetchingSerializerMixin, serializer.ModelSerializer):
    def get_additional_serializers:
        return (
            {
                'relation_and_field': 'other_model',
                'serializer': OtherModelSerializer(),
            },
        )

    other_field = serializers.SerializerMethodField()

    def get_other_field(self, obj):
        return OtherModelSerializer(obj.other_model, auto_prefetch=False)

    class Meta:
        model = SomeModel
        fields = (
            'some',
            'fields',
            'are',
            'defined',
        )

This can be useful to avoid circular dependencies or to add some logic to the prefetching.

Special cases

There are a few situations where you might want to be able to customize the behaviour more. Here are some of the ways you can tweak the Prefetching Serializer to fit the needs of your project.

Adding logic to the PrefetchingListSerializer

If you had a ListSerializer for you Serializer, you will most likely want to keep its logic. This can be done by simply inheriting from PrefetchingListSerializer in that ListSerializer rather than inheriting from serializers.ListSerializer. Note that if you do not do this, but add the PrefetchingSerializerMixin to the main Serializer, you will get an error saying that the list_serializer_class must inherit from PrefetchingListSerializer to be used with the PrefetchingSerializerMixin. This is because the behaviour of the prefetching depends on it.

Adding compatibility with django-zen-queries or other libraries

The goal of this library is to make is easy to not have to think about prefetching. However, this comes with the potential danger of not thinking enough about prefetching. To avoid discovering issues only once in production, you can extend the PrefetchingListSerializer and PrefetchingSerializerMixin by defining a new mixin that will be inherited by both. In this mixin, you can override the queryset_after_prefetch and call_to_representation methods to add some behaviour right after the prefetching has been done on the queryset and right before or after calling super().to_representation(instance) on the serializer. For django-zen-queries, it looks something like this:

from contextlib import nullcontext
from django.conf import settings
import serializer_prefetch
from zen_queries import fetch, queries_disabled


class ZenQueriesPrefetchingMixin:
    def queryset_after_prefetch(self, queryset):
        fetch(queryset)
        return queryset

    def call_to_representation(self, instance):
        with queries_disabled() if settings.DEBUG else nullcontext():
            return super().to_representation(instance)


class PrefetchingListSerializer(
    ZenQueriesPrefetchingMixin, 
    serializer_prefetch.PrefetchingListSerializer
):
    pass


class PrefetchingSerializerMixin(
    ZenQueriesPrefetchingMixin,
    serializer_prefetch.PrefetchingSerializerMixin
):
    default_list_serializer_class = PrefetchingListSerializer

The same logic can be applied to other libraries that could need to interact with the queryset between the time it is prefetched and the time it is split into models. Note that queryset_after_prefetch is only ever called if the instance passed to the ListSerializer is a Queryset, so you do not need to worry about checking for the type. Similarly, call_to_representation is only called if some prefetching was done. If none was done, either because it was not the furthermost parent or because auto_prefetch has been set to False, then this method will not be called. If you need a method to always be called, you can define a to_representation method in the new PrefetchingListSerializer you defined. Do not forget to call the super method!

Important note

While the Prefetching Serializer does work on regular django-rest-framework serializers, it is really intended to be used with their ModelSerializer. If you do use it with a regular Serializer, you will need to define select_related and prefetch_related for all the fields used. Note that this is still better than having the logic in get_queryset in the viewset, as it is kept with its own serializer and will be used properly if the serializer is nested.