Implements Google's partial response in Django RestFramework (Fork from Zapier's package)
Project description
Overview
Forked from https://github.com/zapier/django-rest-framework-jsonmask.
Implements Google's Partial Response in Django REST Framework.
Requirements
- Python (3.7, 3.8, 3.9)
- Django (2.2, 3.0, 3.1, 3.2, 4.0, 4.1, 4)
- Django REST framework (3.10, 3.11, 3.12, 3.13, 3.14)
Installation
Install using pip
...
$ pip install drf-jsonmask
Example
Most DRF addons that support ?fields=
-style data pruning do so purely at the serializaton layer. Many hydrate full ORM objects, including all of their verbose relationships, and then cut unwanted data immediately before JSON serialization. Any unwanted related data is still fetched from the database and hydrated into Django ORM objects, which severely undermines the usefulness of field pruning.
drf_jsonmask
aims to do one better by allowing developers to declaratively augment their queryset in direct relation to individual requests. Under this pattern, you only declare the base queryset and any universal relationships on your ViewSet.queryset, leaving all additional enhancements as runtime opt-ins.
To use drf_jsonmask
, first include its ViewSet and Serializer mixins in your code where appropriate. The following examples are taken from the mini-project used in this library's own unit tests.
# api/views.py
from drf_jsonmask.views import OptimizedQuerySetMixin
class TicketViewSet(OptimizedQuerySetMixin, viewsets.ReadOnlyModelViewSet):
# Normally, for optimal performance, you would apply the `select_related('author')`
# call to the base queryset, but that is no longer desireable for data relationships
# that your frontend may stop asking for.
queryset = Ticket.objects.all()
serializer_class = TicketSerializer
# Data-predicate declaration is optional, but encouraged. This
# is where the library really shines!
@data_predicate('author')
def load_author(self, queryset):
return queryset.select_related('author')
# api/serializers.py
from drf_jsonmask.serializers import FieldsListSerializerMixin
class TicketSerializer(FieldsListSerializerMixin, serializers.ModelSerializer):
# Aside from the mixin, everything else is exactly like normal
author = UserSerializer()
class Meta:
models = my_module.models.Ticket
fields = ('id', 'title', 'body', 'author',)
You have now set up your API to skip unnecessary joins (and possibly prefetches), unless the requesting client requires that data. Let's consider a few hypothetical requests and the responses they will each receive. (For brevity, in each of these examples, I will pretend pagination is turned off.)
GET /api/tickets/
200 OK
[
{
"id": 1,
"title": "This is a ticket",
"body": "This is its text",
"author": {
"id": 5,
"username": "HomerSimpson",
}
}
]
Because no ?fields
querystring parameter was provided, author records were still loaded and serialized like normal.
Note:
drf_jsonmask
treats all requests that lack any field definition as if all possible data is requested, and thus executes all data predicates. In the above example,author
data was loaded viaselected_related('author')
, and not N+1 queries.
GET /api/tickets/?fields=id,title,body
200 OK
[
{
"id": 1,
"title": "This is a ticket",
"body": "This is its text"
}
]
In this example, since author
was not specified, it was not only not returned in the response payload - it was never queried for or serialized in the first place.
GET /api/tickets/?fields=id,title,body,author/username
200 OK
[
{
"id": 1,
"title": "This is a ticket",
"body": "This is its text",
"author": {
"username": "HomerSimpson",
}
}
]
In this example, author
data was loaded via the ?fields
declaration, but no unwanted keys will appear in the response.
Nested Relationships
This is all good and fun, but what if author
has rarely used but expensive relationships, too? drf_jsonmask
supports this, via the exact same mechanisms spelled out above, though sometimes a little extra attention to detail can be important. Let's now imagine that AuthorSerializer
looks like this:
class AuthorSerializer(FieldsListSerializerMixin, serializers.ModelSerializer):
accounts = AccountSerializer(many=True)
class Meta:
model = settings.AUTH_USER_MODEL
fields = ('id', 'username', 'email', 'photo', 'accounts', ...)
Naturally, if accounts
is sensitive, internal data, you simply might not use this serializer for external API consumption. Of course, that would solve your problem about how to decide whether to serialize accounts
data -- the supplied serializer would know nothing about that field! But, let's pretend that in our case, accounts
is safe for public consumption, and some ticketing API calls require it for triaging purposes, whereas others do not. In such a situation, we'll redefine our ViewSet like so:
class TicketViewSet(OptimizedQuerySetMixin, viewsets.ReadOnlyModelViewSet):
queryset = Ticket.objects.all()
serializer_class = TicketSerializer
@data_predicate('author')
def load_author(self, queryset):
return queryset.select_related('author')
# Add this extra data_predicate with prefetches `accounts` if and only if
# the requests promises to use that information
@data_predicate('author.accounts')
def load_author_with_accounts(self, queryset):
return queryset.select_related('author').prefetch_related('author__accounts')
Now, it is up to the client to decide which of the following options (or anything else imaginable) is most appropriate:
# Includes specified local fields plus all author fields and relationships
GET /api/tickets/?fields=id,title,author
200 OK
[
{
"id": 1,
"title": "This is a ticket",
"author": {
"id": 5,
"username": "HomerSimpson",
"accounts": [
{"all_fields": "will_be_present"}
]
}
}
]
or
# Includes specified local fields plus specified author fields and relationships
GET /api/tickets/?fields=id,title,author(username,photo)
200 OK
[
{
"id": 1,
"title": "This is a ticket",
"author": {
"username": "HomerSimpson",
"photo": "image_url"
}
}
]
or
# Includes specified local fields plus specified author fields and relationships plus specified accounts fields and relationships
GET /api/tickets/?fields=id,title,author(id,accounts(id,type_of,date))
200 OK
[
{
"id": 1,
"title": "This is a ticket",
"author": {
"id": 5,
"accounts": [
{
"id": 8001,
"type_of": "business",
"date": "2018-01-01T12:00:00Z"
},
{
"id": 6500,
"type_of": "trial",
"date": "2017-06-01T12:00:00Z"
}
]
}
}
]
In short, know that as long as the entire chain of Serializers implements the FieldsListSerializerMixin
, arbitrarily deep nesting of ?fields
declarations will be honored. However, in practice, because relationships are expensive to hydrate, you will probably want to limit that information and control what data you actually load using the @data_predicate
decorator on ViewSet methods.
Testing
$ make tests
or keep them running on change:
$ make watch
You can also use the excellent tox testing tool to run the tests against all supported versions of Python and Django. Install tox globally, and then simply run:
$ tox
Documentation
$ make docs
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.