Subscriptions for Django REST Framework over Websockets.
Project description
Django REST Live
django-rest-live
adds real-time subscriptions over websockets to Django REST Framework
by leveraging websocket support provided by Django Channels.
Contents
Inspiration and Goals
The goal of this project is to enable realtime subscriptions without requiring any boilerplate or changing
any existing REST Framework views or serializers.
django-rest-live
took initial inspiration from this article by Kit La Touche.
Dependencies
- Django (3.1 and up)
- Django Channels (2.x, 3.0 not yet supported)
- Django REST Framework
channels_redis
for channel layer support in production.
Installation
If your project already uses REST framework, but this is the first realtime component, then make sure to install and properly configure Django Channels before continuing.
You can find details in the Channels documentation.
- Add
rest_live
to yourINSTALLED_APPS
INSTALLED_APPS = [
# Any other django apps
"rest_framework",
"channels",
"rest_live",
]
- Add
rest_live.consumers.SubscriptionConsumer
to your websocket routing. Feel' free to choose any URL path, here we've chosen/ws/subscribe/
.
from channels.auth import AuthMiddlewareStack
from channels.routing import ProtocolTypeRouter, URLRouter
from django.urls import path
from rest_live.consumers import SubscriptionConsumer
websockets = AuthMiddlewareStack(
URLRouter([
path("ws/subscribe/", SubscriptionConsumer, name="subscriptions"),
"Other routing here...",
])
)
application = ProtocolTypeRouter({
"websocket": websockets
})
That's it! You're now ready to configure and use django-rest-live
.
Usage
These docs will use an example to-do app called todolist
with the following models and serializers:
# todolist/models.py
from django.db import models
class List(models.Model):
name = models.CharField(max_length=64)
class Task(models.Model):
text = models.CharField(max_length=140)
done = models.BooleanField(default=False)
list = models.ForeignKey("List", on_delete=models.CASCADE)
# todolist/serializers.py
from rest_framework import serializers
class TaskSerializer(serializers.ModelSerializer):
class Meta:
model = Task
fields = ["id", "text", "done"]
class TodoListSerializer(serializers.ModelSerializer):
tasks = TaskSerializer(many=True, read_only=True)
class Meta:
model = List
fields = ["id", "name", "tasks"]
Basic Usage
An important thing to remember about the django-rest-live
package is that its sole purpose is sending updates to clients
over websocket connections. Clients should still use normal REST framework endpoints generated by ViewSets and views
to get initial data to populate a page, as well as any write-driven behavior (POST
, PATCH
, PUT
, DELETE
).
django-rest-live
gets rid of the need for periodic GET requests for updated data.
Server-Side
In order to tell django-rest-live
that you'd like to allow clients to subscribe to updates for a specific model, the package
provides the @subscribable
class decorator. This decorator is meant to be applied to ModelSerializer
subclasses,
so that the package can register both which models to allow subscriptions to as well as how those models should be
serialized when being sent to the client. To enable clients to subscribe to updates to individual to-dos, all you need
to do is apply the decorator to the TodoSerializer
:
# todolist/serializers.py
from rest_live.decorators import subscribable
...
@subscribable()
class TaskSerializer(serializers.ModelSerializer):
...
Client-Side
Subscribing to model updates from a client requires opening a WebSocket
connection to the URL you specified during setup. In our example case, that URL is /ws/subscribe/
. After the connection
is established, send a JSON message (using JSON.stringify()
) in this format:
{
"model": "todolist.Task",
"value": 1
}
The model label should be in Django's standard app.modelname
format. value
field here is set to the value for
the primary key for the model instance
we're subscribing to. This is generally the value of the id
field, but is equivalent to querying
for Task.objects.filter(pk=<value>)
.
The example message above would subscribe to updates for the todo task with an primary key of 1. As mentioned above, the client should make a GET request to get the entire list, with all its tasks and their associated IDs, to figure out which IDs to subscribe to.
When the Task with primary key 1
updates, a message in this format will be sent over the websocket:
{
"model": "test_app.Todo",
"instance": {"id": 1, "text": "test", "done": true},
"action": "UPDATED",
"group_key_value": 1
}
Valid action
values are UPDATED
, CREATED
, and DELETED
.
group_key_value
might seem erroneous in this example, but is useful for group subscriptions, described in the next
section.
Advanced Usage
Subscribe to groups
As mentioned above, subscriptions are grouped by the primary key by default: you send one message to the websocket to get updates for a single Task with a given primary key. But in the todo list example, you'd generally be interested in an entire list of tasks, including being notified of any tasks which have been created since the page was first loaded.
Rather than subscribe all tasks individually, you want to subscribe to a list: an entire group of tasks.
This is where group keys come in. Pass in the group_key
you'd like to group tasks by
to the @subscribable
decorator to register subscriptions for an entire list:
# todolist/serializers.py
from rest_live.decorators import subscribable
...
@subscribable(group_key="list_id")
class TaskSerializer(serializers.ModelSerializer):
...
On the client side, we now have to specify the group key property
we are subscribing to. In this case, the list_id
of the list you'd like to get updates from:
{
"model": "todolist.Task",
"property": "list_id",
"value": 1
}
This will subscribe you to updates for all Tasks in the list which has ID 1.
What's important to remember here is that while the field is defined as a ForeignKey
called list
on the model,
the underlying integer field in the database that links together Tasks and Lists is called list_id
. More generally,
<fieldname>_id
for any related fieldname on the model.
The subscribable
decorator can be stacked. If you want to enable subscriptions by both list_id
for entire lists and
pk
for individual tasks, add two decorators:
# todolist/serializers.py
from rest_live.decorators import subscribable
...
@subscribable()
@subscribable(group_key="list_id")
class TaskSerializer(serializers.ModelSerializer):
...
Just note that clients which subscribe to list updates and individual pk updates will receive two messages when a task updates.
Permissions
The @subscribable
decorator also takes in a parameter called check_permission
. This is a function which takes in
a User and a model instance and determines whether or not the given user can access the given model. To make sure
users can only subscribe to lists when they are logged in, this code would suffice:
# todolist/serializers.py
from rest_live.decorators import subscribable
def has_auth(user, instance):
return user.is_authenticated
@subscribable(group_key="list_id", check_permission=has_auth)
class TaskSerializer(serializers.ModelSerializer):
...
Conditional Serializer Pattern
A common pattern in Django REST Framework is showing users different serializers based on their authentication status
by overloading get_serializer_class()
in a ViewSet
. This pattern can be mirrored in django-rest-live
using the check_permission
callback. Let's say that for our to-do app, un-authenticated users can view tasks, but
cannot see if they're completed. Users can subscribe with the proper serializer with the following serializers.py
:
# todolist/serializers.py
from rest_framework import serializers
from rest_live.decorators import subscribable
def has_auth(user, instance):
return user.is_authenticated
def has_no_auth(user, instance):
return not has_auth(user, instance)
@subscribable(group_key="list_id", check_permission=has_auth)
class AuthedTaskSerializer(serializers.ModelSerializer):
class Meta:
model = Task
fields = ["id", "text", "done"]
@subscribable(group_key="list_id", check_permission=has_no_auth)
class NoAuthTaskSerializer(serializers.ModelSerializer):
class Meta:
model = Task
fields = ["id", "text"]
Clients in both situations would send the same subscribe request, but would receive different model instances depending on their authentication status.
Testing
As of Django 3.1, you can write asynchronous tests in Django TestCase
s. You can set up a test case by following
the snippet below, adapted from the channels
documentation:
from django.test import TransactionTestCase
from channels.testing import WebsocketCommunicator
from app.routing import application # Replace this line with the import to your ASGI router.
from channels.db import database_sync_to_async
class MyTests(TransactionTestCase):
async def test_subscribe(self):
client = WebsocketCommunicator(application, "/ws/subscribe/")
await client.send_json_to(
{
"model": "app.Model",
"property": "pk",
"value": "1",
}
)
self.assertTrue(await client.receive_nothing())
await database_sync_to_async(Model.objects.create)(...)
response = await client.receive_json_from()
self.assertEqual(response, {
"model": "app.Model",
"instance": { "": "..." },
"action": "CREATED",
"group_key_value": "...",
})
await client.disconnect()
Since REST Live makes use of the database for its functionality, make sure to use django.test.TransactionTestCase
instead of django.test.TestCase
so that database connections within the async test functions get cleaned up approprately.
Remember to wrap all ORM calls in the database_sync_to_async
decorator as demonstrated in the above example. The ORM
is still fully synchronous, and the regular sync_to_async
decorator does not properly clean up connections!
setUp and tearDown
The normal TestCase.setUp
and TestCase.tearDown
methods run in different threads from the actual test itself,
and so they don't work for creating async objects like WebsocketCommunicator
. REST Live comes with a decorator called
@async_test
which will enable test cases to define lifecycle methods asyncSetUp()
and asyncTearDown()
to
run certain code before and after every test case decorated with @async_test
. Here is an example:
...
from rest_live.testing import async_test
class MyTests(TransactionTestCase):
async def asyncSetUp(self):
self.client = WebsocketCommunicator(application, "/ws/subscribe/")
async def asyncTearDown(self):
await self.client.disconnect()
@async_test
async def test_subscribe(self):
... # a new connection has been opened and is accessible in `self.client`
Authentication
Authentication in unit tests for django channels is a bit tricky, but the utility that rest_live
provides
is based on this github issue comment.
The WebsocketCommunicator
class can take HTTP headers as part of its constructor. In order to open a connection
as a logged-in user, you can use rest_live.testing.get_headers_for_user
:
from rest_live.testing import get_headers_for_user
user = await database_sync_to_async(User.objects.create_user)(username="test")
headers = await get_headers_for_user(user)
client = WebsocketCommunicator(application, "/ws/subscribe/", headers)
All permissions checks should work as expected after opening a connection this way.
Limitations
This package works by listening in on model lifecycle events sent off by Django's signal dispatcher.
Specifically, the post_save
and post_delete
signals. This means that django-rest-live
can only pick up changes that Django knows about. Bulk operations, like filter().update()
, bulk_create
and bulk_delete
do not trigger Django's lifecycle signals, so updates will not be sent.
TODO
- Permissions
- Conditional Serializers
- Permissions helpers for DRF
Permission
classes - Error handling and reporting
- Expand related fields from
field
tofield_id
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for django_rest_live-0.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 19aedf29490022b3b866ca939d32cf12e7b60562c53d9c98bfbd9574f8b714ac |
|
MD5 | 212c7945447ba9f157f84a6ab0761726 |
|
BLAKE2b-256 | 0ab21278d4aad0ec6c7c66f2b05c6a025adb75f5689a488113707fcb6a6bf169 |