Drop-in replacement for django admin default pagination that works fast with huge tables.
Project description
django-admin-cursor-paginator
Drop-in replacement for django admin default pagination that works fast with huge tables.
Prerequisites
- You are using django (>2.0) admin
- Some of your models/tables contains so much data that list page lags
- You don't want to make separate CRM page for them and just need fast and simple solution
- You're ok that you won't see total objects or pages count and will navigate only by +-1 page
- Index must be created for the ordering field (default integer pks work out-of-box)
Installation
All you need is to install it via pip
like that:
pip install django-admin-cursor-paginator
...and extend INSTALLED_APPS
(it is needed only for templates):
INSTALLED_APPS = [
'django.contrib.admin',
# ...
'admin_cursor_paginator',
# ...
]
Usage example
So this app will certainly help you if you're using admin to see some (usually) last inserted entries and may be also want to search for particular objects.
But anyways it's just a regular django admin page.
from admin_cursor_paginator import CursorPaginatorAdmin
class YourModelAdmin(CursorPaginatorAdmin):
cursor_ordering_field = '-created_at' # default is '-pk'
The problem
Default django admin list view makes several queries to show data.
The first one is something like select count(*) from app_model
to calculate pages range
and select * from app_model limit 10 offset 1000
to fetch current page data.
Both of them start to produce performance problems as your table grows.
For example, with >1M entries in postgresql table count(*)
took a few seconds (in our case) because of seq scan.
Limit+offset also performs badly on far pages.
The solution
The solution is to use cursor pagination instead of default one (you can find many standalone packages for it).
It doesn't need count(*)
query at all and performs fast by using filter instead of limit+offset.
Some useful links:
- https://cra.mr/2011/03/08/building-cursors-for-the-disqus-api/
- https://www.django-rest-framework.org/api-guide/pagination/#cursorpagination
- https://hakibenita.com/optimizing-the-django-admin-paginator
Test project
In testproject
folder you can (surprisingly) find test project that uses this app
for one model so that you can see live example and check how it works if you want.
All you need is to install requirements from requirements.txt
in some env,
run ./manage.py runserver
, login as admin:admin to django admin and open Product
list.
Project includes sqlite3 file with some test data and preinstalled debug_toolbar
to observe db queries.
You can play with ordering field or data amount or may be test it in your target db.
Example of how pagination will look like:
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for django-admin-cursor-paginator-0.1.0.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | bdfd71c32dd36fbebc48a39954de965c6ef37577528a927dfbf638b5d2602f73 |
|
MD5 | 72c6ee59cdb9ecf18bb12b7a64e45489 |
|
BLAKE2b-256 | 8278fe862a6502abbb4958ade0f65b332744095d3ae755e34dd8e894ed69f549 |
Hashes for django_admin_cursor_paginator-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 479bd626ea7718436a40ec462af2e8190a46ee156c003b3e622f7a2a62cde05e |
|
MD5 | 66c0405077803c8720fccdca6360cfe5 |
|
BLAKE2b-256 | 9498f78bdd73f7bc3cb4ddf692173195609a6f4fdc7bef63e71bf3809c1f1f24 |