Serves up pandas dataframes via the Django REST Framework for client-side (i.e. d3.js) visualizations
Project description
Django REST Framework + pandas = A Model-driven Visualization API
Django REST Pandas (DRP) provides a simple way to generate and serve pandas DataFrames via the Django REST Framework. The resulting API can serve up CSV (and a number of other formats) for consumption by a client-side visualization tool like d3.js.
The design philosophy of DRP enforces a strict separation between data and presentation. This keeps the implementation simple, but also has the nice side effect of making it trivial to provide the source data for your visualizations. This capability can often be leveraged by sending users to the same URL that your visualization code uses internally to load the data.
DRP does not include any JavaScript code, leaving the implementation of interactive visualizations as an exercise for the implementer. That said, DRP is commonly used in conjunction with the wq.app library, which provides wq/chart.js and wq/pandas.js, a collection of chart functions and data loaders that work well with CSV served by DRP and wq.db’s chart module.
Tested on Python 2.7 & 3.4, with Django 1.6 & 1.7, and Django REST Framework 2.4 & 3.1.
Live Demo
The climata-viewer project uses Django REST Pandas and wq/chart.js to provide interactive visualizations and spreadsheet downloads.
Supported Formats
The following output formats are provided by default. These are provided as renderer classes in order to leverage the content type negotiation built into Django REST Framework. This means clients can specify a format via Accepts: text/csv or by appending .csv to the URL (if the URL configuration below is used).
Format |
Content Type |
pandas Dataframe Function |
Notes |
---|---|---|---|
CSV |
text/csv |
to_csv() |
|
TXT |
text/plain |
to_csv() |
Useful for testing, as most browsers will download a CSV file instead of displaying it |
JSON |
application/json |
to_json() |
|
XLSX |
application/vnd.openxml...sheet |
to_excel() |
|
XLS |
application/vnd.ms-excel |
to_excel() |
|
PNG |
image/png |
plot() |
Currently not very customizable, but a simple way to view the data as an image. |
SVG |
image/svg |
plot() |
Eventually these could become a fallback for clients that can’t handle d3.js |
See the implementation notes below for more details.
Usage
Getting Started
pip3 install rest-pandas
Usage Example
The example below assumes you already have a Django project set up with a single TimeSeries model.
# views.py
from rest_pandas import PandasView
from .models import TimeSeries
class TimeSeriesView(PandasView):
# Django REST Framework 2.4
model = TimeSeries
# Django REST Framework 3+
queryset = TimeSeries.objects.all()
serializer_class = TimeSeriesSerializer
# In response to get(), the underlying Django REST Framework ListAPIView
# will load the queryset and then pass it to the following function.
def filter_queryset(self, qs):
# At this point, you can filter queryset based on self.request or other
# settings (useful for limiting memory usage)
return qs
# Then, the default serializer (typically a DRF ModelSerializer) should
# serialize each row in the queryset into a simple dict format. To
# customize which fields to include, create a subclass of ModelSerializer
# and assign it to serializer_class on your view.
# Next, the included PandasSerializer will load the ModelSerializer result
# into a DataFrame and pass it to the following function on the view.
def transform_dataframe(self, dataframe):
# Here you can transform the dataframe based on self.request
# (useful for pivoting or computing statistics)
return dataframe
# For more control over dataframe creation, subclass PandasSerializer and
# set pandas_serializer_class on the view. (Or set list_serializer_class
# on your ModelSerializer subclass' Meta class if you're using DRF 3).
# Finally, the included Renderers will process the dataframe into one of
# the output formats below.
# urls.py
from django.conf.urls import patterns, include, url
from rest_framework.urlpatterns import format_suffix_patterns
from .views import TimeSeriesView
urlpatterns = patterns('',
url(r'^data', TimeSeriesView.as_view()),
)
urlpatterns = format_suffix_patterns(urlpatterns)
The default PandasView will serve up all of the available data from the provided model in a simple tabular form. You can also use a PandasViewSet if you are using Django REST Framework’s ViewSets and Routers, or a PandasSimpleView if you would just like to serve up some data without a Django model as the source.
Implementation Notes
The underlying implementation is a set of serializers that take the normal serializer result and put it into a dataframe. Then, the included renderers generate the output using the built in pandas functionality.
Perhaps counterintuitively, the CSV renderer is the default in Django REST Pandas, as it is the most stable and useful for API building. While the pandas JSON serializer is improving, the primary reason for making CSV the default is the compactness it provides over JSON when serializing time series data. This is particularly valuable for pandas dataframes, in which:
each record has the same keys, and
there are (usually) no nested objects
While a normal CSV file only has a single row of column headers, pandas can produce files with nested columns. This is a useful way to provide metadata about time series that is difficult to represent in a plain CSV file. However, it also makes the resulting CSV more difficult to parse. For this reason, you may be interested in wq/pandas.js, a d3 extension for loading the complex CSV generated by pandas Dataframes.
// mychart.js
define(['d3', 'wq/pandas'], function(d3, pandas) {
d3.csv("/data.csv", render);
// Or
pandas.get('/data.csv' render);
function render(error, data) {
d3.select('svg')
.selectAll('rect')
.data(data)
// ...
}
});
You can override the default renderers by setting PANDAS_RENDERERS in your settings.py, or by overriding renderer_classes in your PandasView subclass. PANDAS_RENDERERS is intentionally set separately from Django REST Framework’s own DEFAULT_RENDERER_CLASSES setting, as it is likely that you will be mixing DRP views with regular DRF views.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.