Skip to main content

A Django model manager providing fast / fuzzy counts for PostgreSQL database tables.

Project description

Created by Stephen McDonald

Introduction

Up until PostgreSQL 9.2, COUNT queries generally required scanning every row in a database table. With millions of rows, this can become quite slow. One work-around for this is to query statistics in PostgreSQL for an approximate row count, which in many cases is an acceptable trade-off.

Given a table called bigdata, the following query will return an approximate row count:

SELECT reltuples FROM pg_class WHERE relname = 'bigdata';

You can read more information about slow COUNT queries in PostgreSQL in the PostgreSQL wiki.

What django-postgres-fuzzycount provides is a way of using this approach directly in your Django model managers. It was originally built for displaying statistics in the kouio RSS reader, a popular alternative to Google Reader, that acquired over 5 million news articles in its database during the first week of its launch.

Installation

The easiest way to install django-postgres-fuzzycount is directly from PyPi using pip by running the following command:

$ pip install -U django-postgres-fuzzycount

Otherwise you can download and install it directly from source:

$ python setup.py install

Usage

By using the fuzzycount.FuzzyCountManager on your Django models, its count() method will return an approximate value when querying PostgreSQL tables without any WHERE OR HAVING clauses:

from django.db import models
from fuzzycount import FuzzyCountManager

class BigData(models.Model):

    big = models.BooleanField(default=True)
    data = models.TextField()

    objects = FuzzyCountManager()

BigData.objects.count()  # Uses fuzzycount
BigData.objects.filter(id__gt=9000).count()  # Doesn't use fuzzycount

The fuzzycount.FuzzyCountManager also checks the database engine being used, and only applies the approximate count query when using PostgreSQL, so other database backends can be used and will behave as usual (for varying definitions of usual, depending on the database :-).

Inspiration

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
django_postgres_fuzzycount-0.1.7-py2.py3-none-any.whl (5.7 kB) Copy SHA256 hash SHA256 Wheel 2.7 Sep 9, 2016
django-postgres-fuzzycount-0.1.7.tar.gz (4.0 kB) Copy SHA256 hash SHA256 Source None Sep 9, 2016

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page