Skip to main content

Samples a percentage of SQL queries and groups them together for easy viewing

Project description

Author: Colin Howe (@colinhowe)

License: Apache 2.0

About

Django Sampler allows you to sample a percentage of your queries (SQL, Mongo, etc) and view the ones that are taking up the most time. The queries are grouped together by where they originated from in your code.

Installation

Install:

pip install git+git://github.com/colinhowe/djangosampler.git#djangosampler

or download and run

python setup.py install
Configure:
  • Add djangosampler to your INSTALLED_APPS

  • Add the tables (manage.py syncdb or manage.py migrate if you use South)

  • Add the views:

    urlpatterns += patterns('',
        (r'^sampler/', include('djangosampler.urls')),
    )
  • Set DJANGO_SAMPLER_FREQ to a value between 0 and 1

  • Set DJANGO_SAMPLER_PLUGINS to a list of plugins. For just sampling SQL a sensible default is:

    DJANGO_SAMPLER_PLUGINS = (
        'djangosampler.plugins.sql.Sql',
        # Plugins are applied in the same order as this list
    )

    There are several plugins available and it is worthwhile reading through them to get the most use out of this tool.

  • If you are using cost based sampling then set DJANGO_SAMPLER_BASE_TIME to the expected duration of a normal query in seconds. By default this is set to 5ms.

Viewing Results

After letting the sampler run for a while you will be able to view queries (grouped by their origin) at the URL you configured.

Configuration

DJANGO_SAMPLER_PLUGINS

Django Sampler has a plugin architecture to allow you to control how much data you want to be collected.

In your settings.py add the following:

DJANGO_SAMPLER_PLUGINS = (
    'djangosampler.plugins.sql.Sql',
    # Plugins are applied in the same order as this list
)

The example above will add the SQL plugin.

Available plugins and their settings are described in the Plugins section below.

DJANGO_SAMPLER_FREQ

DJANGO_SAMPLER_FREQ configures the percentage of queries that will be recorded. It should be between 0.0 and 1.0.

If this is not set then no plugins will be installed and your code will run as normal.

DJANGO_SAMPLER_USE_COST

DJANGO_SAMPLER_USE_COST will enable cost-based sampling. This causes queries that run for a long time to be sampled more often than short queries.

The chance that a query is sampled is multiplied by the total time the query takes. If a query takes 2 seconds then it will be twice as likely to be sampled as a query that takes 1 second.

The cost for a query is adjusted to account for this as follows:

cost = max(1.0, time * DJANGO_SAMPLER_FREQ) / DJANGO_SAMPLER_FREQ

Plugins

A list of available plugins follows. You can write your own plugin and this is described in the section ‘Writing Your Own Plugins’.

Django SQL

Plugin class: djangosampler.plugins.sql.Sql

The SQL sampler plugin will sample a percentage of SQL queries that occur in your application. The samples will be grouped by query and stack traces will be recorded to find where the queries are originating.

Django Requests

Plugin class: djangosampler.plugins.request.Request

The request plugin installs a Middleware that will sample the time taken by requests.

Sample any code

This is not strictly a plugin. This is a context manager that will allow you to mark blocks of code and sample how long the blocks take to run. E.g.:

from djangosampler.sampler import sampling

with sampling('my_code', 'some_fn'):
    do_something_slow()

Celery

Plugin class: djangosampler.plugins.celery_task.Celery

The Celery plugin uses Celery’s signals to sample the time taken to execute tasks.

MongoDB

Plugin class: djangosampler.plugins.mongo.Mongo

The MongoDB plugin will sample a percentage of Mongo commands (queries, inserts, etc) that occur in your application. The samples will be grouped by command and stack traces will be recorded to find where the queries are originating.

Writing Your Own Plugins

TODO. For now, look in the plugins folder and copy :)

Feedback

Feedback is always welcome! Github or twitter (@colinhowe) are the best places to reach me.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

djangosampler-0.6.10.tar.gz (13.3 kB view details)

Uploaded Source

Built Distribution

djangosampler-0.6.10.linux-x86_64.tar.gz (26.7 kB view details)

Uploaded Source

File details

Details for the file djangosampler-0.6.10.tar.gz.

File metadata

File hashes

Hashes for djangosampler-0.6.10.tar.gz
Algorithm Hash digest
SHA256 040dbb0049fbb0530a67b7df13eb90868b17227c9b34b79f37dcac7743067611
MD5 5c0c3b963f6b5c1438b43613120d1bf5
BLAKE2b-256 c99a4cd40266f936423a41686367c84f214e4d8539cf6d1c1e10f612a3dd5198

See more details on using hashes here.

File details

Details for the file djangosampler-0.6.10.linux-x86_64.tar.gz.

File metadata

File hashes

Hashes for djangosampler-0.6.10.linux-x86_64.tar.gz
Algorithm Hash digest
SHA256 d0a0c9db8b6f40287b7cfb2f60ffd79e358a0f4b39179a15974ddc7b414f99b2
MD5 37de9187c5e6e4a054050eeef6951a56
BLAKE2b-256 296a95da5164892ecb11bad50993e03398d0121f690552b1ed3c498c645dfe31

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page