Django API Profiler

Lightweight Django middleware package for API profiling, endpoint analytics, and performance monitoring with intelligent query analysis and automatic regression detection.

---

Features

• Request tracking — response time, status code, HTTP method per request
• Database query monitoring — query count and total execution time per request
• N+1 query detection — automatic identification of repeated query patterns
• Endpoint aggregation — hourly summaries with avg, p95, min, max, error rates
• Performance regression detection — compares windows and flags degradation
• Webhook alerts — fire notifications to Slack or any URL on regression
• Exception tracking — complete error monitoring with type and message
• Async processing — optional Celery integration with automatic sync fallback
• Django admin dashboard — colored response times, filtering, searching

---

Installation

pip install django-api-profiler

# With async support (recommended for production)
pip install "django-api-profiler[async]"

# With webhook alerts
pip install "django-api-profiler[webhook]"

# Everything
pip install "django-api-profiler[all]"

---

Quick Start

# settings.py
INSTALLED_APPS = [
    ...
    'profiler',
]

MIDDLEWARE = [
    ...
    'profiler.middleware.ApiProfilerMiddleware',
]

python manage.py migrate

Visit /admin to view metrics immediately. No other configuration required.

---

⚠️ Performance Warning

By default, metrics are written synchronously on every request. This adds latency to your API because every request waits for a database write to complete before returning a response.

For production use, enable async processing:

pip install "django-api-profiler[async]"

# settings.py
PROFILER = {
    "ASYNC": True,
}

Make sure Redis and a Celery worker are running:

celery -A your_project worker --loglevel=info

This offloads metric writes to a Celery worker, keeping your API response times completely unaffected by the profiler.

---

Endpoint Aggregation

Aggregation computes hourly summaries per endpoint — enabling performance trending over time. Each summary includes total requests, avg/p95/max/min response times, error count, slow request count, and N+1 occurrence count.

With Celery Beat (recommended):

Aggregation runs automatically every hour. See Celery Beat Setup below.

Without Celery:

Run manually or schedule with cron:

# manually
python manage.py compute_aggregations

# via cron — runs every hour
0 * * * * cd /your/project && python manage.py compute_aggregations

---

Celery Beat Setup

If you're using async mode, add the following to your settings.py:

from celery.schedules import crontab

CELERY_BROKER_URL = os.environ.get('CELERY_BROKER_URL', 'redis://localhost:6379/0')
CELERY_RESULT_BACKEND = os.environ.get('CELERY_RESULT_BACKEND', 'redis://localhost:6379/0')
CELERY_TIMEZONE = 'UTC'
CELERY_BEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'

CELERY_BEAT_SCHEDULE = {
    'compute-endpoint-summaries': {
        'task': 'profiler.tasks.run_aggregation',
        'schedule': crontab(minute=0),  # runs every hour
    },
}

Add django_celery_beat to INSTALLED_APPS:

INSTALLED_APPS = [
    ...
    'django_celery_beat',
    'profiler',
]

Create a config/celery.py file:

import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')

app = Celery('config')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

Update config/__init__.py:

from .celery import app as celery_app

__all__ = ('celery_app',)

Run migrations for Celery Beat:

python manage.py migrate

Then start the worker and scheduler:

# terminal 1
celery -A config worker --loglevel=info

# terminal 2
celery -A config beat --loglevel=info

Aggregation will now run automatically every hour.

---

Regression Detection

After each aggregation window, the profiler automatically compares the current window against the previous one for every route. It flags:

• Response time more than 2× the previous window average
• Error rate increase of more than 10%

Detected regressions are logged as warnings in your server logs.

Optional webhook alerts (requires django-api-profiler[webhook]):

# settings.py
PROFILER = {
    "WEBHOOK_URL": "https://hooks.slack.com/services/xxx/yyy/zzz",
}

A POST request with regression details is fired to this URL whenever a regression is detected. Works with Slack, Discord, or any webhook endpoint.

Example payload:

{
    "type": "response_time",
    "route": "api/users/<int:pk>/",
    "previous_avg_ms": 120.5,
    "current_avg_ms": 890.3
}

---

Configuration

All settings are optional. The profiler works out of the box with sensible defaults.

# settings.py
PROFILER = {
    "ASYNC": False,
    "SLOW_REQUEST_THRESHOLD_MS": 1000,
    "AGGREGATION_WINDOW_MINUTES": 60,
    "IGNORED_PATHS": ["/admin", "/static", "/favicon.ico"],
    "N_PLUS_ONE_THRESHOLD": 3,
    "SLOW_QUERY_THRESHOLD_MS": 100,
    "REGRESSION_RESPONSE_TIME_FACTOR": 2.0,
    "REGRESSION_ERROR_RATE_DELTA": 0.1,
    "WEBHOOK_URL": None,
}

Setting	Default	Description
ASYNC	False	Enable Celery async metric ingestion
SLOW_REQUEST_THRESHOLD_MS	1000	Flag requests slower than this as slow
AGGREGATION_WINDOW_MINUTES	60	Size of each aggregation window
IGNORED_PATHS	['/admin', '/static', '/favicon.ico']	Paths to skip profiling
N_PLUS_ONE_THRESHOLD	3	Repeated query count to flag as N+1
SLOW_QUERY_THRESHOLD_MS	100	Individual SQL query slow threshold
REGRESSION_RESPONSE_TIME_FACTOR	2.0	Flag if response time exceeds previous × this
REGRESSION_ERROR_RATE_DELTA	0.1	Flag if error rate increases by more than this
WEBHOOK_URL	None	Webhook URL for regression alerts

---

Architecture

HTTP Request → ApiProfilerMiddleware → ProfilingCursorWrapper (intercepts every SQL query) → build_metric_payload() (pure function, no side effects) → ASYNC=True → Redis queue → Celery Worker → RequestMetric (PostgreSQL) → ASYNC=False → synchronous write → RequestMetric (PostgreSQL) Celery Beat (every hour) — or management command → compute_endpoint_summaries() → EndpointSummary (PostgreSQL) → detect_regression() → logger.warning() → webhook POST (if WEBHOOK_URL configured)

---

Admin Dashboard

Visit /admin after running migrations to access:

• Request Metrics — every request with color-coded response times (green/orange/red), filtering by method, status code, slow flag, exception flag
• Endpoint Summaries — aggregated statistics per route with avg, p95, error rates, and N+1 counts per time window
• Exception Tracking — complete error monitoring with exception type and message per request

---