django-waf

Self-hosted request filtering, bot management, and WAF middleware for Django.

Provides two-layer defence (nginx + Django middleware) with rate limiting, user-agent anomaly scoring, JS proof-of-work challenges, path-based threat scoring, nginx blocklist generation, and collective threat feed integration, all configurable without a reverse-proxy vendor.

This is bot detection and anti-abuse tooling, not a payload-inspecting firewall: it does not scan request bodies for SQL injection, XSS, or other OWASP Core Rule Set-class attacks. See docs/THREAT-MODEL.md for the full capability matrix, trust boundaries, and the current state of automatic-enforcement safety controls (evidence, confidence, and review path) before relying on this package as a site's only defence layer.

Features

• Rate limiting: sliding-window per-IP limits (burst, per-minute, per-5-min)
• UA anomaly scoring: heuristic detection of impossible OS/browser combos, ancient versions, scraper libraries
• Path-based threat scoring: suspicious path detection for credential probes (.env, wp-config, AWS/SSH config files) adds to the anomaly score
• HTTP method filtering: block non-standard methods (e.g. HEAD, OPTIONS, PUT, PATCH, DELETE) before rule evaluation
• JS proof-of-work challenges: hashcash-style SHA-256 challenges for suspicious clients (no CAPTCHAs, no third-party dependencies)
• Challenge auto-escalation: repeat offenders who exceed the challenge threshold are automatically blocked for a configurable TTL. A client whose HTTP fingerprint classifies as a bot counts towards the threshold even when it solves every challenge, since a datacentre CPU solves the proof-of-work almost for free
• No-referer challenge trigger: optionally challenge direct-navigation requests lacking a Referer header
• GeoIP country code population: attach ISO country codes to request log entries using a MaxMind GeoLite2 database
• Composite rules: block rules combining UA pattern with IP/CIDR
• In-process rule cache: version-checked in-memory cache avoids Redis round trips on every request; invalidated automatically when rules change
• Hit count tracking: block rules accumulate hit counts, flushed to the database periodically
• Configurable anomaly score thresholds: separate thresholds for log, challenge, and block verdicts
• nginx blocklist export utility: generates map/geo variable declarations (block and throttle kept separate) for C-level filtering at < 0.01 ms latency, with nginx -t validation and automatic rollback to the last-known-good file before any reload is signalled. django-waf generates the variables; you wire the enforcement, see nginx Integration
• Anomaly detection: auto-creates expiring rules for UA rotation, subnet bursts, challenge farms, unsolved challenges, cloud spray, and scraper 404-ratio patterns
• Collective threat feed: opt-in sync of anonymised threat intelligence across deployments
• Staff dashboard: HTMX-powered real-time analytics with anomaly management
• Form protection: defence-in-depth at the form layer: signed render tokens, honeypots, time-trap, UA-consistency, JS-touch, credential throttle (enumeration-safe), signup velocity, per-submission PoW. Mixin / decorator / template-tag entry points; per-form configuration; optional challenge-replay for false-positive rescue
• Fail-open design: Redis outage never breaks the site

Requirements

• Python >= 3.11
• Django >= 5.2
• Redis (via django-redis >= 5.4)
• httpx >= 0.27 (for threat feed sync)
• Optional: celery >= 5.3 (for scheduled tasks)
• Optional: maxminddb >= 2.4 (for GeoIP lookups)
• Optional: djangorestframework >= 3.14 (for the REST API, see REST API (optional))

Installation

pip install django-waf

With optional extras:

pip install django-waf[geoip]    # adds maxminddb for GeoIP support
pip install django-waf[celery]   # adds celery for scheduled tasks
pip install django-waf[api]      # adds djangorestframework for the REST API

Add to INSTALLED_APPS:

INSTALLED_APPS = [
    # ...
    "django_waf",
]

Add the middleware, placing it after SecurityMiddleware and before other middleware so it can block requests early:

MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    "django_waf.middleware.WafMiddleware",        # <-- here
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.common.CommonMiddleware",
    # ...
]

Include the URL routes for the challenge flow and staff dashboard:

# urls.py
from django.urls import include, path

urlpatterns = [
    path("waf/", include("django_waf.urls")),
    # ...
]

Configure a Redis cache backend (required for rate limiting):

CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://127.0.0.1:6379/0",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
        },
    }
}

Run migrations:

python manage.py migrate django_waf

Settings Reference

All settings are namespaced under DJANGO_WAF_* and have sensible defaults.

Core

Rate Limiting

Challenges

Search engine crawlers

The proof-of-work challenge carries X-Robots-Tag: noindex and a robots meta tag, and it only clears after JavaScript solves it. A search crawler runs no JavaScript, so left unhandled it is served the noindex interstitial on every path, never sees real content, and your pages fall out of the index. This is a silent SEO failure for any site that puts a public marketing surface behind the WAF.

To prevent it, DJANGO_WAF_ALLOW_VERIFIED_CRAWLERS (default True) seeds AllowRule rows for the major crawlers on migrate. Each row is rDNS-gated (verify_rdns=True): a request claiming to be Googlebot only passes if the client IP's reverse-DNS record resolves under googlebot.com/google.com (Bingbot: search.msn.com). A spoofed Googlebot user-agent from an unverified IP does not match and is scored and challenged like any other bot, so the exemption is not a user-agent bypass. Set the setting to False, or deactivate the seeded rows in the admin, to opt out.

If DJANGO_WAF_TRUST_X_FORWARDED_FOR is True, the rDNS check runs against the X-Forwarded-For client IP, so your reverse proxy must set a trustworthy XFF for crawler traffic; otherwise the check verifies the wrong IP and the rule silently fails to match.

Operators of the Drystane collective feed receive a curated, always-current crawler allowlist (new engines and revised reverse-DNS suffixes) over the same feed that ships block rules; the built-in seed covers Googlebot and Bingbot without any feed.

Anomaly Scoring

Auto-generated rules that are created quarantined, or that a detector in DJANGO_WAF_ANOMALY_OBSERVE_ONLY_DETECTORS creates, appear in the dashboard's Anomalies panel with review_status=pending. A superuser confirms (promotes to a permanent, enforcing rule) or rejects (deactivates) them from there; unreviewed quarantined rules that expire are marked expired_unreviewed rather than left pending forever. See docs/THREAT-MODEL.md section 4 for the full approval-queue design.

Logging

GeoIP

nginx Integration

django-waf is an export utility, not an auto-enforcing package. generate_nginx_blocklist() (run every 5 minutes by the generate_blocklist Celery task, BR-BL-004) writes an nginx configuration file declaring four variables from active BlockRule rows, it does not itself reject, throttle, or otherwise touch any request. You wire the enforcement (the if ($waf_block_ip) { return 403; } style checks, and the limit_req throttle zone) into your own server{}/location{} blocks, once, by hand. This is a deliberate split: which server blocks are public-facing, and what nginx features are already in use there, is a decision the package cannot safely make on an operator's behalf.

Reference wiring is shipped as package data under django_waf/conf/nginx/ (importable path: importlib.resources.files("django_waf") / "conf" / "nginx"), based on the config shape proven in production:

Why http{} scope: nginx's map and geo directives are only valid at the http{} configuration level, never inside a server{} or location{} block. Drop http-include.conf.example (or its include line) into your http{}-scope include directory (typically /etc/nginx/conf.d/ on most distributions), never into a server block.

Writable-path pattern: the Django process running generate_blocklist is typically an unprivileged application user, and /etc/nginx/ is root-owned on most distributions. Either grant that user write access to a specific file under /etc/nginx/conf.d/, or point DJANGO_WAF_NGINX_BLOCKLIST_PATH at a location the application user already owns (e.g. /var/lib/django-waf/blocklist.conf) and include that absolute path from a root-owned, one-time-edited file under /etc/nginx/conf.d/. The generated file itself then never needs to live under /etc/nginx/ at all.

Block vs throttle: the generated file declares block and throttle rules to separate variables, $waf_block_ip / $waf_block_ua for action=block, $waf_throttle_ip / $waf_throttle_ua for action=throttle, so a throttle rule never appears in the block variables and vice versa. Wire block to an unconditional return 403; wire throttle to nginx's own limit_req (requires a limit_req_zone at http{} scope). See server-include.conf.example for the full example, including the limit_req_zone declaration.

Validation and rollback: when DJANGO_WAF_NGINX_VALIDATE is True (the default), generate_nginx_blocklist() preserves the previous file as a .last-good copy, activates the new candidate, then runs DJANGO_WAF_NGINX_TEST_COMMAND (nginx -t by default). A validation failure restores the .last-good copy immediately, so a syntactically broken candidate never survives on disk as a reload timebomb waiting for the next restart. If the test command's binary is not on PATH (e.g. the Django process runs somewhere without a local nginx binary), validation is skipped gracefully and logged, the candidate still activates, exactly as if DJANGO_WAF_NGINX_VALIDATE were False.

Collective Threat Feed

Form protection (v0.11.0)

The form-protection subsystem is opt-in per form. Defaults are inert until a form opts in via the mixin, decorator, or template tag.

Usage: Django Form mixin (recommended for new forms):

from django import forms
from django_waf.forms import FormProtection, ProtectedForm

class ContactForm(ProtectedForm, forms.Form):
    name = forms.CharField()
    email = forms.EmailField()
    message = forms.CharField(widget=forms.Textarea)

    waf = FormProtection(
        form_id="contact",
        defences=("render_token", "honeypot", "time_trap", "ua_consistency"),
    )

In the view:

def contact_view(request):
    form = ContactForm(request.POST or None, request=request)
    if request.method == "POST" and form.is_valid():
        # form.waf_result holds the FormEvaluationResult.
        ...

In the template:

<form method="post">
  {% csrf_token %}
  {{ form.waf_fields }}
  {{ form.as_p }}
  <button type="submit">Send</button>
</form>

Handwritten HTML (forms that bypass Django's Form layer):

# views.py
from django_waf.forms import waf_protect_post

@waf_protect_post(form_id="contact-handwritten",
                  defences=("honeypot", "time_trap"))
def contact_view(request):
    if request.method == "POST":
        ...

<!-- contact.html -->
{% load waf_form_tags %}
<form method="post">
  {% csrf_token %}
  {% waf_protect form_id="contact-handwritten" %}
  <input type="email" name="email">
  <button type="submit">Send</button>
</form>

Hooking the login flow (required for the credential_throttle defence): the defence only reads the failure counters at submit time. Neither the mixin nor the decorator increments them, because both run before your authentication check and cannot tell a wrong password from any other validation failure. Your login view must call waf_record_credential_failure after its own credential check fails, unconditionally, whether or not the account exists: an attacker must not be able to tell "wrong password" from "no such account" by observing side effects.

from django_waf.forms import waf_record_credential_failure

# In the login view, after the credential check fails (authenticate()
# returns None, or form.is_valid() is False for an AuthenticationForm):
waf_record_credential_failure(request, form.cleaned_data["username"])

Without that call the per-account counter never increments, so credential_attack_observed never fires, and the per-IP counter never increments, so the challenge never triggers. The helper returns (account_count, ip_count) and fails open, returning (0, 0) on a Redis outage, so it is safe to call from the hot path of a login view.

HTMX compatibility: the form-protection render token persists across HTMX re-renders of the same form (a user fixing a validation error keeps the same token). The Redis marker that backs replay protection is consumed only on a PASS verdict, so submitting twice in succession after a validation error works correctly. Operators must ensure the HTMX target includes {{ form.waf_fields }} / {% waf_protect %} in the swapped fragment.

Authenticated forms: set skip_for_authenticated=True on FormProtection to drop the spam-style defences for logged-in users while keeping render_token for integrity:

waf = FormProtection(
    form_id="team-invite",
    defences=("render_token",),
    skip_for_authenticated=True,
)

Structured Logging

django_waf.logging.WafStructuredFormatter renders each log record as one JSON object per line, ready for a log aggregator (ELK, Loki, CloudWatch Logs, etc.). It always includes timestamp, level, logger, and message; it additionally includes ip, verdict, rule_id, anomaly_score, latency_ms, path, method, and user_agent (truncated to 200 characters) whenever those attributes are present on the log record. Fields that are absent are omitted entirely rather than emitted as null.

Wire it into the django_waf logger via LOGGING:

LOGGING = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "waf_json": {
            "()": "django_waf.logging.WafStructuredFormatter",
        },
    },
    "handlers": {
        "waf_json": {
            "class": "logging.StreamHandler",
            "formatter": "waf_json",
        },
    },
    "loggers": {
        "django_waf": {
            "handlers": ["waf_json"],
            "level": "INFO",
            "propagate": False,
        },
    },
}

Celery Beat Schedule

If using Celery, django_waf.conf.DJANGO_WAF_CELERY_BEAT_SCHEDULE provides a ready-made schedule fragment covering every periodic django-waf task. Merge it into your project's CELERY_BEAT_SCHEDULE rather than hand-transcribing task names and cadences:

from django_waf.conf import DJANGO_WAF_CELERY_BEAT_SCHEDULE

CELERY_BEAT_SCHEDULE = {
    **DJANGO_WAF_CELERY_BEAT_SCHEDULE,
    # ... your project's own periodic tasks
}

The */N minute tasks (generate_blocklist, flush_rule_hit_counts, detect_anomalies, parse_access_log, expire_rules, update_ip_reputation) are expressed as plain second-interval schedules, so they are always present regardless of whether celery is installed.

The wall-clock tasks (prune_request_logs, prune_challenge_tokens, sync_threat_feed, report_threat_telemetry, update_geoip_database) need celery.schedules.crontab to build their schedule. Because django_waf.conf must stay importable even when celery is entirely absent from the environment, the crontab import is guarded: if celery is not installed, these five entries are simply omitted from the dict rather than approximated. Install the celery extra (pip install django-waf[celery]) to get the full schedule:

You may of course still hand-write CELERY_BEAT_SCHEDULE entries yourself (or override DJANGO_WAF_CELERY_BEAT_SCHEDULE via the setting of the same name) if you need different cadences.

Management Commands

A dead probe_detectors schedule is silent. The task logs one line every hour: INFO when every detector is alive, WARNING naming any silent detector. This package is stateless and does not persist a last-run timestamp, so a paused Celery worker, a missing Beat entry, or a broken schedule produces no log line at all rather than a failure log. Alert on the absence of the hourly django_waf.detector_probe log line within your expected cadence, not only on its WARNING content: the probe cannot detect that it has stopped running from the inside.

Dashboard

The staff dashboard is available at /waf/dashboard/ for authenticated staff users. It provides:

• Real-time traffic counters (allowed, blocked, challenged, throttled), with a Today / 7d / 30d time-range selector
• Top 10 blocked IPs, filterable by the same time-range selector
• Rule effectiveness: the top 10 active block rules by hit count, plus a list of active rules with zero hits (removal candidates)
• Auto-detected anomalies awaiting review

Superusers can confirm auto-generated rules (promoting them to permanent) or reject them (deactivating) directly from the anomalies panel.

REST API (optional)

An optional DRF API is available under waf/api/ for integrating django-waf with external tooling (a hosted console, a CLI, a second site's dashboard). It is off by default.

Enable it:

pip install django-waf[api]

DJANGO_WAF_API_ENABLED = True

The routes are mounted automatically by django_waf.urls when the setting is true and djangorestframework is importable, no extra include() needed beyond the existing path("waf/", include("django_waf.urls", namespace="django_waf")). If the setting is true but djangorestframework is not installed, the routes are skipped and a warning is logged at import time; every endpoint also returns 503 Service Unavailable while the setting is false.

Endpoints (all under waf/api/, registered with a DRF DefaultRouter):

Permission model:

• IsWafAdmin, grants access to superusers, and to staff users holding the django_waf.change_blockrule permission. Used on the two writable endpoints (block-rules/, allow-rules/).
• IsAdminUser (DRF's built-in), grants access to is_staff users. Used on the two read-only audit endpoints (request-logs/, ip-reputation/), which carry no write actions to restrict further.

hit_count, last_hit_at, and source on BlockRuleViewSet are read-only: they are maintained by the rule engine and the threat-feed sync task, not by API clients.

Architecture

Client → nginx (C-level blocklist, < 0.01 ms)
       → Django WafMiddleware (dynamic analysis, < 0.5 ms)
       → Application views

The middleware evaluates requests in this order:

1. Exempt paths/hosts bypass: static assets, health endpoints, and exempt hosts skip all evaluation
2. HTTP method filtering: disallowed methods receive 405 immediately
3. Master switch check: DJANGO_WAF_ENABLED = False passes all requests through
4. Staff/superuser bypass: authenticated staff skip rule evaluation
5. Valid challenge cookie check: previously-solved challenges are honoured
6. Allow rules → Block rules → Rate limits: explicit rule matching
7. No-referer challenge: optionally challenge requests with no Referer header
8. Path scoring (always) + UA scoring (after 10 requests): anomaly score accumulates from suspicious paths and UA heuristics; score thresholds determine the verdict (log / challenge / block)
9. Challenge escalation: IPs exceeding the challenge threshold are auto-blocked for DJANGO_WAF_ESCALATION_BLOCK_TTL seconds. A solved challenge resets the count unless the client's fingerprint classifies as a bot, in which case it still counts
10. Verdict dispatch: response rendered (allow / block / challenge / throttle), sampled logging written, and WAF signal emitted

Development

# Run tests
pytest

# Run tests with coverage
pytest --cov=src --cov-report=term-missing

# Lint
ruff check src/ tests/
ruff format src/ tests/

# Type check. PYTHONPATH=. is required, not optional: the django-stubs
# plugin resolves `django_settings_module = "tests.settings"` as a dotted
# path from the repo root, and without it mypy exits 2 having checked
# nothing at all, which reads like a pass when scripted.
PYTHONPATH=. mypy src/

Every one of these runs in CI (.github/workflows/ci.yml): ruff check and ruff format --check in the lint job, mypy in the typecheck job, and the suite across the Python/Django matrix plus dedicated legs for real Redis and real PostgreSQL. A branch green on pytest alone is not verified.

Type information for consumers

The package ships a PEP 561 py.typed marker, so a project that type-checks its own code sees the real annotations on everything it imports from django_waf rather than Any.

Licence

MIT