django-icv-sitemaps

Django's built-in django.contrib.sitemaps loads every URL into memory at request time. On a site with tens of thousands of pages that means slow responses, high memory pressure, and no incremental updates when content changes. At a million URLs it simply does not work.

django-icv-sitemaps replaces that approach entirely. Sitemaps are built in the background by Celery tasks, written atomically to any Django storage backend (local, S3, GCS), and served as static files. Only sections whose content has changed are ever rebuilt. The full protocol is covered: standard, image, video, and news sitemaps, automatic file splitting, gzip compression, and search engine pinging, plus a complete set of web discovery files (robots.txt, llms.txt, ads.txt, security.txt, humans.txt) managed from the database.

Part of the ICV-Django ecosystem, but fully standalone: no other ICV packages required.

Features

• Background generation: sitemaps are generated by Celery tasks (optional), written to Django storage backends (local, S3, GCS), and served statically
• Incremental updates: post_save/post_delete signals mark affected sections as stale; only changed sections are regenerated
• All four sitemap types: standard, image, video, and news sitemaps with correct XML namespaces per the sitemap protocol
• hreflang alternates: declare per-page language alternates via get_sitemap_alternates(), rendered as xhtml:link elements on every sitemap type
• Automatic splitting: files are split at 50,000 URLs or 50 MB per the protocol limits, sized on the bytes actually rendered
• SitemapMixin: declare any Django model as sitemap-includable with a small set of class attributes
• Auto-sections: ICV_SITEMAPS_AUTO_SECTIONS wires signal handlers automatically, like Django's ICV_SEARCH_AUTO_INDEX
• robots.txt: dynamic, database-driven rules merged with settings; includes Sitemap: directive automatically
• llms.txt: AI crawler guidance served at /llms.txt
• ads.txt / app-ads.txt: IAB-format authorised seller declarations
• security.txt: RFC 9116 compliant, served at /.well-known/security.txt
• humans.txt: team credits
• URL redirects: database-driven redirect rules (301/302/307/308/410) with exact, prefix, and regex matching, priority ordering, expiry, hit tracking, and CSV import/export
• 404 tracking: automatic detection of recurring 404s with hit counts and referrer tracking; create redirect rules directly from admin
• RedirectMiddleware: opt-in middleware; a redirect rule (301/302/307/308) is evaluated before Django's URL resolver, a 410 rule only once the resolver has returned a 404, so it can never shadow a live view; fail-open design never breaks the request cycle
• Search engine ping: Google, Bing, Yandex notified on content changes (conditional on checksum comparison)
• Multi-tenancy: all discovery files are tenant-scoped; sitemap paths include tenant prefix to prevent collisions; tenant IDs are sanitised to prevent path-traversal attacks
• Gzip support: compressed .xml.gz output with correct headers
• Atomic writes: temp file then rename; no partially-written files served
• 6 management commands: setup, generate, ping, validate, stats, redirects
• Django admin: all 8 models registered with actions, list filters, and read-only views
• Celery graceful degradation: tasks work synchronously when Celery is not installed
• Testing utilities: 8 factory-boy factories, pytest fixtures, and helpers in icv_sitemaps.testing

Installation

pip install django-icv-sitemaps

Add to INSTALLED_APPS:

INSTALLED_APPS = [
    # ...
    "icv_sitemaps",
]

Run migrations:

python manage.py migrate icv_sitemaps

Include the URL configuration:

# urls.py
from django.urls import include, path

urlpatterns = [
    path("", include("icv_sitemaps.urls")),
    # ...
]

This registers all discovery file endpoints at the root (/sitemap.xml, /robots.txt, /llms.txt, /ads.txt, /app-ads.txt, /.well-known/security.txt, /humans.txt).

Quick Start

1. Make your model sitemap-includable

# myapp/models.py
from django.db import models
from icv_sitemaps.mixins import SitemapMixin


class Article(SitemapMixin, models.Model):
    sitemap_section_name = "articles"
    sitemap_changefreq = "weekly"
    sitemap_priority = 0.7

    title = models.CharField(max_length=200)
    slug = models.SlugField(unique=True)
    is_published = models.BooleanField(default=True)
    updated_at = models.DateTimeField(auto_now=True)

    def get_absolute_url(self):
        return f"/articles/{self.slug}/"

    @classmethod
    def get_sitemap_queryset(cls):
        return cls.objects.filter(is_published=True)

2. Configure auto-sections

# settings.py
ICV_SITEMAPS_BASE_URL = "https://example.com"

ICV_SITEMAPS_AUTO_SECTIONS = {
    "articles": {
        "model": "blog.Article",
        "sitemap_type": "standard",
        "changefreq": "weekly",
        "priority": 0.7,
    },
    "product_images": {
        "model": "catalogue.ProductImage",
        "sitemap_type": "image",
    },
    "videos": {
        "model": "media.Video",
        "sitemap_type": "video",
    },
    "breaking_news": {
        "model": "news.BreakingStory",
        "sitemap_type": "news",
    },
}

3. Set up and generate

# Create SitemapSection records from config
python manage.py icv_sitemaps_setup

# Generate all sitemaps
python manage.py icv_sitemaps_generate --all

# Validate output
python manage.py icv_sitemaps_validate

# Check stats
python manage.py icv_sitemaps_stats

4. Automatic regeneration

When an Article is saved or deleted, its section is marked stale. The regenerate_stale_sitemaps task picks it up on the next run.

# Celery beat schedule (optional)
from celery.schedules import crontab

CELERY_BEAT_SCHEDULE = {
    "icv-sitemaps-regenerate-stale": {
        "task": "icv_sitemaps.tasks.regenerate_stale_sitemaps",
        "schedule": crontab(minute="*/15"),
    },
    "icv-sitemaps-regenerate-all": {
        "task": "icv_sitemaps.tasks.regenerate_all_sitemaps",
        "schedule": crontab(hour=3, minute=0),
    },
    "icv-sitemaps-cleanup-logs": {
        "task": "icv_sitemaps.tasks.cleanup_generation_logs",
        "schedule": crontab(hour=4, minute=0),
    },
    "icv-sitemaps-cleanup-orphans": {
        "task": "icv_sitemaps.tasks.cleanup_orphan_files",
        "schedule": crontab(day_of_week=0, hour=5, minute=0),
    },
}

Sitemap Types

Standard

Standard XML sitemaps with <loc>, <lastmod>, <changefreq>, and <priority> per the sitemaps.org protocol.

Image

Uses the http://www.google.com/schemas/sitemap-image/1.1 namespace. Configure image fields on your mixin:

class ProductImage(SitemapMixin, models.Model):
    sitemap_section_name = "product_images"
    sitemap_type = "image"
    sitemap_image_field = "image_url"
    sitemap_image_caption_field = "caption"
    sitemap_image_title_field = "title"

Video

Uses the http://www.google.com/schemas/sitemap-video/1.1 namespace:

class Video(SitemapMixin, models.Model):
    sitemap_section_name = "videos"
    sitemap_type = "video"
    sitemap_video_url_field = "video_url"
    sitemap_video_thumbnail_field = "thumbnail_url"
    sitemap_video_title_field = "title"
    sitemap_video_description_field = "description"
    sitemap_video_duration_field = "duration_seconds"

News

Uses the http://www.google.com/schemas/sitemap-news/0.9 namespace. Entries older than ICV_SITEMAPS_NEWS_MAX_AGE_DAYS (default 2) are automatically excluded:

class BreakingStory(SitemapMixin, models.Model):
    sitemap_section_name = "breaking_news"
    sitemap_type = "news"
    sitemap_news_publication_name = "Example News"
    sitemap_news_language = "en"
    sitemap_news_title_field = "headline"
    sitemap_news_date_field = "published_at"

Alternates (hreflang)

Every sitemap type accepts <xhtml:link rel="alternate" .../> elements, one per language variant of a page. Override get_sitemap_alternates() on your model to return the full cluster: this page's own language plus every alternate, including an "x-default" entry where one applies. There is no class-attribute field mapping for this, unlike the image, video and news fields above: the cluster comes from your project's i18n routing, not a model field, so the package has no way to derive it from a single field name.

class Product(SitemapMixin, models.Model):
    sitemap_section_name = "products"
    sitemap_type = "standard"

    def get_sitemap_alternates(self):
        return [
            {"hreflang": "en", "href": self.get_absolute_url()},
            {"hreflang": "de", "href": f"/de{self.get_absolute_url()}"},
            {"hreflang": "x-default", "href": self.get_absolute_url()},
        ]

A static section reads the same shape from an "alternates" key on each entry dict:

def marketing_urls():
    return [
        {
            "loc": "/pricing/",
            "alternates": [
                {"hreflang": "de", "href": "/de/pricing/"},
                {"hreflang": "x-default", "href": "/pricing/"},
            ],
        },
    ]

Every <url> element the package generates, plus every namespace it declares, includes xmlns:xhtml="http://www.w3.org/1999/xhtml" whether or not the section uses alternates: an unused namespace declaration costs about 45 bytes and keeps every section's header identical.

Static Sections: URLs without a model

Every section above resolves a Django model. Sites also have pages with no model behind them: the homepage, a pricing page, marketing landing pages. A section_type="static" section covers exactly this, sourcing its URLs from a declared list or a callable instead of a queryset.

ICV_SITEMAPS_AUTO_SECTIONS = {
    "marketing-pages": {
        "section_type": "static",
        "sitemap_type": "standard",
        "changefreq": "weekly",
        "priority": "0.7",
        "settings": {"url_provider": "myapp.sitemaps:marketing_urls"},
    },
}

# myapp/sitemaps.py
def marketing_urls():
    """Return an iterable of sitemap entry dicts for pages with no model."""
    return [
        {"loc": "/", "changefreq": "daily", "priority": 1.0},
        {"loc": "/pricing/", "changefreq": "weekly", "priority": 0.8},
        {"loc": "/about/"},
    ]

Two ways to declare a static section's URLs, read from the section's settings JSONField:

Each entry dict is the same shape a model section produces internally: {"loc": str, "lastmod": ..., "changefreq": ..., "priority": ..., "alternates": [...], "images"/"video"/"news": ...}. Only loc is required.

from icv_sitemaps.services import create_section

create_section(
    "marketing-pages",
    url_provider="myapp.sitemaps.marketing_urls",
)

# or an inline list, no callable needed
create_section(
    "legal-pages",
    urls=[{"loc": "/terms/"}, {"loc": "/privacy/"}],
)

The callable owns host and locale concerns. Generation often runs as a background Celery task, outside any request context, so url_provider must build fully-qualified or base-relative URLs itself: it cannot rely on the current request's host or the active locale. For a multi-host or multi-locale site, reverse routes explicitly (e.g. pass urlconf=... to django.urls.reverse and wrap in translation.override(...)).

Static sections do not go stale automatically. They have no model, so post_save/post_delete never fires for them; connect_auto_section_signals() skips them silently. Their content changes on deploy, not on data writes, so call mark_section_stale("marketing-pages") from your deploy pipeline, or rely on the periodic regenerate_all_sitemaps task to pick them up.

Discovery contract: the mixin is a convenience, not a requirement

SitemapMixin is the fastest way to make a model sitemap-includable, but it is optional. Sitemap generation has two requirements, both read via duck-typed getattr(), never by checking for the mixin:

1. The model is registered in ICV_SITEMAPS_AUTO_SECTIONS (see above).
2. The instance provides the get_sitemap_*() methods that _extract_entry() reads, plus a get_sitemap_queryset() classmethod that generate_section() calls to enumerate rows.

None of these checks test for SitemapMixin. Any object providing the methods it needs for its sitemap_type works, with no inheritance from an icv-sitemaps base class required. SitemapMixin.get_sitemap_url() itself just delegates to your model's own get_absolute_url(), which is the usual Django convention, so most models need no override at all.

This follows ADR-025 principle 2 (discovery is pull-based, by protocol): a package that enumerates objects owned elsewhere must accept any object satisfying the documented protocol, never require a mixin. Content packages that want sitemap coverage (a CMS page model, for example) are never obliged to know icv_sitemaps exists beyond implementing these methods.

A model implementing the protocol directly, without SitemapMixin:

# blog/models.py
from django.db import models

class Article(models.Model):
    title = models.CharField(max_length=200)
    slug = models.SlugField(unique=True)
    is_published = models.BooleanField(default=True)
    updated_at = models.DateTimeField(auto_now=True)

    def get_absolute_url(self) -> str:
        return f"/articles/{self.slug}/"

    def get_sitemap_url(self) -> str:
        return self.get_absolute_url()

    def get_sitemap_lastmod(self):
        return self.updated_at

    def get_sitemap_changefreq(self) -> str:
        return "weekly"

    def get_sitemap_priority(self) -> float:
        return 0.7

    @classmethod
    def get_sitemap_queryset(cls):
        return cls.objects.filter(is_published=True)

# settings.py
ICV_SITEMAPS_AUTO_SECTIONS = {
    "articles": {"model": "blog.Article", "sitemap_type": "standard"},
}

That is the whole contract for a standard sitemap. SitemapMixin exists because most models want the same class-attribute-to-method mapping (and the image/video/news field wiring); reach for it when that saves you writing boilerplate, skip it when you would rather implement the methods directly.

Discovery Files

robots.txt

Database-driven rules managed via Django admin or the service layer:

from icv_sitemaps.services import add_robots_rule

# Block AI crawlers from /private/
add_robots_rule("GPTBot", "disallow", "/private/")
add_robots_rule("CCBot", "disallow", "/")

# Block all bots from /admin/
add_robots_rule("*", "disallow", "/admin/")

Extra directives from settings are appended after database rules:

ICV_SITEMAPS_ROBOTS_EXTRA_DIRECTIVES = [
    "Crawl-delay: 10",
]

Raw bulk_create bypasses cache invalidation

RobotsRule's cache is invalidated by a post_save/post_delete signal handler, and Django's bulk_create emits neither signal. Writing RobotsRule rows directly with RobotsRule.objects.bulk_create(...) leaves the cached robots.txt content stale until ICV_SITEMAPS_CACHE_TIMEOUT expires (default 3600 seconds), even though the rows already exist in the database.

Call RobotsRule.objects.bulk_create(...) yourself followed by invalidate_robots_cache(tenant_id=...):

from icv_sitemaps.services import invalidate_robots_cache
from icv_sitemaps.models import RobotsRule

RobotsRule.objects.bulk_create([...])
invalidate_robots_cache(tenant_id="acme")

ads.txt / app-ads.txt

IAB-format authorised seller declarations:

from icv_sitemaps.services import add_ads_entry

add_ads_entry("google.com", "pub-1234567890", "DIRECT", certification_id="f08c47fec0942fa0")
add_ads_entry("adnetwork.com", "pub-9876543210", "RESELLER")

# For app-ads.txt
add_ads_entry("google.com", "pub-1234567890", "DIRECT", is_app_ads=True)

Raw bulk_create bypasses cache invalidation

AdsEntry's cache is invalidated by a post_save/post_delete signal handler, and Django's bulk_create emits neither signal. Writing AdsEntry rows directly with AdsEntry.objects.bulk_create(...) leaves the cached ads.txt or app-ads.txt content stale until ICV_SITEMAPS_CACHE_TIMEOUT expires (default 3600 seconds), even though the rows already exist in the database. AdsEntry maps to two distinct cache keys depending on is_app_ads, so pass the matching value.

Call AdsEntry.objects.bulk_create(...) yourself followed by invalidate_ads_cache(is_app_ads=..., tenant_id=...):

from icv_sitemaps.services import invalidate_ads_cache
from icv_sitemaps.models import AdsEntry

AdsEntry.objects.bulk_create([...])  # is_app_ads=False rows
invalidate_ads_cache(is_app_ads=False, tenant_id="acme")

llms.txt, security.txt, humans.txt

Free-form text content managed via DiscoveryFileConfig:

from icv_sitemaps.services import set_discovery_file_content

set_discovery_file_content("llms_txt", """# llms.txt
# AI training and crawl guidance for example.com

Allow: /blog/
Disallow: /private/
""")

set_discovery_file_content("security_txt", """Contact: mailto:security@example.com
Expires: 2027-01-01T00:00:00.000Z
Preferred-Languages: en
""")

set_discovery_file_content("humans_txt", """/* TEAM */
Lead: Nigel Copley
Site: example.com
""")

Raw bulk_create bypasses cache invalidation

DiscoveryFileConfig's cache is invalidated by a post_save/post_delete signal handler, and Django's bulk_create emits neither signal. Writing DiscoveryFileConfig rows directly with DiscoveryFileConfig.objects.bulk_create(...) leaves the cached content stale until ICV_SITEMAPS_CACHE_TIMEOUT expires (default 3600 seconds), even though the rows already exist in the database. Each file_type is cached under its own key, so pass the matching value.

Call DiscoveryFileConfig.objects.bulk_create(...) yourself followed by invalidate_discovery_cache(file_type, tenant_id=...):

from icv_sitemaps.services import invalidate_discovery_cache
from icv_sitemaps.models import DiscoveryFileConfig

DiscoveryFileConfig.objects.bulk_create([...])  # all file_type="llms_txt"
invalidate_discovery_cache("llms_txt", tenant_id="acme")

URL Redirects & 404 Tracking

Redirect Rules

Database-driven redirect rules evaluated by RedirectMiddleware. A 301/302/307/308 rule is evaluated before Django's URL resolver, so it wins even for a path a view would otherwise serve: useful for a promo route or masking a page mid-migration. A 410 rule is different: it asserts the resource is permanently removed, which is incoherent for a path the resolver can still serve, so it is only evaluated once the resolver has already returned a 404. A 410 rule can never shadow a live view.

from icv_sitemaps.services import add_redirect

# Permanent redirect
add_redirect("/old-page/", "/new-page/", 301)

# Temporary redirect
add_redirect("/promo/", "/summer-sale/", 302)

# 410 Gone: page permanently removed
add_redirect("/deleted-product/", "", 410)

# Prefix match: all paths under /blog/2023/ redirect
add_redirect("/blog/2023/", "/archive/2023/", 301, match_type="prefix")

# Regex match
add_redirect(r"/product/\d+/", "/products/", 301, match_type="regex")

# Bulk import from CSV
from icv_sitemaps.services import bulk_import_redirects

with open("redirects.csv") as f:
    import csv
    rows = list(csv.DictReader(f))
    result = bulk_import_redirects(rows)
    # {"created": 150, "updated": 3, "errors": []}

# Bulk create (insert-only, faster for a large one-off import)
from icv_sitemaps.services import bulk_create_redirects

result = bulk_create_redirects(rows)
# {"created": 150, "errors": []}

Raw bulk_create bypasses cache invalidation

RedirectRule's cache is invalidated by a post_save/post_delete signal handler, and Django's bulk_create emits neither signal. Writing RedirectRule rows directly with RedirectRule.objects.bulk_create(...) leaves the cached prefix/regex rule list stale until ICV_SITEMAPS_REDIRECT_CACHE_TIMEOUT expires (default 300 seconds), even though the rows already exist in the database. Exact-match rules are unaffected: check_redirect always resolves an exact match with a direct database query, never from the cache.

Either use bulk_create_redirects above, which does one bulk_create(ignore_conflicts=True) and invalidates the cache once at the end, or call RedirectRule.objects.bulk_create(...) yourself followed by invalidate_redirect_cache(tenant_id=...):

from icv_sitemaps.services import invalidate_redirect_cache
from icv_sitemaps.models import RedirectRule

RedirectRule.objects.bulk_create([...])
invalidate_redirect_cache(tenant_id="acme")

Enable the Middleware

# settings.py
MIDDLEWARE = [
    # ... security/WAF middleware first ...
    "icv_sitemaps.middleware.RedirectMiddleware",
    "django.middleware.common.CommonMiddleware",
    # ...
]

ICV_SITEMAPS_REDIRECT_ENABLED = True

404 Tracking

Enable automatic 404 tracking to identify broken URLs:

# settings.py
ICV_SITEMAPS_404_TRACKING_ENABLED = True
ICV_SITEMAPS_404_TRACKING_SAMPLE_RATE = 1.0   # Track all 404s (reduce for high traffic)
ICV_SITEMAPS_404_IGNORE_PATTERNS = [
    r"\.(?:css|js|ico|png|jpg|jpeg|gif|svg|woff2?|ttf|eot|map)$",
]

Review top 404s and create redirects:

from icv_sitemaps.services import get_top_404s

# Top 50 unresolved 404s with at least 5 hits
for entry in get_top_404s(min_hits=5):
    print(f"{entry.path}: {entry.hit_count} hits, referrers: {entry.referrers}")

Or from the command line:

python manage.py icv_sitemaps_redirects --top-404s
python manage.py icv_sitemaps_redirects --list
python manage.py icv_sitemaps_redirects --import redirects.csv
python manage.py icv_sitemaps_redirects --export redirects.csv
python manage.py icv_sitemaps_redirects --prune   # Remove expired rules

Gone Resolution Hook

ICV_SITEMAPS_GONE_RESOLVER lets a consumer answer "is this already-404 path deliberately gone?" from their own data, without materialising a RedirectRule row per deleted object. It is called only on the 404 path, and only after a gone RedirectRule lookup has already found nothing, so a hand-authored rule always takes precedence over the resolver.

# myapp/services.py
def is_gone(request):
    if Product.deleted_objects.filter(old_url=request.path).exists():
        return 410
    return None

# settings.py
ICV_SITEMAPS_GONE_RESOLVER = "myapp.services.is_gone"

The callable takes the request and returns 410 or None; 410 is the only supported status code. Any other return value, including other status codes, is logged as a warning and treated as None (the 404 passes through unchanged): this hook exists to express "this URL is gone", not to rewrite arbitrary responses. An exception raised by the callable is caught, logged, and treated the same as None (fail-open, never breaks the response). A path resolved this way is served the same 410 response as a matching 410 RedirectRule and is not additionally recorded by the 404 tracker.

Configuration

Settings Reference

All settings are namespaced under ICV_SITEMAPS_*. Every setting has a sensible default so the package works out of the box for local development.

Auto-Sections Configuration

Each key in ICV_SITEMAPS_AUTO_SECTIONS is the section name. The value is a configuration dict:

Service Functions

All functions are importable from icv_sitemaps.services:

from icv_sitemaps.services import (
    # Sitemap generation
    generate_section,
    generate_all_sections,
    generate_index,
    mark_section_stale,
    get_generation_stats,
    # Section management
    create_section,
    delete_section,
    # Search engine ping
    ping_search_engines,
    # robots.txt
    render_robots_txt,
    add_robots_rule,
    get_robots_rules,
    invalidate_robots_cache,
    # ads.txt
    render_ads_txt,
    add_ads_entry,
    invalidate_ads_cache,
    # Discovery files
    get_discovery_file_content,
    set_discovery_file_content,
    invalidate_discovery_cache,
    # Redirects
    check_redirect,
    add_redirect,
    bulk_import_redirects,
    bulk_create_redirects,
    invalidate_redirect_cache,
    record_404,
    get_top_404s,
)

Management Commands

Signals

All signals are defined in icv_sitemaps.signals:

Celery Tasks

Multi-Tenancy

Enable tenant-scoped discovery files by setting ICV_SITEMAPS_TENANT_PREFIX_FUNC to a dotted path to a callable that returns the tenant identifier:

# myapp/tenancy.py
def get_tenant_id(request):
    return getattr(request, "tenant_id", "")

# settings.py
ICV_SITEMAPS_TENANT_PREFIX_FUNC = "myapp.tenancy.get_tenant_id"

The callable must either return a safe tenant identifier (matching [\w\-]+) or a falsy value for single-tenant use; if it raises, or returns anything else, tenant resolution fails closed rather than falling back to the single-tenant bucket: views raise TenantResolutionError (a 500), and RedirectMiddleware passes the request through unmodified.

Each tenant gets isolated robots.txt, ads.txt, sitemaps, and all other discovery files. Sitemap files are stored with tenant-prefixed paths (e.g. sitemaps/acme/products-0.xml).

Tenant foreign key (issue #50)

The six tenant-keyed models (SitemapSection, RobotsRule, AdsEntry, DiscoveryFileConfig, RedirectRule, RedirectLog) also carry a nullable tenant_ref foreign key, resolved from ICV_TENANT_MODEL (default "auth.Group", a functional floor so migration import never crashes when unset). This is a fleet-standard shape (ADR-019 section 2), not this package implementing tenancy: tenant_id remains the scoping key every filter, storage path, and cache key uses; tenant_ref adds referential integrity and a column a django-boundary RLS policy can key on.

The two are kept consistent by the package itself. On clean() and save() of any of the six models:

• If tenant_ref is unset, nothing changes.
• If tenant_ref is set and tenant_id is blank, tenant_id is derived as str(tenant_ref_id).
• If tenant_ref is set and tenant_id already holds a different value, save()/clean() raises ValidationError rather than silently overwriting it.

A consumer's tenant resolver should return str(request.tenant.pk), so the derived tenant_id always agrees with the FK. A consumer wiring django-boundary row-level security on one of these tables passes the FK's column explicitly, since it is not named tenant_id:

from boundary.migrations_ops import CreateTenantPolicy

CreateTenantPolicy("sitemapsection", tenant_column="tenant_ref_id")

Set ICV_TENANT_MODEL once, before the first migrate, and leave it unchanged thereafter (it is baked into swappable migration metadata, the get_user_model() pattern). icv_sitemaps.W003 warns at manage.py check time while the auth.Group floor is active; icv_sitemaps.E001 errors if the configured value cannot resolve to a model.

# settings.py
ICV_TENANT_MODEL = "myapp.Tenant"

Production Configuration

# settings.py
ICV_SITEMAPS_BASE_URL = "https://example.com"
STORAGES = {
    "default": {"BACKEND": "django.core.files.storage.FileSystemStorage"},
    "sitemaps": {"BACKEND": "storages.backends.s3boto3.S3Boto3Storage"},
    "staticfiles": {"BACKEND": "django.contrib.staticfiles.storage.StaticFilesStorage"},
}
ICV_STORAGES_ALIAS = "sitemaps"
ICV_SITEMAPS_STORAGE_PATH = "sitemaps/"
ICV_SITEMAPS_GZIP = True
ICV_SITEMAPS_PING_ENGINES = ["google", "bing"]
ICV_SITEMAPS_BATCH_SIZE = 10000
ICV_SITEMAPS_ASYNC_GENERATION = True

Testing

The package provides testing utilities for consuming projects:

from icv_sitemaps.testing import (
    SitemapSectionFactory,
    StaticSitemapSectionFactory,
    SitemapFileFactory,
    SitemapGenerationLogFactory,
    RobotsRuleFactory,
    AdsEntryFactory,
    DiscoveryFileConfigFactory,
    RedirectRuleFactory,
    RedirectLogFactory,
)

To run the package's own tests:

git clone https://github.com/icvoss/django-icv-sitemaps.git
cd django-icv-sitemaps
pip install -e ".[dev]"
pytest tests/ -v

Models

URL Endpoints

Requirements

• Python 3.11+
• Django 5.1+
• httpx 0.27+ (for search engine pings)
• Celery 5.3+ (optional, for background generation)

Licence

MIT