Skip to main content

collective.purgebyid

Latest Version Supported - Python Versions Number of PyPI downloads License Tests Coverage

collective.purgbyid is a new method for cache invalidation of Plone based web sites. It uses the idea of adding an extra header, called X-Ids-Involved, which contains the uuids of the objects involved in the construction of the resources. For example, an image contains just its uuid:

% wget -S http://localhost:8080/Plone/image01
...
  X-Ids-Involved: #c8d7c0bc2b794325b916d990de91d7ee#

Other pages may be more complicated. Then a new purge rewrite rule adds a custom URL to the set of URLs to purge: the “purge by id” custom URL is in the form /@@purgebyid/<UUID> where UUID is the object’s uuid to be purged.

Last, Varnish is configured so that, when an URL /@@purgebyid/<UUID> is purged, it will ban all the objects that match an X-Ids-Involved header of the right type (i.e. containing the uuid of the resource to purge). This means that when a resources is purged, it is enough to purge also it /@@purgebyid/<UUID> URL because it will be Varnish responsibily to also catch all of the occurrences of the resources whenever the URL which is used to access it.

Compatibility

  • 2.x supports Plone 6.0, 6.1, 6.2 and 6.3, on Python 3.9 to 3.14.

  • 1.2.x (branch 1.x) supports Plone 4.3, 5.0, 5.1, 5.2 and 6.0, on Python 2.7 and 3.7 to 3.12.

Varnish without xkey varnish module

Without the xkey module, the way to purge a resource is to ban all objects which have the X-Ids-Involved header with the id of the resource to be purged.

For a better understanding of the differences between the two approaches (ban vs. purge), please read: https://varnish-cache.org/docs/trunk/users-guide/purging.html

Config example:

sub vcl_recv {
  if (req.method == "PURGE") {
     if (!client.ip ~ purge) {
        return (synth(405, "This IP is not allowed to send PURGE requests."));
     }
     if (req.url ~ "^/@@purgebyid/") {
        ban("obj.http.x-ids-involved ~ #" + regsub(req.url, "^/@@purgebyid/", "") + "#");
        return(synth(200, "Ban added"));
    }
    return(purge);
  }
}

sub vcl_deliver {
    unset resp.http.x-ids-involved;
}

Varnish with xkey varnish module

By default, Varnish uses the URL as the hash key for purging, but with the xkey module (https://github.com/varnish/varnish-modules/blob/master/src/vmod_xkey.vcc) there comes a secondary hash for doing so. Cached objects being tagged can be specifically purged for a more targeted cache control.

To have xkey working, it is mandatory to provide a special HTTP header called “Xkey” which contains all the tags (separated by white-space). Few additional codes in the vcl_backend_response transforms the X-Ids-Involved header into an XKey.

Config example:

import xkey;

sub vcl_recv {
    if (req.method == "PURGE") {
        if (!client.ip ~ purge) {
            return (synth(405, "This IP is not allowed to send PURGE requests."));
        }
        if (req.url ~ "^/@@purgebyid/") {
            set req.http.n-gone = xkey.purge(regsub(req.url, "^/@@purgebyid/", ""));
            # or: set req.http.n-gone = xkey.softpurge(regsub(req.url, "^/@@purgebyid/", ""));
            return (synth(200, "Invalidated "+req.http.n-gone+" objects"));
        }
    }
    return(purge);
}

sub vcl_backend_response {
    if (beresp.http.x-ids-involved) {
        set beresp.http.xkey = regsuball(beresp.http.x-ids-involved, "#", " ");
    }
}

sub vcl_deliver {
    unset resp.http.x-ids-involved;
    unset resp.http.xkey;
}

A softpurge differs from a regular purge in that it resets an object’s TTL but keeps it available for grace mode and conditional requests for the remainder of its configured grace and keep time.

How does it work? How to extend it?

During the publishing process all involved IDs (UUIDs and custom IDs) are collected (by subscribing to IPubAfterTraversal).

Important are the adapters for IInvolvedID, which are responsible for collecting IDs for their given context. The base implementation looks for the UUIDs, but may be specialized for your custom content types.

Apart from the adapter approach, there is the inline approach. You may call the following methods from collective.purgebyid.api:

  • mark_involved_objects(request, objs, stoponfirst=False)

  • mark_involved(request, single_id)

Whereas the first method uses internally the adapters for IInvolvedIDs for the given objects, the second one allows setting any arbitrary IDs. These methods combined might be used in your views, whenever a certain object or part is being rendered.

Additionally, there is a utility browser view “purgebyid”, that can be used in your template as follows:

<body tal:define="purgeutils nocall:context/@@purgebyid">
...
    <tal:image tal:define="image python:context.get_image()" tal:condition="python: image">

        <tal:mark-involved tal:define="dummy python:purgeutils.mark(image)" />
        <!-- put image rendering here -->
        ...

    </tal:image>
...
</body>

Alternatively, you can again set arbitrary IDs:

<tal:mark-involved tal:define="dummy python:purgeutils.mark('my_custom_id')" />

After having collected all IDs a ITransform adapter puts the expected X-Ids-Involved header in the HTTP response header.

When Plone sends a purge request to the configured Cache Proxy, it sends additionally a specialized request for handling objects with tags.

References

Contributors

Mauro Amico, Author

Changelog

2.0.0 (2026-11-13)

  • Tests: migrate from the deprecated plone.testing.z2 API to plone.testing.zope (installProduct, WSGI_SERVER_FIXTURE, Browser). [mamico]

  • Plone < 6.0 and Python 2.7 are no longer supported by 2.x: they remain maintained on the 1.x branch (1.2.x releases). [mamico]

  • Configure the package with plone.meta 2.11.1 (tox.ini, pre-commit, GHA workflows). Backwards incompatible: this drops support for Plone < 6.0 and Python < 3.9 (previously tested down to Plone 4.3 and Python 2.7/3.7/3.8). Supported versions are now Plone 6.0, 6.1, 6.2, 6.3 on Python 3.9 to 3.14. [mamico]

  • Switch the collective namespace to a native PEP 420 namespace package, fixing a ConfigurationError on collective.monkeypatcher under recent setuptools/Python versions. [mamico]

  • Declare all runtime and test dependencies actually imported by the package (Zope, Products.CMFCore, plone.resource, plone.transformchain, z3c.caching, plone.testing), drop unused ones (setuptools, six). [mamico]

  • Remove CI/config leftovers made obsolete by the plone.meta migration: the pre-plone.meta GitHub Actions test workflow, the Bandit workflow (broken, unmaintained Docker image), and the legacy buildout configuration/requirements files. [mamico]

1.2.3 (2024-12-12)

  • Do not register IInvolvedID adapter for ISiteRoot on Plone6 because now it is a content with UID. [cekk]

1.2.2 (2024-02-29)

  • plone 6.0 / python 3.11, 3.12 support [mamico]

1.2.1 (2022-12-08)

  • plone 6.0 / python 3.10 support [mamico]

  • avoid marking requests with the UUID of the plonesite [mamico]

1.2.0 (2022-08-04)

  • collective.xkey backports. Add utility browser view. [pgrunewald, mamico]

  • fix p.a.multilingual IUUID adapter inconsistency [mamico]

1.1.2 (2021-11-22)

  • Remove unused importDependencies (for pip install compatibility). [cekk]

1.1.1 (2019-06-05)

  • Python 3 support [mamico]

1.1.0 (2018-05-14)

  • moved headers mutator from PubSuccess event to plone.transformchain. fix missing header using p.a.caching’s ramcache operations #2 [mamico]

  • added IIDinvolved adapter for easy implements “involved id” extractors [mamico]

  • manage resourcedirectory, because previously all resources were marked as “involved” by navigation root [mamico]

  • fix issue where IUUID-adaptation did not have default value [datakurre]

1.0.0 (2016-01-14)

  • use zope.annotation on request [mamico]

  • unused generic setup profile removed [mamico]

1.0.0a1 (2013-09-11)

  • Package created using templer [Mauro Amico]

Metadata

Release files for collective.purgebyid 2.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for collective.purgebyid 2.0.0
File Size Uploaded Provenance Actions
collective_purgebyid-2.0.0.tar.gz 28.4 kB Without provenance Details for collective_purgebyid-2.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for collective.purgebyid 2.0.0
File Interpreter ABI Platform Actions
collective_purgebyid-2.0.0-py3-none-any.whl Python 3 none any Details for collective_purgebyid-2.0.0-py3-none-any.whl

Total release size: 50.2 kB

Release files / collective_purgebyid-2.0.0.tar.gz

Download URL collective_purgebyid-2.0.0.tar.gz
Size 28.4 kB
Tags Source
SHA-256 checksum
How to use checksums
1bd2ef5cc6d8f3add01451e7d344ec3fcb9ff2aeef01c9c661d4c63160089096
BLAKE2b-256 checksum
How to use checksums
f204f64b9ccf78af7be2b8a4266ea8a47c413c8996606a56db5155fc4e2a9074
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.14

Release files / collective_purgebyid-2.0.0-py3-none-any.whl

Download URL collective_purgebyid-2.0.0-py3-none-any.whl
Size 21.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
005386f220f8e0f0f46ac520798b1e2283729a803260d0ac01742fab6d195880
BLAKE2b-256 checksum
How to use checksums
1fad0f5b44d14fc512a9611e32ca2b9240e34452ab59af82dfe0b28efc6738f9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.11.14

Security for collective.purgebyid 2.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page