Products.CacheSetup 1.2

Control caching of Plone sites

Introduction

CacheFu speeds up Plone sites transparently using a combination of memory, proxy, and browser caching. Can be used by itself or with Squid, Varnish, and/or Apache. Once installed, your site should run much faster (about 10x faster by itself or about 50x faster with Squid).

CacheFu is a collection of products and recipes. The central product is Products.CacheSetup which when installed via easy_install or buildout takes care of pulling in the rest of the products from the bundle.

The full bundle includes:

• http://pypi.python.org/pypi/Products.CacheSetup

• http://pypi.python.org/pypi/Products.CMFSquidTool

• http://pypi.python.org/pypi/Products.PageCacheManager

• http://pypi.python.org/pypi/Products.PolicyHTTPCacheManager

Additional optional components include some Squid, Varnish, and Apache configuration helpers. See the installation instructions for more info about these.

The latest information about releases can be found at http://plone.org/products/cachefu

CacheFu has been tested with Plone 2.5+ and Plone 3.0. For earlier Plone versions, try the CacheFu 1.0.3 bundle instead.

Products.CacheSetup Installation

Products.CacheSetup is the python egg version of the old CacheSetup product. The egg version automatically takes care of installing all the dependancies that make up the CacheFu bundle. However, the proxy cache configs must still be downloaded and configured separately.

References to the “Manual” below refer to the new CacheFu manual nearing completion as of this release. Visit the official CacheFu site, http://plone.org/products/cachefu, for the latest instructions.

There are three options for installation, and then some post-installation steps:

(1) Global Install

To install Products.CacheSetup into the global Python environment (or a workingenv) using a traditional Zope 2 instance:

• Run easy_install Products.CacheSetup. Find out how to install setuptools (and EasyInstall) here: http://peak.telecommunity.com/DevCenter/EasyInstall

• If you are using Zope 2.9 (not 2.10), get pythonproducts and install it into your Zope instance via:

  python setup.py install --home /path/to/instance

(2) Local Install (standard buildout install)

To install Products.CacheSetup into the local zope instance if you are using zc.buildout and the plone.recipe.zope2instance recipe to manage your project:

• Add Products.CacheSetup to the list of eggs to install:

  [buildout]
  ...
  eggs =
      ...
      Products.CacheSetup

• Re-run buildout:

  $ ./bin/buildout

(3) Local Install (old style Product install)

Because its top level python namespace package is called Products, this package can also be installed in Zope 2 as an old style Zope 2 Product.

For an old style Product install, move (or symlink) the CacheSetup folder of this project (Products.CacheSetup/Products/CacheSetup) into the Products directory of the Zope instance, and restart the server.

Note that in this case, dependancies are not automatically installed so you will have to repeat this process for the rest of the CacheFu bundle: Products.CMFSquidTool, Products.PageCacheManager, and Products.PolicyHTTPCacheManager.

Post-Installation Steps

If installing for the first time:

• After starting up Zope, install the CacheSetup product via the Plone “Add/Remove Products” page

• Optional: Configure via the “Cache Configuration Tool” (see “Configuring CacheFu” in the Manual)

• Optional: Install and configure Squid or Varnish (see below)

If upgrading from a pre-1.2 version:

• Remove the previous version from the Products directory of the instance. CacheSetup, CMFSquidTool, PageCacheManager, and PolicyHTTPCacheManager have all been replaced by Products.CacheSetup, Products.CMFSquidTool, etc. If CacheFu was previously installed in a buildout via plone.recipe.distros, then remove the old CacheFu settings from this section and re-run buildout.

• Restart Zope

• Click on the reinstall link next to the CacheSetup product in the Plone “Add/Remove Products” page

• Note: A reinstall keeps your old pre-1.1 configuration in a Cache Policy named “Old Cache Policy”. Any other cache policies with ids that conflict with the new ones will be backed up and relabeled appropriately

• Optional: Configure via the “Cache Configuration Tool” (see “Configuring CacheFu” in the Manual)

• Optional: Install and configure Squid or Varnish (see below)

• IMPORTANT: There is no upgrade path from the pre-1.0 versions of CacheFu. In this case, the old CacheFu products should be completely uninstalled via the Plone “Add/Remove Products” page and then deleted from the Products directory of the instance BEFORE installing the new version.

Proxy Cache Installation and Configuration

Buildout recipes for building and configuring Squid, Varnish, and Apache are available:

• http://pypi.python.org/pypi/plone.recipe.squid

• http://pypi.python.org/pypi/plone.recipe.varnish

• http://pypi.python.org/pypi/plone.recipe.apache

Previously, the CacheFu bundle also included a collection of auto-generated proxy configs for Squid, Varnish, and Apache. Starting with version 1.2, this collection is now maintained as a separate download found at http://plone.org/products/cachefu

Changelog

Changelog for the current CacheFu release. For a history of changes prior to this release, see docs/HISTORY.txt

1.2 - released 2008-05-02

Release manager:

Ricardo Newbery, [e-mail]

Compatible with:

Plone 3.0 and Plone 2.5.x

• Refactor the GS export/import code. Add support for multiple caching policies. The format of the XML file is now consistent with standard property and object manager formats. [wichert]

• Allow installation via a GenericSetup profile. Keep using the old style external method as default installation option. This means you can now install CacheSetup completely through a GenericSetup profile dependency. [wichert]

• Repackaged as an egg. [wichert]

1.1.2 - released 2008-04-23

Release manager:

Ricardo Newbery, [e-mail]

Compatible with:

Plone 3.0 and Plone 2.5.x

• Removed some deprecated tests. [newbery]

• Synced up the squid and varnish configs with the changes in plone.recipe.squid and plone.recipe.varnish. Mostly removed the special handling of cookie-authenticated requests and in the varnish case, added some handling of cache-control tokens. [newbery]

• Added a default_ttl of zero seconds to the varnish runner. [newbery]

• Unauthorized requests now do not trigger a full CacheFu disable. CacheFu is still disabled but no objects are actually removed. A temporary fix for http://plone.org/products/cachefu/issues/134 [newbery]

• Removed no-cache from the default cache policies and added a warning to the noCache and noStore fields about the IE/SSL bug. Fixes http://plone.org/products/cachefu/issues/117 [newbery]

• Fixed reordering UI. http://plone.org/products/cachefu/issues/131 [newbery]

• Removed lazy expression cleanup routine since we don’t do macro caching anymore. [newbery]

• Fixed UnicodeDecodeError in PT_pt_render() http://plone.org/products/cachefu/issues/126 [newbery]

• Fixed rewritePurgeUrls.py. Was referencing the method before its definition. [newbery]

• Cleaned up the descriptions of some cache rules. [newbery]

• Removed RSS from the container rule since we do this in a separate rule now. [newbery]

• Fixed typo in getHeaderSetIdForCSSAndJs.py which would throw an error if still using the old cache policy (pre-1.1.1). [newbery]

1.1.1 - released 2008-02-03

Release manager:

Ricardo Newbery, [e-mail]

Compatible with:

Plone 3.0 and Plone 2.5.3

• Added an VHM ‘inside-out’ example to the custom rewritePurgeUrls script. [newbery]

• Fixed the migration from pre-1.1 cache policies. [newbery]

• After reinstall we now move the new policies to the top of the list. [newbery]

• For most use cases, Accept-Language in Vary is unnecessarily conservative. Even in many multi-lingual sites, Accept-Language is not common. So it’s no longer the default. [newbery]

• Cachefu is now disabled if filesystem version doesn’t match installed version. We also update the schemas upon reinstall. The combination of both of these should catch most problems with mutating schemas. [newbery]

• Implemented feature enhancement #9: stale-while-revalidate / stale-if-error [newbery]

• We no longer magically override the headerset settings based on the proxy cache configuration selection. We might have a reverse proxy cache that is not purgeable so now we always respect the policy settings. This is better done explicitly anyway. [newbery]

• The id widget for the cachetool is now always hidden. Changing this id would break things badly. [newbery]

• Fixed several svn external properties broken because of svn reorganization. [erral]

• Reorganized several imports and schema definitions to improve code readablity. [newbery]

• Fixed the reinstall sequence to route around potential cachepolicy id collisions. [newbery]

• Removed two policy sets from the default install. Squid-without-vary because latest squid now properly supports purge with vary. Default-cache-policy-v2 because tests don’t show any improvement over the old default policy and because of potential (but unconfirmed) problems with the no-cache rule. [newbery]

• Added kss registry to resource cache rule [newbery]

• Added a cache rule for RSS feeds [newbery]

• Added ‘b_start’ to etag for templates cache rule. Fixes http://plone.org/products/cachefu/issues/97 [newbery]

• Removed cachetool.unindexObject() from Install.py since we’ve already disabled the indexing and it was contributing to http://plone.org/products/cachefu/issues/92 [newbery]

• Eliminated a possible source for zodb write conflicts in high write scenarios. Hopefully fixes http://plone.org/products/cachefu/issues/105 [newbery]

• Fixed traceback_supplement for Plone 3.0 [rossp]

• After Plone 3.0.1, something changed in Archetypes trunk that breaks a couple of our templates. Added an explicit definition for the “allow_tabbing” variable in these templates to fix it. [newbery]

• Fixed references to ‘CacheFu’ in install instructions where it should be ‘CacheSetup’ [erikrose]

• Products.PageTemplates.TALES is obsolete. We now import zope.tales.tales instead [wichert]

• With plone 3 trunk and zope 2.10.4 the (un)registerUtility keywords are ‘component’ and ‘provided’, in IFiveSiteManager (Five 1.5.5) they are the old way (utility, interface) but IFiveSiteManager does not seem to be used here so this is changed [svincic]

• Added keyword arguments when calling (un)registerUtility, because the order of the arguments is the other way round in the IFiveSiteManager interface (at least with Five 1.4.4) [witsch]

• Was previously doing a naive check for Plone 3 style site managers which would fail on Plone 2.5 if the portal had been converted to a local site with Five 1.4. Fixed to test for five.localsitemanager. [rocky]

• Added method getEnabled to cache_tool to partly fix http://plone.org/products/cachefu/issues/96 [maurits]

• Cleaned up the icon. Properly transparent with no white border, slightly bigger, a little color, and a bit of 3D embellishment [newbery]

• Added a documentFirstHeading class and removed the icon from the main tab to match the other control panels – don’t want to stand out do we? [newbery]

• Added BaseCacheRule to types_not_searched. Fixes http://plone.org/products/cachefu/issues/113 [newbery]

• Some tests were updated; some new tests added; some test refactoring. Added policy http cache manager cache rule implementation tests. Testing schemas is deprecated. Added tests for class and skeleton tests for methods. Removed unused imports [hvelarde]

• Moved BaseContent class inheritance to BaseCacheRule to simplify code in final classes. BaseCacheRule class is initialized now to avoid potential security issues. Updated test suite to reflect these changes. [hvelarde]

• Added plone_tool marker to cache_tool [hvelarde]

• Brought back cachingPoliciesDummy.html. This was accidently removed in the last release. It’s a cosmetic thing; caching_policy_manager needs this for it’s ZMI view. [newbery]

• Proxy config changes:

  1. Cleaned up and updated the proxy configs. Moved ‘config’ directory to the more descriptive ‘cachefu-proxy-configs’. Cleaned up the readme instructions. Renamed the makeconfig configuration to makeconfig.cfg, to clarify it’s purpose. Added new configs for Squid and Varnish proxy cache setups. Added some example configs for an Apache-only proxy cache setup. Eventually, we’re probably going to deprecate this and recommend buildouts for the proxy configs. [newbery]

  2. Bug fixes: Corrected the log file path for squid log files, but kept them commented. Removed references to unexistent scripts in proxy config deploy script [erral]

  3. Bug fix: Moved ‘;’ to the tal:define in xxxxxxx. Was just behind it [redcor]

  4. Bug fixes: Fixed print statement in makeconfig. Removed unnecessary squid_direct condition. Removed “direct” directive and adjusted makeconfig to match. [newbery]

  5. Bug fixes: In apache configs, uncommented the “RewriteEngine On” line. Added an ‘all’ acl to the Squid configs, since Squid appears to need this in some situations. Fixed the proxy deploy script to not try to chmod files that do not exist. Fixed some bugs in makeconfig. [wichert]

• Bundled the current tagged CMFSquidTool which includes the latest changes:

  1. Many Squid configurations ignore query strings, but not all. Query strings are not retained in purge urls. [alecm]

  2. Fixed getUrlsToPurge when the object has no REQUEST. [maurits, reinout]

  3. Fixed a blocking Queue.put call that never raised Queue.Full and so was never caught [rossp]

  4. Squid (at least version 2.6.STABLE14) returns 404’s for PURGE requests using HTTP 1.1 when an “Accept-Encoding: identity” header is included in the request. Squid now doesn’t generate this header. [dunny]