This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Introduction

collective.warmup has been created to warm up web application’s caches upon start and restart.

It works by reading a configuration file containing a list of urls that are then invoked.

collective.warmup is inspired by Gil Forcada’s warmup_plone.py script.

Why not a simple bash script?

While the basic use case is very simple, collective.warmup offers the following extra features:

  • Verifies the correctness of the response body (e.g. contains a certain string)
  • Can operate in crawl mode, following links in pages to warm up all related pages
  • Can filter out certain urls according to a general definition

Installation

collective.warmup can be installed in two ways:

  • As a normal python package via pip
  • Inside a buildout, with optional integration with Plone

Dependencies

Manual installation

To install it (using virtualenv, on a GNU/Debian machine):

$ sudo apt-get install build-essential python-dev python-lxml python-virtualenv libxml2-dev libxslt1-dev
$ virtualenv warmup && cd warmup && souce bin/activate
(warmup) $ pip install collective.warmup

and launch it with:

(warmup) $ bin/warmup <path/file.ini>

Alternative installation with buildout

To integrate collective.warmup in a buildout with Plone, add this to your configuration:

[buildout]
...
parts =
    ...
    warmup


[instance]
...
eggs +=
    collective.warmup


[instance]
environment-vars +=
    WARMUP_BIN ${buildout:directory}/bin/warmup
    WARMUP_INI ${buildout:directory}/warmup.ini
    WARMUP_HEALTH_THRESHOLD 50000


[warmup]
recipe = zc.recipe.egg:scripts
eggs = collective.warmup

After executing the buildout you will find the warmup script in the bin directory.

In this example, the script will be executed automatically by the Zope instance each time it is started.

Configuration file

This is a sample warmup.ini configuration:

[warmup]
enabled = True
sleep = 2
base_url = http://localhost
logfile = /path/to/warmup.log

urls =
    Home page


[config]
max_attempts = 2
base_path = mysite


[Home page]
path = ${config:base_path}/
max_attempts = ${config:max_attempts}
check_exists =
    Welcome
check_not_exists =
    p0wned by
follow_links = True
ignore_middle =
    @@
    ++theme++
    #
    ?
ignore_end =
    .css
    .js
    .png
    .jpg
    .jpeg
    .gif
    .xml
    RSS
    .ico

Options

Global [warmup]

enable : True or False
If False, the script will do nothing when invoked.
sleep : integer
The number of seconds the script waits between url retrievals. Defaults to 2.
base_url : a valid URL (don’t forget the protocol!)
The base URL to check (all paths in the various URL sections are relative to this URL).
log_file : a filesystem path
The file where the logs will be written.
urls : a list of sections (separated by newline)
The URLs that we want to check. Each URL must have its own section in the configuration file (see below) and we reference these sections here (do not put real URLs here!). It also set an order for the checks (which are executed sequentially).

URL section

max_attempts : integer
The maximum number of attempts to check the url. Defaults to 2
path : the path to check
The path will be added to the base_url parameter in order to retrieve the page url
check_exists : list of strings
A list of strings that must be present in the page
check_not_exists : list of strings
A list of strings that must not be present in the page
follow_links : True or False
If True the script will follow the links in the page and will perform the same checks for each link.
ignore_middle : list of strings
If follow_links is True, the links containing one of these strings will be ignored
ignore_end : list of strings
If follow_links is True, the links ending with one of these strings will be ignored

Health check

In order not to mark backend healthy too early by the load-balancer, before proper warmup, this package defines a browser view called @@health.check which can be used within your load-balancer probe mechanism. See bellow a Varnish configuration example:

backend instance_1 {
    .host = "localhost";
    .port = "8081";
    .probe = {
         .url = "/health.check";
         .interval = 5s;
         .timeout = 1s;
         .window = 5;
         .threshold = 3;
     }
}

This way Varnish will mark the Zope instance backend healthy when ZODB cache-size is bigger than WARMUP_HEALTH_THRESHOLD. If you do not define the WARMUP_HEALTH_THRESHOLD environment variable, the Zope instance backend will be marked healthy as soon as Zope will be able to handle requests.

Credits

Changelog

1.2 (2016-09-08)

  • Check for write permissions within logfile and fallback to a temporary file if can’t write instead of sys.exit [avoinea]
  • Add @@health.check browser view to be used with load-balancer health-check in order not to mark backend healthy too early before proper warmup. [avoinea]
  • Avoid zombie processes. [jone]
  • Add zcml:condition for Plone 3 compatibility [gborelli]

1.1 (2014-07-18)

  • Python 2.4 support [gborelli]

1.0 (2014-04-06)

  • First release
Release History

Release History

1.2

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
collective.warmup-1.2.tar.gz (23.5 kB) Copy SHA256 Checksum SHA256 Source Sep 8, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting