sauna.reload 0.3.1

Instant code reloading for Plone using a fork loop.

sauna.reload: so that you can finish your Plone development today and relax in sauna after calling it a day

• Introduction

• Installation

  • Prerequisitements

  • Using new buildout file for the development

    • OSX special notes

  • Updating the existing buildout.cfg

• Usage: start Plone in reload enabled manner

  • Manual reload

• Debugging with sauna.reload

• Background

• Events

• Limitations

• Troubleshooting

  • My code does not reload properly

  • sauna.reload is not active - nothing printed on console

• Source

• Credits

• Changelog

  • 0.3.1 – 2011-08-19

  • 0.3.0 – 2011-08-12

  • 0.2.1 – 2011-08-03

  • 0.2.0 – 2011-08-04

  • 0.1.1 – 2011-08-03

  • 0.1.0 – 2011-08-03

Introduction

sauna.reload partially restarts Plone and reloads your changed source code every time you save a file.

• Edit your code

• Save

• Go to a browser and hit Refresh ➔ your latest changes are active

It greatly simplifies your Plone development workflow and gives back the agility of Python.

It works with any code.

sauna.reload works on OSX and Linux with Plone 4.0 and 4.1. In theory works on Windows, but no one has looked into that yet.

Installation

Here are brief installation instructions.

Prerequisitements

In order to take the advantage of sauna.reload

• You know how to develop your own Plone add-ons and basics of buildout folder structure

• You know UNIX command line basics

• You know how to run buildout

No knowledge for warming up sauna is needed in order to use this product.

Note

Currently it is recommended to install sauna.reload directly from GitHub as it is under heavy development.

Using new buildout file for the development

This is the recommended approach how to enable sauna.reload for your development environment.

Use git to fetch sauna.reload source code to your buildout environment:

cd src
git clone git://github.com/epeli/sauna.reload.git

Create a new buildout file development.cfg which extends your existing buildout.cfg – this way you can easily keep development stuff separate from your main buildout.cfg which you can also use on the production server.

development.cfg:

[buildout]

extends = buildout.cfg

develop +=
    src/sauna.reload

[instance]

# XXX: May conflict with existing zope-conf-additional directives
zope-conf-additional =
    %import sauna.reload

eggs +=
    sauna.reload

Note

With this approach you do not need to modify the existing buildout.cfg.

Then build it out with this special config file:

bin/buildout -c development.cfg

I like to buildout buildout. I like to buildout buildout…

OSX special notes

Note

The following applies to OSX only until Watchog 0.5.4+ is released.

For OSX you need to install trunk version of WatchDog providing FSEvents based file-system monitoring support.

Use git to fetch watchdog source code to your buildout environment:

cd src
git clone git://github.com/gorakhargosh/watchdog.git

Add watchdog as a developed egg into your buildout configuration just next to sauna.reload.

development.cfg:

[buildout]

develop +=
    src/watchdog
    src/sauna.reload

If you are using vim (or macvim) on OSX, you must disable vim’s writebackups to allow WatchDog to see your modifications (otherwise vim will technically create a new file on each save and WatchDog doesn’t really understand what happened).

So, Add the following to the end of your .vimrc:

set noswapfile
set nobackup
set nowritebackup

Updating the existing buildout.cfg

Alternatively you can just hack your existing buildout.cfg to have sauna.reload.

Add this package to your buildout eggs and add following zope-conf-additional line to you instance part of buildout.cfg:

[instance]
recipe = plone.recipe.zope2instance
...
zope-conf-additional = %import sauna.reload

Usage: start Plone in reload enabled manner

To start Plone with reload functionality you need to give special environment variable RELOAD_PATH for your instance command:

RELOAD_PATH=src bin/instance fg

Or if you want to optimize load speed you can directly specify only some of your development products:

RELOAD_PATH=src/my.product:src/my.another.product bin/instance fg

Warning

If other products depend on your product, e.g CMFPlone dependencies, sauna.reload does not kick in early enough and the reload does not work.

When reload is active you should see something like this in your console when Zope starts up:

2011-08-10 13:28:59 INFO sauna.reload Starting file monitor on /Users/moo/code/x/plone4/src
2011-08-10 13:29:02 INFO sauna.reload We saved at least 29.8229699135 seconds from boot up time
2011-08-10 13:29:02 INFO sauna.reload Packages marked for reload are listed in here: http://127.0.0.1:8080/@@saunareload
2011-08-10 13:29:02 INFO sauna.reload Fork loop starting on process 14607
2011-08-10 13:29:02 INFO sauna.reload Booted up new new child in 0.104816913605 seconds. Pid 14608

… and when you save some file in src folder:

2011-08-10 13:29:41 INFO SignalHandler Caught signal SIGINT
2011-08-10 13:29:41 INFO Z2 Shutting down
2011-08-10 13:29:42 INFO SignalHandler Caught signal SIGCHLD
2011-08-10 13:29:42 INFO sauna.reload Booted up new new child in 0.123936891556 seconds. Pid 14609

CTRL+C should terminate Zope normally. There might be stil some kinks and error messages with shutdown.

Note

Your reloadable eggs must be included using z3c.autoinclude mechanism.

Only eggs loaded through z3c.autoinclude. can be reloaded. Make sure you don’t use buildout.cfg zcml = directive for your eggs or sauna.reload silently ignores changes.

Manual reload

There is also a view on Zope2 root from which it is possible to manually reload code:

http://127.0.0.1:8080/@@saunareload

Debugging with sauna.reload

Regular import pdb; pdb.set_trace() will work just fine with sauna.reload. When reload happens while in pdb, though, pdb will get killed. To avoid losing your terminal echo, because of reload unexpectedly killing your pdb, you may add the following to your ~/.pdbrc:

import termios, sys
term_fd = sys.stdin.fileno()
term_echo = termios.tcgetattr(term_fd)
term_echo[3] = term_echo[3] | termios.ECHO
term_result = termios.tcsetattr(term_fd, termios.TCSADRAIN, term_echo)

Background

sauna.reload is an attempt to recreate plone.reload without the issues it has. Like being unable to reload new grokked views or portlet code. This project was started on Plone Sauna Sprint 2011. There for the name, sauna.reload.

It can reload:

• Portlets

• Schema Interface changes

• Adapters

• Meta programming magic (e.g. Grok)

• ZCML

• Translations (changes in PO files)

• etc.

sauna.reload does reloading by using a fork loop. So actually it does not reload the code, but restarts small part of Zope2.

It does following on Zope2 startup:

1. Defers loading of your development packages by hooking into PEP 302 loader and changing their z3c.autoinclude target module

2. Starts a watcher thread which monitors changes in your development py-files

3. Stops loading of Zope2 in zope.processlifetime.IProcessStarting event by stepping into a infinite loop; Just before this, tries to load all non-developed dependencies of your development packages (resolved by z3c.autoinclude)

4. It forks a new child and lets it pass the loop

5. Loads all your development packages invoking z3c.autoinclude. This is fast!

6. And now every time when the watcher thread detects a change in development files it will signal the child to shutdown and the child will signal the parent to fork a new child when it is just about to close itself

7. Just before dying, the child saves Data.fs.index to help the new child to see the changes in ZODB (by loading the saved index)

8. GOTO 4

Internally sauna.reload uses WatchDog Python component for monitoring file-system change events.

Events

Note

The following concerns you only if your code needs to react specially to reloads (clear caches, etc.)

sauna.reload emits couple of events during reloading.

sauna.reload.events.INewChildForked

Emited immediately after new process is forked. No development packages have been yet installed. Useful if you want to do something before your code gets loaded. Note that you cannot listen this event on a package that is marked for reloading as it is not yet installed when this is fired.

sauna.reload.events.INewChildIsReady

Emitted when all the development packages has been installed to the new forked child. Useful for notifications etc.

Limitations

Deferring installation of development packages to the end of Plone boot up process means that reloading of Core Plone packages is tricky (or impossible?). For example plone.app.form is depended by CMFPlone and CMFPlone really must be installed before the fork loop or there would be no speed difference between sauna.reload and normal Plone restart. So we cannot defer the installation of plone.app.form to the end of boot up process. You would have to remove the dependency from CMFPlone for development to make it work…

Also because the product installation order is altered you may find some issues if your product does something funky on installation or at import time.

Currently only FileStorage (ZODB) is supported.

Please report any other issues at: https://github.com/epeli/sauna.reload/issues.

Troubleshooting

Report all issues on GitHub.

My code does not reload properly

You’ll see reload process going on in the terminal, but your code is still not loaded.

You should see following warnings with zcml-paths from your products:

2011-08-13 09:38:12 ERROR sauna.reload.child Failed to defer
src/sauna.reload/sauna/reload/configure.zcml. IT WILL NOT BE RELOADABLE.

Make sure your code is hooked into Plone through z3c.autoinclude and NOT using explicit zcml = directive in buildout.cfg.

• Retrofit your eggs with autoinclude support if needed

• Remove zcml = lines for your eggs in buildout.cfg

• Rerun buildout (remember bin/buildout -c development.cfg)

• Restart Plone with sauna.reload enabled

sauna.reload is not active - nothing printed on console

Check that your buildout.cfg includes zope-conf-additionalzope-conf-additional line.

If using separate development.cfg make sure you run your buildout using it:

bin/buildout -c development.cfg

Source

On GitHub.

Credits

• Esa-Matti Suuronen [esa-matti aet suuronen.org]

• Asko Soukka [asko.soukka aet iki.fi]

• Mikko Ohtamaa (idea, doccing)

• Vilmos Somogyi (logo). The logo was originally the logo of Sauna Sprint 2011 and it was created by Vilmos Somogyi.

• Yesudeep Mangalapilly for creating WatchDog component and providing support for Sauna Sprint team using it

Thanks to all happy hackers on Sauna Sprint 2011!

300 kg of beer was consumed to create this package (at least). Also several kilos of firewood, one axe, one chainsaw and one boat.

We still need testers and contributors. You are very welcome!

Changelog

0.3.1 – 2011-08-19

• Support for Five-initialized products (e.g. Archetypes) on Plone 4.1.

• Licensed under ZPL.

0.3.0 – 2011-08-12

• Support Plone 4.1

• Heavily edited readme [miohtama]

• Show nice error log message if product installation failed to defer ie. is not reloadable.

0.2.1 – 2011-08-03

• Add INewChildIsReady event and move INewChildForked event

• Remove overly used prints and use logging

• Can now reload __init__.py from package root

0.2.0 – 2011-08-04

• Support for Five-initialized products (e.g. Archetypes).

• Browser View for manual reloading

• Emit INewChildForked events

• Many bug fixes

0.1.1 – 2011-08-03

• Added missing egg description.

• Added forgotten reload for localizations.

• Prefixed commands in README with $.

0.1.0 – 2011-08-03

• First experimental release.