Skip to main content

Easier test setup for Zope 3 projects and other Python packages.

Project description


is a package that provides some convenience stuff to enable rapid test setup for Zope 3 components and normal Python packages. Currently doctests (normal unit doctests and functional doctests) and usual Python tests made of unittest.TestCase definitions are supported.

Doctests and test modules are found throughout a whole package and registered with sensible, modifiable defaults.

Also support for reusable test setups is provided by so-called TestGetters and TestCollectors.

Setting up doctests (contrary to writing those tests) can become cumbersome with Zope 3. In the environment you often have to prepare complex things like test layers, setup functions, teardown functions and much more. Often this steps have to be done again and again.

z3c.testsetup can shorten this work by setting sensible defaults for the most important aspects of test setups.

See README.txt in the src/z3c/testsetup directory for API documentation. There is also extensive documentation for (hopefully) every aspect of the package in this directory. See all the .txt files to learn more.

Note, that this is alphaware! Do not use it in productive environments!

Important Note:

This version of z3c.testsetup uses a new set of testfile markers than previous releases did. If you are still using Test-Layer: unit or similar, please read the README.txt in the source directory carefully to learn how to switch to the new names.


You need:

- Python 2.4. Rumors are, that also Python 2.5 will do.

Other needed packages will be downloaded during installation. Therefore you need an internet connection during installation.


From the root of the package run:

$ python2.4 bootstrap/

This will download and install everything to run buildout in the next step. Afterwards an executable script buildout should be available in the freshly created bin/ directory.

Next, fetch all needed packages, install them and create provided scripts:

$ bin/buildout

This should create a test script in bin/.


$ bin/test

you can test the installed package.


See README.txt and the other .txt files in the src/z3c/testsetup directory for API documentation.

Note: z3c.testsetup does not fetch all packages you might

need to run your tests automatically by default. Namely the and zope.component packages are not included if you require only z3c.testsetup in your

This is, to make z3c.testsetup compatible with packages that refuse collaboration when those packages are installed. For instance including zope.component in the dependencies would exclude Plone developers from using z3c.testsetup.

Therefore you must include zope.component and to your requires list in if you want to use Zope 3 tests. This, however, is done anyway by other packages used in Zope 3 projects and in most cases you do not have to care about this matter.

Detailed Documentation


Easy testsetups for Zope 3 and Python projects.

Setting up tests for Zope 3 projects sometimes tends to be cumbersome. z3c.testsetup jumps in here, to support much flatter test setups. The package supports normal Python unit tests and doctests.

Note, that if you want integration or functional tests, that you have to make sure, that the package is available during test runs. z3c.testsetup does not depend on it.

The package works in two steps:

  1. It looks for testfiles in a given package.

  2. It registers the tests according to your specifications.

This is a general introduction to z3c.testsetup. For setup examples you might see the othercave package contained in the tests/ directory. More details on special topics can be found in the appropriate .txt files in this directory.

Basic Example

Before we can find, register and execute tests, we first have to write them down. We already have some ready to use tests available, which can be found in a subpackage:

>>> import os
>>> cavepath = os.path.dirname(__file__)
>>> cavepath = os.path.join(cavepath, 'tests', 'othercave')

In this subpackage there is a simple doctest doctest01.txt (please ignore the pipes on the left):

>>> print_file(os.path.join(cavepath, 'doctest01.txt'))
|  A doctest
|  =========
|  :doctest:
|  This is a simple doctest.
|    >>> 1+1
|    2

As we can see, the doctest is marked by a special marker


This marker tells the testsetup machinery, that this file contains doctest examples that should be registered during test runs. Without this marker, a testfile won’t be registered during tests!

This is the only difference to ‘normal’ doctests here.

Other markers detected by z3c.testsetup are:

  • :unittest:

    A replacement for :doctest:, marking a Python module as containing unittests to run. Replaces old Test-Layer: python marker.

  • :setup: <>

    Execute the given setup function before running doctests in this file.

  • :teardown: <>

    Execute the given teardown function after running doctests in this file.

  • :layer: <>

    Use the given layer definition for tests in this file.

  • :zcml-layer: <ZCML_filename>

    Use the given ZCML file and run tests in this file on a ZCML layer. Tests are registered using zope.testing.doctest.DocFileSuite.

  • :functional-zcml-layer: <ZCML_filename>

    Use the given ZCML file and run tests in this file registered with

See below for explanations of the respective markers.

Now, that we have a doctest available, we can write a testsetup routine, that collects all tests, registers them and passes them to the testrunner.

We have such a simple testsetup already available:

>>> print open(os.path.join(cavepath, '')).read()
import z3c.testsetup
test_suite = z3c.testsetup.register_all_tests(

This is all we need in simple cases. We use


to tell the setup machinery, where to look for test files. Note, that also files in subpackages will be found, registered and executed, when they are marked approriately.

Let’s start the testrunner and see what it gives:

>>> import sys
>>> sys.argv = [sys.argv[0],]
>>> defaults = [
...     '--path', cavepath,
...     '--tests-pattern', '^simplesetup01$',
...     ]
>>> from zope.testing import testrunner
  Running z3c...functional.layer.DefaultZCMLLayer [ftesting.zcml] tests:
    Set up z3c....layer.DefaultZCMLLayer [ftesting.zcml] in N.NNN seconds.
    Ran 3 tests with 0 failures and 0 errors in N.NNN seconds.
  Running z3c...functional.layer.DefaultZCMLLayer [ftesting2.zcml] tests:
    Tear down z3c...layer.DefaultZCMLLayer [ftesting.zcml] ... not supported
    Running in a subprocess.
    Set up z3c...layer.DefaultZCMLLayer [ftesting2.zcml] in N.NNN seconds.
    Ran 1 tests with 0 failures and 0 errors in N.NNN seconds.
    Tear down z3c...layer.DefaultZCMLLayer [ftesting2.zcml] ... not supported
  Running z3c.testsetup.tests.othercave.testing.UnitLayer2 tests:
    Running in a subprocess.
    Set up z3c.testsetup.tests.othercave.testing.UnitLayer1 in N.NNN seconds.
    Set up z3c.testsetup.tests.othercave.testing.UnitLayer2 in N.NNN seconds.
      Running testSetUp of UnitLayer1
      Running testSetUp of UnitLayer2
      Running testTearDown of UnitLayer2
      Running testTearDown of UnitLayer1
    Ran 1 tests with 0 failures and 0 errors in N.NNN seconds.
    Tear down z3c...tests.othercave.testing.UnitLayer2 in N.NNN seconds.
    Tear down z3c...tests.othercave.testing.UnitLayer1 in N.NNN seconds.
  Running zope.testing.testrunner.layer.UnitTests tests:
    Running in a subprocess.
    Set up zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
      Custom setUp for  <DocTest doctest05.txt from ... (2 examples)>
      Custom tearDown for  <DocTest doctest05.txt from ... (2 examples)>
    Ran 7 tests with 0 failures and 0 errors in N.NNN seconds.
    Tear down zope.testing.testrunner.layer.UnitTests in N.NNN seconds.
  Total: 12 tests, 0 failures, 0 errors in N.NNN seconds.

As we can see, there were regular unittests as well as functional tests run. Some of the unittests used their own layer (UnitLayer1) whose location were printed and the functional tests used different ZCML-files for configuration.

Of course, there were more tests than only the ones defined in doctest01.txt. Let’s have a look at the other stuff.

Defining doctests in Python modules

The doctest file described above was a pure .txt file. By default z3c.testsetup looks for doctests in files with filename extension .txt, .rst and .py. This means, that also doctests in Python modules are found by default as in the following example:

>>> print_file(os.path.join(cavepath, ''))
|  """
|  Doctests in a Python module
|  ===========================
|  We can place doctests also in Python modules.
|  :doctest:
|  Here the Cave class is defined::
|    >>> from z3c.testsetup.tests.othercave.doctest08 import Cave
|    >>> Cave
|    <class 'z3c.testsetup...doctest08.Cave'>
|  """
|  class Cave(object):
|      """A Cave.
|      A cave has a number::
|        >>> hasattr(Cave, 'number')
|        True
|      """
|      number = None
|      def __init__(self, number):
|          """Create a Cave.
|          We have to give a number if we create a cave::
|            >>> c = Cave(12)
|            >>> c.number
|            12
|          """
|          self.number = number

Here we placed the marker string :doctest: into the docstring of the module. Without it, the module would not have been considered a testfile.

Note that you have to import the entities (classes, functions, etc.) from the very same file if you want to use them.

Registering regular unittests from Python modules

z3c.testsetup provides also (limited) support for regular unittest deployments as usually written in Python. An example file could look like this:

>>> print_file(os.path.join(cavepath, ''))
|  """
|  Tests with real TestCase objects.
|  :unittest:
|  """
|  import unittest
|  class TestTest(unittest.TestCase):
|      def setUp(self):
|          pass
|      def testFoo(self):
|          self.assertEqual(2, 1+1)

The module contains a marker :unittest: in its module docstring instead of the :doctest: marker used in the other examples above. It is also the replacement for the formely used :Test-Layer: python marker.

This means, that this file is registered as a regular unittest.

If you use unittests instead of doctests, then you are mainly on your own with setting up and tearing down tests. All this should be done by the test cases themselves.

The only advantage of using z3c.testsetup here is, that those tests are found and run automatically when they provide the marker.


The register_all_tests function mentioned above accepts a bunch of keyword parameters:

register_all_tests(pkg_or_dotted_name [, extensions] [, encoding]
                   [, checker] [, globs] [, optionflags]
                   [, setup] [, teardown]
                   [, zcml_config] [, layer_name] [, layer])

where all but the first parameter are keyword paramters and all but the package parameter are optional.

While the extensions parameter determines the set of testfiles to be found, the other paramters tell how to setup single tests.

The last five parameters are only fallbacks, that should better be configured in doctest files themselves via marker strings.

  • extensions:

    a list of filename extensions to be considered during doctest search. Default value for doctests is [‘.txt’, ‘.rst’, ‘.py’]. Python tests are not touched by this (they have to be regular Python modules with ‘.py’ extension).

    If we want to register .foo files, we can do so:

    >>> import z3c.testsetup
    >>> test_suite = z3c.testsetup.register_all_tests(
    ...     'z3c.testsetup.tests.cave',
    ...     extensions=['.foo'])
    >>> suite = test_suite()
    >>> get_basenames_from_suite(suite)
    ['', '', '']

    Note, that only files that contain an appropriate marker are found, regardless of the filename extension.

  • encoding:

    the encoding of testfiles. ‘utf-8’ by default. Setting this to None means using the default value. We’ve hidden one doctest file, that contains umlauts. If we set the encoding to ascii, we get an error:

    >>> test_suite = z3c.testsetup.register_all_tests(
    ...     'z3c.testsetup.tests.cave',
    ...     encoding='ascii')
    >>> suite = test_suite()
    Traceback (most recent call last):
    UnicodeDecodeError: 'ascii' codec can't decode ...: ordinal
    not in range(128)

    While using ‘latin-1’ will work:

    >>> test_suite = z3c.testsetup.register_all_tests(
    ...     'z3c.testsetup.tests.cave',
    ...     encoding='latin-1')
    >>> suite = test_suite()

    No traceback here.

    You can always overwrite an encoding setting for a certain file by following PEP 0263 ( ).

  • checker:

    An output checker for doctests. None by default. A typical output checker can be created like this:

    >>> import re
    >>> from zope.testing import renormalizing
    >>> mychecker = renormalizing.RENormalizing([
    ...    (re.compile('[0-9]*[.][0-9]* seconds'),
    ...     '<SOME NUMBER OF> seconds'),
    ...    (re.compile('at 0x[0-9a-f]+'), 'at <SOME ADDRESS>'),
    ... ])

    This would match for example output like 0.123 seconds if you write in your doctest:

    <SOME NUBMER OF> seconds

    Please see testrunner.txt for examples of usage.

    Checkers are applied to functional doctests only!

  • globs:

    A dictionary of things that should be available immediately (without imports) during tests. Default is an empty dict, which might be populated by appropriate layers (see below). ZCML layers for example get you the getRootFolder method automatically.

    This parameter is a fallback which can be overriden by testfile markers specifying a certain layer (see below).

    The globs parameter applies only to doctests.

  • optionflags:

    Optionflags influence the behaviour of the testrunner. They are logically or’d so that you can add them arithmetically. See

    for details.

  • setup:

    A callable that takes a test argument and is executed before every single doctest.

    The default function does nothing.

    This parameter is a fallback which can be overriden by testfile markers specifying a certain layer (see below).

    Specifying setup functions in a layer is also the recommended way.

  • teardown:

    The equivalent to setup.

    The default function runs


    unless overriden by a layer.

    Specifying teardown functions in a layer is also the recommended way.

  • zcml_config:

    A filepath of a ZCML file which is registered with functional doctests. In the ZCML file you can for example register principals (users) usable by functional doctests.

    By default any ftesting.zcml file from the root of the given package is taken. If this does not exist, an empty ZCML file of the z3c.testsetup package is used (ftesting.zcml).

    This parameter has no effect, if also a layer parameter is given or a docfile specifies its own layer/ZCML config (see below).

    This is a fallback parameter. Use of docfile specific layer markers is recommended.

  • layer_name:

    You can name your layer, to distinguish different setups of functional doctests. The layer name can be an arbitrary string.

    This parameter has no effect, if also a layer parameter is given or a docfile specifies its own layer/ZCML config (see below).

    This is a fallback parameter. Use of docfile specific layer markers is recommended.

  • layer:

    You can register a ZCML layer yourself and pass it as the layer parameter. If you only have a filepath to the according ZCML file, use the zcml_config paramter instead.

    This parameter overrides any zcml_config and layer_name parameter.

    This is a fallback parameter and has no effect for docfiles specifying their own layer or ZCML config.

Deprectated/unsupported parameters

The following register_all_tests-parameters are deprecated, starting with z3c.testsetup 0.3:

  • filter_func

    and related (ufilter_func, pfilter_func, etc.)

  • All testtype specific parameters

    Support for testfile specific parameters (uextensions, fextensions, etc.) is running out and its use deprecated.

Layers and setup/teardown functions

Starting with z3c.testsetup 0.3 there is first reasonable support for setting up layers per testfile. This way you can easily create setup-functions that are only run before/after certain tests.

Overall, use of layers is the recommended way from now on.

Setting up a unittest layer

We can tell z3c.testsetup to use a certain unittest layer using the :layer: marker as in the following example (see tests/othercave/doctest02.txt):

A doctests with layer
:layer: z3c.testsetup.tests.othercave.testing.UnitLayer2
  >>> 1+1

The :doctest: marker was used here as well, because without it the file would not have been detected as a registerable doctest file (we want developers to be explicit about that).



marker then tells, where the testsetup machinery can find the layer definition. It is given in dotted name notation.

How does the layer definition look like? It is defined as regualr Python code:

>>> print open(os.path.join(cavepath, '')).read()
import os
class UnitLayer1(object):
    """This represents a layer.
    A layer is a way to have common setup and teardown that happens
    once for a whole group of tests.
    It must be an object with a `setUp` and a `tearDown` method, which
    are run once before or after all the tests applied to a layer
    Optionally you can additionally define `testSetUp` and
    `testTearDown` methods, which are run before and after each single
    This class is not instantiated. Therefore we use classmethods.
    def setUp(self):
        """This gets run once for the whole test run, or at most once per
        TestSuite that depends on the layer.
        (The latter can happen if multiple suites depend on the layer
        and the testrunner decides to tear down the layer after first
        suite finishes.)
    def tearDown(self):
        """This gets run once for the whole test run, or at most
        once per TestSuite that depends on the layer,
        after all tests in the suite have finished.
    def testSetUp(self):
        """This method is run before each single test in the current
        layer. It is optional.
        print "    Running testSetUp of UnitLayer1"
    def testTearDown(self):
        """This method is run before each single test in the current
        layer. It is optional.
        print "    Running testTearDown of UnitLayer1"
class UnitLayer2(UnitLayer1):
    """This Layer inherits ``UnitLayer1``.
    This way we define nested setups. During test runs the testrunner
    will first call the setup methods of ``UnitTest1`` and then those
    of this class. Handling of teardown-methods will happen the other
    way round.
    def setUp(self):
    def testSetUp(self):
        print "    Running testSetUp of UnitLayer2"
    def testTearDown(self):
        print "    Running testTearDown of UnitLayer2"

In a layer you can do all the special stuff that is needed to run a certain group of tests properly. Our setup here is special in that we defined a nested one: UnitLayer2 inherits UnitLayer1 so that during test runs the appropriate setup and teardown methods are called (see testrunner output above).

More about test layers can be found at the documentation of testrunner layers API.

Specifying a ZCML file

When it comes to integration or functional tests, we need to specify a ZCML file to which configures the test environment for us. We can do that using the

:zcml-layer: <ZCML-file-name>

marker. It expects a ZCML filename as argument and sets up a ZCML-layered testsuite for us. An example setup might look like so (see tests/othercave/doctest03.txt):

A doctest with a ZCML-layer

:zcml-layer: ftesting.zcml

  >>> 1+1

Here we say, that the the local file ftesting.zcml should be used as ZCML configuration. As we can see in the above output of testruner, this file is indeed read during test runs and used by a ZCML layer called DefaultZCMLLayer. This layer is in fact only a

The ZCML file is looked up in the same directory as the doctest file.

When using the :zcml-layer: marker, the concerned tests are set up via special methods and functions from This way you get ‘functional’ or ‘integration’ tests out of the box: in the beginning an empty ZODB db is setup, getRootFolder, sync and other functions are pulled into the test namespace and several things more.

If you want a plain setup instead then use your own layer definition using :layer: and remove the :zcml-layer: marker.

Setting up a functional ZCML layer

Sometimes we want tests to be registered using the FunctionalDocFileSuite function from (other tests are set up using zope.testing.doctest.DocFileSuite). This function pulls in even more functions into globs, like http (a HTTPCaller instance), wraps your setUp and tearDown methods into ZODB-setups and several things more. See the definition in

This setup needs also a ZCML configuration file, which can be specified via:

:functional-zcml-layer: <ZCML-file-name>

If a functional ZCML layer is specified in a testfile this way, it will override any simple :zcml-layer: or :layer: definition.

An example setup might look like this (see tests/othercave/doctest04.txt):

>>> print_file(os.path.join(cavepath, 'doctest04.txt'))
|  A functional doctest with ZCML-layer
|  ====================================
|  :doctest:
|  :functional-zcml-layer: ftesting.zcml
|  We didn't define a real environment in ftesting.zcml, but in
|  functional tests certain often needed functions should be available
|  automatically::
|    >>> getRootFolder()
|    <zope...folder.Folder object at 0x...>

The placeholder between zope and folder was used because the location of the Folder class changed recently. This way we cover setups with old packages as well as recent ones.

Specifying setUp and tearDown methods

We can specify a setUp(test) and tearDown(test) method for the examples in a doctest file, which will be executed once for the whole doctest file. This can be done using:

:setup: <>
:teardown: <>

The callables denoted by the dotted names must accept a test parameter which will be the whole test suite of examples in the current doctest file.

An example can be found in doctest05.txt:

>>> print_file(os.path.join(cavepath, 'doctest05.txt'))
|  A doctest with custom setup/teardown functions
|  ==============================================
|  :doctest:
|  :setup: z3c.testsetup.tests.othercave.testing.setUp
|  :teardown: z3c.testsetup.tests.othercave.testing.tearDown
|    >>> 1+1
|    2
|  We make use of a function registered during custom setup::
|    >>> myfunc(2)
|    4

The setup/teardown functions denoted in the example look like this:

>>> print open(os.path.join(cavepath, ''), 'rb').read()
import os
def setUp(test):
    print "    Custom setUp for ", test
    # We register a function that will be available during tests.
     test.globs['myfunc'] = lambda x: 2*x
def tearDown(test):
    print "    Custom tearDown for ", test
    del test.globs['myfunc'] # unregister function

As we can see, there is a function myfunc pulled into the namespace of the doctest. We could, however, do arbitrary other things here, set up a relational test database or whatever.

How to upgrade from z3c.testsetup < 0.3

With the 0.3 release of z3c.testsetup the set of valid marker strings changed, introducing support for file-dependent setups, layers, etc.

If you still mark your testfiles with the :Test-Layer: marker, update your testfiles as follows:

  • :Test-Layer: unit

    Change to: :doctest:

  • :Test-Layer: python

    Change to: :unittest:

  • :Test-Layer: functional

    Change to: :functional-zcml-layer: <ZCML-FILE>

    The ZCML file must explicitly be given.

If you used custom setups passed to register_all_tests, consider declaring those setup/teardown functions in the appropriate doctest files using :setup: and :teardown:.

You might also get better structured test suites when using the new layer markers :layer:, :zcml-layer: and functional-zcml-layer:.

Try to get rid of all parameters passed to register_all_tests except those, mentioned in README.txt.


0.3 (2009-02-23)

Bug fixes

  • Updated doctest examples to reflect new zope.testing behaviour.

  • z3c.testsetup really shouldn’t require any more. If you use it in an environment without this package, then you cannot register functional tests, which is determined when loading register_all_tests from z3c.testsetup.

  • Broken modules are ignored while scanning for tests.

  • Modules are not loaded anymore if their source code does not provide a suitable marker string. For this to work, the default checker method isTestModule now expects a martian.scan.ModuleInfo as argument and not a real module. Module infos can be easily created by using module_info_from_dotted_name and module_info_from_package from the martian.scan package.

Feature Changes

  • New set of testfile markers:

    • :doctest:

      marks a testfile as a doctest.

    • :unittest:

      marks a testfile as a regular unittest.

    • :layer:

      applies the given layer definition to the tests in the doctest file.

    • :zcml-layer: filename.zcml

      sets up a ZCML layer with the given filename and applies this layer to the doctests in the doctest file.

    • :functional-zcml-layer: filename.zcml

      sets up a ZCML layer with the given filename and applies this layer to the doctests in the doctest file. Furthermore the tests are set up as functional doc tests.

    • :setup:

      applies the setUp function denoted by the dotted name to the tests in the doctest file.

    • :teardown:

      applies the tearDown function denoted by the dotted name to the tests in the doctests file.

    See the examples in tests/othercave and README.txt to learn more about using these new directives.

    The old :test-layer: marker is still supported but it is deprecated now and will vanish at least with the 0.5 version of z3c.testsetup.

0.2.2 (2008-02-29)

Bug fixes

  • z3c.testsetup now does not require zope.component nor for usage in other packages. You must take care, that those packages are available during tests, for example by adding those packages to your

0.2.1 (2008-02-18)

Bug fixes

  • Fix faulty upload egg.

0.2 (2008-02-17)

Feature Changes

  • An ftesting.zcml in the root of a handled package is now taken as default layer for functional doctests if it exists.

Bug fixes

0.1 (2008-02-15)

Feature changes

  • Initial Release


Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

z3c.testsetup-0.3.tar.gz (72.9 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page