Testing infrastructure for Zope and Plone projects.
Project description
Introduction
plone.testing provides tools for writing unit and integration tests in a Zope and Plone environment. It is not tied to Plone, and it does not depend on Zope 2 (although it has some optional Zope 2-only features).
plone.testing builds on zope.testing, in particular its layers concept. This package also aims to promote some “good practice” for writing tests of various types.
Note: If you are working with Plone, there is a complementary package plone.app.testing, which builds on plone.testing to provide additional layers useful for testing Plone add-ons.
If you are new to automated testing and test driven development, you should spend some time learning about those concepts. Some useful references include:
Bear in mind that different Python frameworks have slightly different takes on how to approach testing. Therefore, you may find examples that are different to those shown below. The core concepts should be consistent, however.
Compatibility
plone.testing 4.x has only been tested with Python 2.6. If you’re using the optional Zope 2 layers, you must use Zope version 2.12 or later. Look at plone.testing 3.x for Zope 2.10 support.
Definitions
In this documentation, we will use a number of testing-related terms. The following definitions apply:
- Unit test
An automated test (i.e. one written in code) that tests a single unit (normally a function) in isolation. A unit test attempts to prove that the given function works as expected and gives the correct output given a particular input. It is common to have a number of unit tests for a single function, testing different inputs, including boundary cases and errors. Unit tests are typically quick to write and run.
- Integration test
An automated test that tests how a number of units interact. In a Zope context, this often pertains to how a particular object or view interacts with the Zope framework, the ZODB persistence engine, and so on. Integration tests usually require some setup and can be slower to run than unit tests. It is common to have fewer integration tests than unit test.
- Functional test
An automated test that tests a feature in an “end-to-end” fashion. In a Zope context, that normally means that it invokes an action in the same way that a user would, i.e. through a web request. Functional tests are normally slower to run than either unit or integration tests, and can be significantly slower to run. It is therefore common to have only a few functional tests for each major feature, relying on unit and integration tests for the bulk of testing.
- Black box testing
Testing which only considers the system’s defined inputs and outputs. For example, a functional test is normally a black box test that provides inputs only through the defined interface (e.g. URLs published in a web application), and makes assertions only on end outputs (e.g. the response returned for requests to those URLs).
- White box testing
Testing which examines the internal state of a system to make assertions. Authors of unit and integration tests normally have significant knowledge of the implementation of the code under test, and can examine such things as data in a database or changes to the system’s environment to determine if the test succeeded or failed.
- Assertion
A check that determines whether a test succeeds or fails. For example, if a unit test for the function foo() expects it to return the value 1, an assertion could be written to verify this fact. A test is said to fail if any of its assertions fail. A test always contains one or more assertions.
- Test case
A single unit, integration or functional test. Often shortened to just test. A test case sets up, executes and makes assertions against a single scenario that bears testing.
- Test fixture
The state used as a baseline for one or more tests. The test fixture is set up before each test is executed, and torn down afterwards. This is a pre-requisite for test isolation - the principle that tests should be independent of one another.
- Layer
The configuration of a test fixture shared by a number of tests. All test cases that belong to a particular layer will be executed together. The layer is set up once before the tests are executed, and torn down once after. Layers may depend on one another. Any base layers are set up before and torn down after a particular child layer is used. The test runner will order test execution to minimise layer setup and tear-down.
- Test suite
A collection of test cases (and layers) that are executed together.
- Test runner
The program which executes tests. This is responsible for calling layer and test fixture set-up and tear-down methods. It also reports on the test run, usually by printing output to the console.
- Coverage
To have confidence in your code, you should ensure it is adequately covered by tests. That is, each line of code, and each possible branching point (loops, if statements) should be executed by a test. This is known as coverage, and is normally measured as a percentage of lines of non-test code covered by tests. Coverage can be measured by the test runner, which keeps track of which lines of code were executed in a given test run.
- Doctest
A style of testing where tests are written as examples that could be typed into the interactive Python interpreter. The test runner executes each example and checks the actual output against the expected output. Doctests can either be placed in the docstring of a method, or in a separate file. The use of doctests is largely a personal preference. Some developers like to write documentation as doctests, which has the advantage that code samples can be automatically tested for correctness. You can read more about doctests on Wikipedia.
Installation and usage
To use plone.testing in your own package, you need to add it as a dependency. Most people prefer to keep test-only dependencies separate, so that they do not need to be installed in scenarios (such as on a production server) where the tests will not be run. This can be achieved using a test extra.
In setup.py, add or modify the extras_require option, like so:
extras_require = {
'test': [
'plone.testing',
]
},
You can add other test-only dependencies to that list as well, of course.
To run tests, you need a test runner. If you are using zc.buildout, you can install a test runner using the zc.recipe.testrunner recipe. For example, you could add the following to your buildout.cfg:
[test]
recipe = zc.recipe.testrunner
eggs =
my.package [test]
defaults = ['--auto-color', '--auto-progress']
You’ll also need to add this part to the parts list, of course:
[buildout]
parts =
...
test
In this example, have listed a single package to test, called my.package, and asked for it to be installed with the [test] extra. This will install any regular dependencies (listed in the install_requires option in setup.py), as well as those in the list associated with the test key in the extras_require option.
Note that it becomes important to properly list your dependencies here, because the test runner will only be aware of the packages explicitly listed, and their dependencies. For example, if your package depends on Zope 2, you need to list Zope2 in the install_requires list in setup.py; ditto for Plone, or indeed any other package you import from.
Once you have re-run buildout, the test runner will be installed as bin/test (the executable name is taken from the name of the buildout part). You can execute it without arguments to run all tests of each egg listed in the eggs list:
$ bin/test
If you have listed several eggs, and you want to run the tests for a particular one, you can do:
$ bin/test -s my.package
If you want to run only a particular test within this package, use the -t option. This can be passed a regular expression matching either a doctest file name or a test method name.
$ bin/test -s my.package -t test_spaceship
There are other command line options, which you can find by running:
$ bin/test --help
Also note the defaults option in the buildout configuration. This can be used to set default command line options. Some commonly useful options are shown above.
Coverage reporting
When writing tests, it is useful to know how well your tests cover your code. You can create coverage reports via the excellent coverage library. In order to use it, we need to install it and a reporting script:
[buildout]
parts =
...
test
coverage
report
[coverage]
recipe = zc.recipe.egg
eggs = coverage
initialization =
include = '--source=${buildout:directory}/src'
sys.argv = sys.argv[:] + ['run', include, 'bin/test', '--all']
[report]
recipe = zc.recipe.egg
eggs = coverage
scripts = coverage=report
initialization =
sys.argv = sys.argv[:] + ['html', '-i']
This will run the bin/test script with arguments like –all to run all layers. You can also specify no or some other arguments. It will place coverage reporting information in a .coverage file inside your buildout root. Via the --source argument you specify the directories containing code you want to cover. The coverage script would otherwise generate coverage information for all executed code, including other packages and even the standard library.
Running the bin/report script will generate a human readable HTML representation of the run in the htmlcov directory. Open the contained index.html in a browser to see the result.
If you want to generate an XML representation suitable for the Cobertura plugin of Hudson, you can add another part:
[buildout]
parts =
...
report-xml
[report-xml]
recipe = zc.recipe.egg
eggs = coverage
scripts = coverage=report-xml
initialization =
sys.argv = sys.argv[:] + ['xml', '-i']
This will generate a coverage.xml file in the buildout root.
Optional dependencies
plone.testing comes with a core set of tools for managing layers, which depends only on zope.testing and unittest2. In addition, there are several layers and helper functions which can be used in your own tests (or as bases for your own layers). Some of these have deeper dependencies. However, these dependencies are optional and not installed by default. If you don’t use the relevant layers, you can safely ignore them.
plone.testing does specify these dependencies, however, using the setuptools “extras” feature. You can depend on one or more extras in your own setup.py install_requires or extras_require option using the same square bracket notation shown for the [test] buildout part above. For example, if you need both the zca and publisher extras, you can have the following in your setup.py:
extras_require = {
'test': [
'plone.testing [zca, publisher]',
]
},
The available extras are:
- zodb
ZODB testing. Depends on ZODB3. The relevant layers and helpers are in the module plone.testing.zodb.
- zca
Zope Component Architecture testing. Depends on core Zope Component Architecture packages such as zope.component and zope.event. The relevant layers and helpers are in the module plone.testing.zca.
- security
Security testing. Depends on zope.security. The relevant layers and helpers are in the module plone.testing.security.
- publisher
Zope Publisher testing. Depends on zope.app.publisher and sets up ZCML directives. The relevant layers and helpers are in the module plone.testing.publisher.
- z2
Zope 2 testing. Depends on the Zope2 egg, which includes all the dependencies of the Zope 2 application server. The relevant layers and helpers are in the module plone.testing.z2
Adding a test buildout to your package
When creating re-usable, mostly stand-alone packages, it is often useful to be able to include a buildout with the package sources itself that can be used to create a test runner. This is a popular approach for many Zope packages, for example. In fact, plone.testing itself uses this kind of layout.
To have a self-contained buildout in your package, the following is required:
You need a buildout.cfg at the root of the package.
In most cases, you always want a bootstrap.py file to make it easier for people to set up a fresh buildout.
Your package sources need to be inside a src directory. If you’re using namespace packages, that means the top level package should be in the src directory.
The src directory must be referenced in setup.py.
For example, plone.testing has the following layout:
plone.testing/ plone.testing/setup.py plone.testing/bootstrap.py plone.testing/buildout.cfg plone.testing/README.txt plone.testing/src/ plone.testing/src/plone plone.testing/src/plone/__init__.py plone.testing/src/plone/testing/ plone.testing/src/plone/testing/*
In setup.py, the following arguments are required:
packages=find_packages('src'),
package_dir={'': 'src'},
This tells setuptools where to find the source code.
The buildout.cfg for plone.testing looks like this:
[buildout]
extends =
http://download.zope.org/Zope2/index/2.12.12/versions.cfg
parts = coverage test report report-xml
develop = .
[test]
recipe = collective.xmltestreport
eggs =
plone.testing [test]
defaults = ['--auto-color', '--auto-progress']
[coverage]
recipe = zc.recipe.egg
eggs = coverage
initialization =
include = '--source=${buildout:directory}/src'
sys.argv = sys.argv[:] + ['run', include, 'bin/test', '--all', '--xml']
[report]
recipe = zc.recipe.egg
eggs = coverage
scripts = coverage=report
initialization =
sys.argv = sys.argv[:] + ['html', '-i']
[report-xml]
recipe = zc.recipe.egg
eggs = coverage
scripts = coverage=report-xml
initialization =
sys.argv = sys.argv[:] + ['xml', '-i']
Obviously, you should adjust the package name in the eggs list and the version set in the extends line as appropriate.
You can of course also add additional buildout parts, for example to include some development/debugging tools, or even a running application server for testing purposes.
Hint: If you use this package layout, you should avoid checking any files or directories generated by buildout into your version control repository. You want to ignore:
.coverage
.installed.cfg
bin
coverage.xml
develop-eggs
htmlcov
parts
src/*.egg-info
Layers
In large part, plone.testing is about layers. It provides:
A set of layers (outlined below), which you can use or extend.
A set of tools for working with layers
A mini-framework to make it easy to write layers and manage shared resources associated with layers.
We’ll discuss the last two items here, before showing how to write tests that use layers.
Layer basics
Layers are used to create test fixtures that are shared by multiple test cases. For example, if you are writing a set of integration tests, you may need to set up a database and configure various components to access that database. This type of test fixture setup can be resource-intensive and time-consuming. If it is possible to only perform the setup and tear-down once for a set of tests without losing isolation between those tests, test runs can often be sped up significantly.
Layers also allow re-use of test fixtures and set-up/tear-down code. plone.testing provides a number of useful (but optional) layers that manage test fixtures for common Zope testing scenarios, letting you focus on the actual test authoring.
At the most basic, a layer is an object with the following methods and attributes:
- setUp()
Called by the test runner when the layer is to be set up. This is called exactly once for each layer used during a test run.
- tearDown()
Called by the test runner when the layer is to be torn down. As with setUp(), this is called exactly once for each layer.
- testSetUp()
Called immediately before each test case that uses the layer is executed. This is useful for setting up aspects of the fixture that are managed on a per-test basis, as opposed to fixture shared among all tests.
- testTearDown()
Called immediately after each test case that uses the layer is executed. This is a chance to perform any post-test cleanup to ensure the fixture is ready for the next test.
- __bases__
A tuple of base layers.
Each test case is associated with zero or one layer. (The syntax for specifying the layer is shown in the section “Writing tests” below.) All the tests associated with a given layer will be executed together.
Layers can depend on one another (as indicated in the __bases__ tuple), allowing one layer to build on the fixture created by another. Base layers are set up before and torn down after their dependants.
For example, if the test runner is executing some tests that belong to layer A, and some other tests that belong to layer B, both of which depend on layer C, the order of execution might be:
1. C.setUp() 1.1. A.setUp() 1.1.1. C.testSetUp() 1.1.2. A.testSetUp() 1.1.3. [One test using layer A] 1.1.4. A.testTearDown() 1.1.5. C.testTearDown() 1.1.6. C.testSetUp() 1.1.7. A.testSetUp() 1.1.8. [Another test using layer A] 1.1.9. A.testTearDown() 1.1.10. C.testTearDown() 1.2. A.tearDown() 1.3. B.setUp() 1.3.1. C.testSetUp() 1.3.2. B.testSetUp() 1.3.3. [One test using layer B] 1.3.4. B.testTearDown() 1.3.5. C.testTearDown() 1.3.6. C.testSetUp() 1.3.7. B.testSetUp() 1.3.8. [Another test using layer B] 1.3.9. B.testTearDown() 1.3.10. C.testTearDown() 1.4. B.tearDown() 2. C.tearDown()
A base layer may of course depend on other base layers. In the case of nested dependencies like this, the order of set up and tear-down as calculated by the test runner is similar to the way in which Python searches for the method to invoke in the case of multiple inheritance.
Writing layers
The easiest way to create a new layer is to use the Layer base class and implement the setUp(), tearDown(), testSetUp() and testTearDown() methods as needed. All four are optional. The default implementation of each does nothing.
By convention, layers are created in a module called testing.py at the top level of your package. The idea is that other packages that extend your package can re-use your layers for their own testing.
A simple layer may look like this:
>>> from plone.testing import Layer >>> class SpaceShip(Layer): ... ... def setUp(self): ... print "Assembling space ship" ... ... def tearDown(self): ... print "Disasembling space ship" ... ... def testSetUp(self): ... print "Fuelling space ship in preparation for test" ... ... def testTearDown(self): ... print "Emptying the fuel tank"
Before this layer can be used, it must be instantiated. Layers are normally instantiated exactly once, since by nature they are shared between tests. This becomes important when you start to manage resources (such as persistent data, database connections, or other shared resources) in layers.
The layer instance is conventionally also found in testing.py, just after the layer class definition.
>>> SPACE_SHIP = SpaceShip()Note: Since the layer is instantiated in module scope, it will be created as soon as the testing module is imported. It is therefore very important that the layer class is inexpensive and safe to create. In general, you should avoid doing anything non-trivial in the __init__() method of your layer class. All setup should happen in the setUp() method. If you do implement __init__(), be sure to call the super version as well.
The layer shown above did not have any base layers (dependencies). Here is an example of another layer that depends on it:
>>> class ZIGSpaceShip(Layer): ... defaultBases = (SPACE_SHIP,) ... ... def setUp(self): ... print "Installing main canon">>> ZIG = ZIGSpaceShip()
Here, we have explicitly listed the base layers on which ZIGSpaceShip depends, in the defaultBases attribute. This is used by the Layer base class to set the layer bases in a way that can also be overridden: see below.
Note that we use the layer instance in the defaultBases tuple, not the class. Layer dependencies always pertain to specific layer instances. Above, we are really saying that instances of ZIGSpaceShip will, by default, require the SPACE_SHIP layer to be set up first.
Note: You may find it useful to create other layer base/mix-in classes that extend plone.testing.Layer and provide helper methods for use in your own layers. This is perfectly acceptable, but please do not confuse a layer base class used in this manner with the concept of a base layer as described above:
A class deriving from plone.testing.Layer is known as a layer class. It defines the behaviour of the layer by implementing the lifecycle methods setUp(), tearDown(), testSetUp() and/or testTearDown().
A layer class can be instantiated into an actual layer. When a layer is associated with a test, it is the layer instance that is used.
The instance is usually a shared, module-global object, although in some cases it is useful to create copies of layers by instantiating the class more than once.
Subclassing an existing layer class is just straightforward OOP re-use: the test runner is not aware of the subclassing relationship.
A layer instance can be associated with any number of layer bases, via its __bases__ property (which is usually via the defaultBases variable in the class body and/or overridden using the bases argument to the Layer constructor). These bases are layer instances, not classes. The test runner will inspect the __bases__ attribute of each layer instance it sets up to calculate layer pre-requisites and dependencies.
Also note that the zope.testing documentation contains examples of layers that are “old-style” classes where the setUp() and tearDown() methods are classmethod methods and class inheritance syntax is used to specify base layers. Whilst this pattern works, we discourage its use, because the classes created using this pattern are not really used as classes. The concept of layer bases is slightly different from class inheritance, and using the class keyword to create layers with base layers leads to a number of “gotchas” that are best avoided.
Advanced - overriding bases
In some cases, it may be useful to create a copy of a layer, but change its bases. One reason to do this may if you are re-using a layer from another module, and you need to change the order in which layers are set up and torn down.
Normally, of course, you would just re-use the layer instance, either directly in a test, or in the defaultBases tuple of another layer, but if you need to change the bases, you can pass a new list of bases to the layer instance constructor:
>>> class CATSMessage(Layer): ... ... def setUp(self): ... print "All your base are belong to us" ... ... def tearDown(self): ... print "For great justice">>> CATS_MESSAGE = CATSMessage()>>> ZERO_WING = ZIGSpaceShip(bases=(SPACE_SHIP, CATS_MESSAGE,), name="ZIGSpaceShip:CATSMessage")
Please note that when overriding bases like this, the name argument is required. This is because each layer (using in a given test run) must have a unique name. The default is to use the layer class name, but this obviously only works for one instantiation. Therefore, plone.testing requires a name when setting bases explicitly.
Please take great care when changing layer bases like this. The layer implementation may make assumptions about the test fixture that was set up by its bases. If you change the order in which the bases are listed, or remove a base altogether, the layer may fail to set up correctly.
Also, bear in mind that the new layer instance is independent of the original layer instance, so any resources defined in the layer are likely to be duplicated.
Layer combinations
Sometimes, it is useful to be able to combine several layers into one, without adding any new fixture. One way to do this is to use the Layer class directly and instantiate it with new bases:
>>> COMBI_LAYER = Layer(bases=(CATS_MESSAGE, SPACE_SHIP,), name="Combi")
Here, we have created a “no-op” layer with two bases: CATS_MESSAGE and SPACE_SHIP, named Combi.
Please note that when using Layer directly like this, the name argument is required. This is to allow the test runner to identify the layer correctly. Normally, the class name of the layer is used as a basis for the name, but when using the Layer base class directly, this is unlikely to be unique or descriptive.
Layer resources
Many layers will manage one or more resources that are used either by other layers, or by tests themselves. Examples may include database connections, thread-local objects, or configuration data.
plone.testing contains a simple resource storage abstraction that makes it easy to access resources from dependant layers or tests. The resource storage uses dictionary notation:
>>> class WarpDrive(object): ... """A shared resource""" ... ... def __init__(self, maxSpeed): ... self.maxSpeed = maxSpeed ... self.running = False ... ... def start(self, speed): ... if speed > self.maxSpeed: ... print "We need more power!" ... else: ... print "Going to warp at speed", speed ... self.running = True ... ... def stop(self): ... self.running = False>>> class ConstitutionClassSpaceShip(Layer): ... defaultBases = (SPACE_SHIP,) ... ... def setUp(self): ... self['warpDrive'] = WarpDrive(8.0) ... ... def tearDown(self): ... del self['warpDrive']>>> CONSTITUTION_CLASS_SPACE_SHIP = ConstitutionClassSpaceShip()>>> class GalaxyClassSpaceShip(Layer): ... defaultBases = (CONSTITUTION_CLASS_SPACE_SHIP,) ... ... def setUp(self): ... # Upgrade the warp drive ... self.previousMaxSpeed = self['warpDrive'].maxSpeed ... self['warpDrive'].maxSpeed = 9.5 ... ... def tearDown(self): ... # Restore warp drive to its previous speed ... self['warpDrive'].maxSpeed = self.previousMaxSpeed>>> GALAXY_CLASS_SPACE_SHIP = GalaxyClassSpaceShip()
As shown, layers (that derive from plone.testing.Layer) support item (dict-like) assignment, access and deletion of arbitrary resources under string keys.
Important: If a layer creates a resource (by assigning an object to a key on self as shown above) during fixture setup-up, it must also delete the resource on tear-down. Set-up and deletion should be symmetric: if the resource is assigned during setUp() it should be deleted in tearDown(); if it’s created in testSetUp() it should be deleted in testTearDown().
A resource defined in a base layer is accessible from and through a child layer. If a resource is set on a child using a key that also exists in a base layer, the child version will shadow the base version until the child layer is torn down (presuming it deletes the resource, which it should), but the base layer version remains intact.
Note: Accessing a resource is analogous to accessing an instance variable. For example, if a base layer assigns a resource to a given key in its setUp() method, a child layer shadows that resource with another object under the same key, the shadowed resource will by used during the testSetUp() and testTearDown() lifecycle methods if implemented by the base layer as well. This will be the case until the child layer “pops” the resource by deleting it, normally in its tearDown().
Conversely, if (as shown above) the child layer accesses and modifies the object, it will modify the original.
Note: It is sometimes necessary (or desirable) to modify a shared resource in a child layer, as shown in the example above. In this case, however, it is very important to restore the original state when the layer is torn down. Otherwise, other layers or tests using the base layer directly may be affected in difficult-to-debug ways.
If the same key is used in multiple base layers, the rules for choosing which version to use are similar to those that apply when choosing an attribute or method to use in the case of multiple inheritance.
In the example above, we used the resource manager for the warpDrive object, but we assigned the previousMaxSpeed variable to self. This is because previousMaxSpeed is internal to the layer and should not be shared with any other layers that happen to use this layer as a base. Nor should it be used by any test cases. Conversely, warpDrive is a shared resource that is exposed to other layers and test cases.
The distinction becomes even more important when you consider how a test case may access the shared resource. We’ll discuss how to write test cases that use layers shortly, but consider the following test:
>>> import unittest2 as unittest >>> class TestFasterThanLightTravel(unittest.TestCase): ... layer = GALAXY_CLASS_SPACE_SHIP ... ... def test_hyperdrive(self): ... warpDrive = self.layer['warpDrive'] ... warpDrive.start(8)
This test needs access to the shared resource. It knows that its layer defines one called warpDrive. It does not know or care that the warp drive was actually initiated by the ConstitutionClassSpaceShip base layer.
If, however, the base layer had assigned the resource as an instance variable, it would not inherit to child layers (remember: layer bases are not base classes!). The syntax to access it would be:
self.layer.__bases__[0].warpDrive
which is not only ugly, but brittle: if the list of bases is changed, the expression above may lead to an attribute error.
Writing tests
Tests are usually written in one of two ways: As methods on a class that derives from unittest.TestCase (this is sometimes known as “Python tests” or “JUnit-style tests”), or using doctest syntax.
You should realise that although the relevant frameworks (unittest, unittest2 and doctest) often talk about unit testing, these tools are also used to write integration and functional tests. The distinction between unit, integration and functional tests is largely practical: you use the same techniques to set up a fixture or write assertions for an integration test as you would for a unit test. The difference lies in what that fixture contains, and how you invoke the code under test. In general, a true unit test will have a minimal or no test fixture, whereas an integration test will have a fixture that contains the components your code is integrating with. A functional test will have a fixture that contains enough of the full system to execute and test an “end-to-end” scenario.
Python tests
Python tests use the Python unittest module, or its cousin unittest2 (see below). They should be placed in a module or package called tests for the test runner to pick them up.
For small packages, a single module called tests.py will normally contain all tests. For larger packages, it is common to have a tests package that contains a number of modules with tests. These need to start with the word test, e.g. tests/test_foo.py or tests/test_bar.py. Don’t forget the __init__.py in the tests package, too!
unittest2
In Python 2.7+, the unittest module has grown several new and useful features. To make use of these in Python 2.4, 2.5 and 2.6, an add-on module called unittest2 can be installed. plone.testing depends on unittest2 (and uses it for its own tests), so you will have access to it if you depend on plone.testing.
We will use unittest2 for the examples in this document, but import it with an alias of unittest. This makes the code forward compatible with Python 2.7, where the built-in unittest module will have all the features of the unittest2 module.
Please note that the zope.testing test runner at the time of writing (version 3.9.3) does not (yet) support the new setUpClass(), tearDownClass(), setUpModule() and tearDownModule() hooks from unittest2. This is not normally a problem, since we tend to use layers to manage complex fixtures, but it is important to be aware of nonetheless.
Test modules, classes and functions
Python tests are written with classes that derive from the base class TestCase. Each test is written as a method that takes no arguments and has a name starting with test. Other methods can be added and called from test methods as appropriate, e.g. to share some test logic.
Two special methods, setUp() and tearDown(), can also be added. These will be called before or after each test, respectively, and provide a useful place to construct and clean up test fixtures without writing a custom layer. They are obviously not as re-usable as layers, though.
Hint: Somewhat confusingly, the setUp() and tearDown() methods in a test case class are the equivalent of the testSetUp() and testTearDown() methods of a layer class.
A layer can be specified by setting the layer class attribute to a layer instance. If layers are used in conjunction with setUp() and tearDown() methods in the test class itself, the class’ setUp() method will be called after the layer’s testSetUp() method, and the class’ tearDown() method will be called before the layer’s testTearDown() method.
The TestCase base class contains a number of methods which can be used to write assertions. They all take the form self.assertSomething(), e.g. self.assertEqual(result, expectedValue). See the unittest and/or unittest2 documentation for details.
Putting this together, let’s expand on our previous example unit test:
>>> import unittest2 as unittest>>> class TestFasterThanLightTravel(unittest.TestCase): ... layer = GALAXY_CLASS_SPACE_SHIP ... ... def setUp(self): ... self.warpDrive = self.layer['warpDrive'] ... self.warpDrive.stop() ... ... def tearDown(self): ... self.warpDrive.stop() ... ... def test_warp8(self): ... self.warpDrive.start(8) ... self.assertEqual(self.warpDrive.running, True) ... ... def test_max_speed(self): ... tooFast = self.warpDrive.maxSpeed + 0.1 ... self.warpDrive.start(tooFast) ... self.assertEqual(self.warpDrive.running, False)
A few things to note:
The class derives from unittest.TestCase.
The layer class attribute is set to a layer instance (not a layer class!) defined previously. This would typically be imported from a testing module.
There are two tests here: test_warp8() and test_max_speed().
We have used the self.assertEqual() assertion in both tests to check the result of executing the start() method on the warp drive.
We have used the setUp() method to fetch the warpDrive resource and ensure that it is stopped before each test is executed. Assigning a variable to self is a useful way to provide some state to each test method, though be careful about data leaking between tests: in general, you cannot predict the order in which tests will run, and tests should always be independent.
We have used the tearDown() method to make sure the warp drive is really stopped after each test.
Test suites
If you are using version 3.8.0 or later of zope.testing, a class like the one above is all you need: any class deriving from TestCase in a module with a name starting with test will be examined for test methods. Those tests are then collected into a test suite and executed.
With older versions of zope.testing, you need to add a test_suite() function in each module that returns the tests in the test suite. The unittest module contains several tools to construct suites, but one of the simplest is to use the default test loader to load all tests in the current module:
>>> def test_suite(): ... return unittest.defaultTestLoader.loadTestsFromName(__name__)
If you need to load tests explicitly, you can use the TestSuite API from the unittest module. For example:
>>> def test_suite(): ... suite = unittest.TestSuite() ... suite.addTests([ ... unittest.makeSuite(TestFasterThanLightTravel) ... ]) ... return suite
The makeSuite() function creates a test suite from the test methods in the given class (which must derive from TestCase). This suite is then appended to an overall suite, which is returned from the test_suite() method. Note that addTests() takes a list of suites (which are coalesced into a single suite). We’ll add additional suites later.
See the unittest documentation for other options.
Note: Adding a test_suite() method to a module disables automatic test discovery, even when using a recent version of zope.testing.
Doctests
Doctests can be written in two ways: as the contents of a docstring (usually, but not always, as a means of illustrating and testing the functionality of the method or class where the docstring appears), or as a separate text file. In both cases, the standard doctest module is used. See its documentation for details about doctest syntax and conventions.
Doctests are used in two different ways:
To test documentation. That is, to ensure that code examples contained in documentation are valid and continue to work as the software is updated.
As a convenient syntax for writing tests.
These two approaches use the same testing APIs and techniques. The difference is mostly about mindset. However, it is important to avoid falling into the trap that tests can substitute for good documentation or vice-a-versa. Tests usually need to systematically go through inputs and outputs and cover off a number of corner cases. Documentation should tell a compelling narrative and usually focus on the main usage scenarios. Trying to kill these two birds with one stone normally leaves you with an unappealing pile of stones and feathers.
Docstring doctests
Doctests can be added to any module, class or function docstring:
def canOutrunKlingons(warpDrive):
"""Find out of the given warp drive can outrun Klingons.
Klingons travel at warp 8
>>> drive = WarpDrive(5)
>>> canOutrunKlingons(drive)
False
We have to be faster than that to outrun them.
>>> drive = WarpDrive(8.1)
>>> canOutrunKlingons(drive)
True
We can't outrun them if we're travelling exactly the same speed
>>> drive = WarpDrive(8.0)
>>> canOutrunKlingons(drive)
False
"""
return warpDrive.maxSpeed > 8.0
To add the doctests from a particular module to a test suite, you need to use the test_suite() function hook:
>>> import doctest
>>> def test_suite():
... suite = unittest.TestSuite()
... suite.addTests([
... unittest.makeSuite(TestFasterThanLightTravel), # our previous test
... doctest.DocTestSuite('spaceship.utils'),
... ])
... return suite
Here, we have given the name of the module to check as a string dotted name. It is also possible to import a module and pass it as an object. The code above passes a list to addTests(), making it easy to add several sets of tests to the suite: the list can contain be constructed from calls to DocTestSuite(), DocFileSuite() (shown below) and makeSuite() (shown above).
Remember that if you add a test_suite() function to a module that also has TestCase-derived python tests, those tests will no longer be automatically picked up by zope.testing, so you need to add them to the test suite explicitly.
The example above illustrates a documentation-oriented doctest, where the doctest forms part of the docstring of a public module. The same syntax can be used for more systematic unit tests. For example, we could have a module spaceship.tests.test_spaceship with a set of methods like:
# It's often better to put the import into each method, but here we've
# imported the code under test at module level
from spaceship.utils import WarpDrive, canOutrunKlingons
def test_canOutrunKlingons_too_small():
"""Klingons travel at warp 8.0
>>> drive = WarpDrive(7.9)
>>> canOutrunKlingons(drive)
False
"""
def test_canOutrunKlingons_big():
"""Klingons travel at warp 8.0
>>> drive = WarpDrive(8.1)
>>> canOutrunKlingons(drive)
True
"""
def test_canOutrunKlingons_must_be_greater():
"""Klingons travel at warp 8.0
>>> drive = WarpDrive(8.0)
>>> canOutrunKlingons(drive)
False
"""
Here, we have created a number of small methods that have no body. They merely serve as a container for docstrings with doctests. Since the module has no globals, each test must import the code under test, which helps make import errors more explicit.
File doctests
Doctests contained in a file are similar to those contained in docstrings. File doctests are better suited to narrative documentation covering the usage of an entire module or package.
For example, if we had a file called spaceship.txt with doctests, we could add it to the test suite above with:
>>> def test_suite():
... suite = unittest.TestSuite()
... suite.addTests([
... unittest.makeSuite(TestFasterThanLightTravel),
... doctest.DocTestSuite('spaceship.utils'),
... doctest.DocFileSuite('spaceship.txt'),
... ])
... return suite
By default, the file is located relative to the module where the test suite is defined. You can use ../ (even on Windows) to reference the parent directory, which is sometimes useful if the doctest is inside a module in a tests package.
Note: If you put the doctest test_suite() method in a module inside a tests package, that module must have a name starting with test. It is common to have tests/test_doctests.py that contains a single test_suite() method that returns a suite of multiple doctests.
It is possible to pass several tests to the suite, e.g.:
>>> def test_suite():
... suite = unittest.TestSuite()
... suite.addTests([
... unittest.makeSuite(TestFasterThanLightTravel),
... doctest.DocTestSuite('spaceship.utils'),
... doctest.DocFileSuite('spaceship.txt', 'warpdrive.txt',),
... ])
... return suite
The test runner will report each file as a separate test, i.e. the DocFileSuite() above would add two tests to the overall suite. Conversely, a DocTestSuite() using a module with more than one docstring containing doctests will report one test for each eligible docstring.
Doctest fixtures and layers
A docstring doctest will by default have access to any global symbol available in the module where the docstring is found (e.g. anything defined or imported in the module). The global namespace can be overridden by passing a globs keyword argument to the DocTestSuite() constructor, or augmented by passing an extraglobs argument. Both should be given dictionaries.
A file doctest has an empty globals namespace by default. Globals may be provided via the globs argument to DocFileSuite().
To manage a simple test fixture for a doctest, you can define set-up and tear-down functions and pass them as the setUp and tearDown arguments respectively. These are both passed a single argument, a DocTest object. The most useful attribute of this object is globs, which is a mutable dictionary of globals available in the test.
For example:
>>> def setUpKlingons(doctest): ... doctest.globs['oldStyleKlingons'] = True>>> def tearDownKlingons(doctest): ... doctest.globs['oldStyleKlingons'] = False>>> def test_suite(): ... suite = unittest.TestSuite() ... suite.addTests([ ... doctest.DocTestSuite('spaceship.utils', setUp=setUpKlingons, tearDown=tearDownKlingons), ... ]) ... return suite
The same arguments are available on the DocFileSuite() constructor. The set up method is called before each docstring in the given module for a DocTestSuite, and before each file given in a DocFileSuite.
Of course, we often want to use layers with doctests too. Unfortunately, the unittest API is not aware of layers, so you can’t just pass a layer to the DocTestSuite() and DocFileSuite() constructors. Instead, you have to set a layer attribute on the suite after it has been constructed.
Furthermore, to use layer resources in a doctest, we need access to the layer instance. The easiest way to do this is to pass it as a glob, conventionally called ‘layer’. This makes a global name ‘layer’ available in the doctest itself, giving access to the test’s layer instance.
To make it easier to do this, plone.testing comes with a helper function called layered(). Its first argument is a test suite. The second argument is the layer.
For example:
>>> from plone.testing import layered
>>> def test_suite():
... suite = unittest.TestSuite()
... suite.addTests([
... layered(doctest.DocTestSuite('spaceship.utils'), layer=CONSTITUTION_CLASS_SPACE_SHIP),
... ])
... return suite
This is equivalent to:
>>> def test_suite():
... suite = unittest.TestSuite()
...
... spaceshipUtilTests = doctest.DocTestSuite('spaceship.utils', globs={'layer': CONSTITUTION_CLASS_SPACE_SHIP})
... spaceshipUtilTests.layer = CONSTITUTION_CLASS_SPACE_SHIP
... suite.addTest(spaceshipUtilTests)
...
... return suite
(In this example, we’ve opted to use addTest() to add a single suite, instead of using addTests() to add multiple suites in one go.)
Zope testing tools
Everything described so far in this document relies only on the standard unittest/unittest2 and doctest modules and zope.testing, and you can use this package without any other dependencies.
However, there are also some tools (and layers) available in this package, as well as in other packages, that are specifically useful for testing applications that use various Zope-related frameworks.
Test cleanup
If a test uses a global registry, it may be necessary to clean that registry on set up and tear down of each test fixture. zope.testing provides a mechanism to register cleanup handlers - methods that are called to clean up global state. This can then be invoked in the setUp() and tearDown()` fixture lifecycle methods of a test case.
>>> from zope.testing import cleanup
Let’s say we had a global registry, implemented as a dictionary
>>> SOME_GLOBAL_REGISTRY = {}
If we wanted to clean this up on each test run, we could call clear() on the dict. Since that’s a no-argument method, it is perfect as a cleanup handler.
>>> cleanup.addCleanUp(SOME_GLOBAL_REGISTRY.clear)
We can now use the cleanUp() method to execute all registered cleanups:
>>> cleanup.cleanUp()
This call could be placed in a setUp() and/or tearDown() method in a test class, for example.
Event testing
You may wish to test some code that uses zope.event to fire specific events. zope.component provides some helpers to capture and analyse events.
>>> from zope.component import eventtesting
To use this, you first need to set up event testing. Some of the layers shown below will do this for you, but you can do it yourself by calling the eventtesting.setUp() method, e.g. from your own setUp() method:
>>> eventtesting.setUp()
This simply registers a few catch-all event handlers. Once you have executed the code that is expected to fire events, you can use the getEvents() helper function to obtain a list of the event instances caught:
>>> events = eventtesting.getEvents()
You can now examine events to see what events have been caught since the last cleanup.
getEvents() takes two optional arguments that can be used to filter the returned list of events. The first (event_type) is an interface. If given, only events providing this interface are returned. The second (filter) is a callable taking one argument. If given, it will be called with each captured event. Only those events where the filter function returns True will be included.
The eventtesting module registers a cleanup action as outlined above. When you call cleanup.cleanUp() (or eventtesting.clearEvents(), which is the handler it registers), the events list will be cleared, ready for the next test. Here, we’ll do it manually:
>>> eventtesting.clearEvents()
Mock requests
Many tests require a request object, often with particular request/form variables set. zope.publisher contains a useful class for this purpose.
>>> from zope.publisher.browser import TestRequest
A simple test request can be constructed with no arguments:
>>> request = TestRequest()
To add a body input stream, pass a StringIO or file as the first parameter. To set the environment (request headers), use the environ keyword argument. To simulate a submitted form, use the form keyword argument:
>>> request = TestRequest(form=dict(field1='foo', field2=1))
Note that the form dict contains marshalled form fields, so modifiers like :int or :boolean should not be included in the field names, and values should be converted to the appropriate type.
Registering components
Many test fixtures will depend on having a minimum of Zope Component Architecture (ZCA) components registered. In normal operation, these would probably be registered via ZCML, but in a unit test, you should avoid loading the full ZCML configuration of your package (and its dependencies).
Instead, you can use the Python API in zope.component to register global components instantly. The three most commonly used functions are:
>>> from zope.component import provideAdapter >>> from zope.component import provideUtility >>> from zope.component import provideHandler
See the zope.component documentation for details about how to use these.
When registering global components like this, it is important to avoid test leakage. The cleanup mechanism outlined above can be used to tear down the component registry between each test. See also the plone.testing.zca.UNIT_TESTING layer, described below, which performs this cleanup automatically via the testSetUp()/testTearDown() mechanism.
Alternatively, you can “stack” a new global component registry using the plone.testing.zca.pushGlobalRegistry() and plone.testing.zca.popGlobalRegistry() helpers. This makes it possible to set up and tear down components that are specific to a given layer, and even allow tests to safely call the global component API (or load ZCML - see below) with proper tear-down. See the layer reference below for details.
Loading ZCML
Integration tests often need to load ZCML configuration. This can be achieved using the zope.configuration API.
>>> from zope.configuration import xmlconfig
The xmlconfig module contains two methods for loading ZCML.
xmlconfig.string() can be used to load a literal string of ZCML:
>>> xmlconfig.string("""\
... <configure xmlns="http://namespaces.zope.org/zope" package="plone.testing">
... <include package="zope.component" file="meta.zcml" />
... </configure>
... """)
<zope.configuration.config.ConfigurationMachine object at ...>
Note that we need to set a package (used for relative imports and file locations) explicitly here, using the package attribute of the <configure /> element.
Also note that unless the optional second argument (context) is passed, a new configuration machine will be created every time string() is called. It therefore becomes necessary to explicitly <include /> the files that contain the directives you want to use (the one in zope.component is a common example). Layers that set up ZCML configuration may expose a resource which can be passed as the context parameter, usually called configurationContext - see below.
To load the configuration for a particular package, use xmlconfig.file():
>>> import zope.component
>>> context = xmlconfig.file('meta.zcml', zope.component)
>>> xmlconfig.file('configure.zcml', zope.component, context=context)
<zope.configuration.config.ConfigurationMachine object at ...>
This takes two required arguments: the file name and the module relative to which it is to be found. Here, we have loaded two files: meta.zcml and configure.zcml. The first call to xmlconfig.file() creates and returns a configuration context. We re-use that for the subsequent invocation, so that the directives configured are available.
Installing a Zope 2 product
Some packages (including all those in the Products.* namespace) have the special status of being Zope 2 “products”. These are recorded in a special registry, and may have an initialize() hook in their top-level __init__.py that needs to be called for the package to be fully configured.
Zope 2 will find and execute any products during startup. For testing, we need to explicitly list the products to install. Provided you are using plone.testing with Zope 2, you can use the following:
from plone.testing import z2
with z2.zopeApp() as app:
z2.installProduct(app, 'Products.ZCatalog')
This would normally be used during layer setUp(). Note that the basic Zope 2 application context must have been set up before doing this. The usual way to ensure this, is to use a layer that is based on z2.STARTUP - see below.
To tear down such a layer, you should do:
from plone.testing import z2
with z2.zopeApp() as app:
z2.uninstallProduct(app, 'Products.ZCatalog')
Note:
Unlike the similarly-named function from ZopeTestCase, these helpers will work with any type of product. There is no distinction between a “product” and a “package” (and no installPackage()). However, you must use the full name (Products.*) when registering a product.
Installing a product in this manner is independent of ZCML configuration. However, it is almost always necessary to install the package’s ZCML configuration first.
Functional testing
For functional tests that aim to simulate the browser, you can use zope.testbrowser in a Python test or doctest:
>>> from zope.testbrowser.browser import Browser >>> browser = Browser()
This provides a simple API to simulate browser input, without actually running a web server thread or scripting a live browser (as tools such as Windmill and Selenium do). The downside is that it is not possible to test JavaScript- dependent behaviour.
If you are testing a Zope 2 application, you need to change the import location slightly, and pass the application root to the method:
from plone.testing.z2 import Browser browser = Browser(app)
You can get the application root from the app resource in any of the Zope 2 layers in this package.
Beyond that, the zope.testbrowser documentation should cover how to use the test browser.
Hint: The test browser will usually commit at the end of a request. To avoid test fixture contamination, you should use a layer that fully isolates each test, such as the z2.INTEGRATION_TESTING layer described below.
Layer reference
plone.testing comes with several layers that are available to use directly or extend. These are outlined below.
Zope Component Architecture
The Zope Component Architecture layers are found in the module plone.testing.zca. If you depend on this, you can use the [zca] extra when depending on plone.testing.
Unit testing
Layer: |
plone.testing.zca.UNIT_TESTING |
Class: |
plone.testing.zca.UnitTesting |
Bases: |
None |
Resources: |
None |
This layer does not set up a fixture per se, but cleans up global state before and after each test, using zope.testing.cleanup as described above.
The net result is that each test has a clean global component registry. Thus, it is safe to use the zope.component Python API (provideAdapter(), provideUtility(), provideHandler() and so on) to register components.
Be careful with using this layer in combination with other layers. Because it tears down the component registry between each test, it will clobber any layer that sets up more permanent test fixture in the component registry.
Event testing
Layer: |
plone.testing.zca.EVENT_TESTING |
Class: |
plone.testing.zca.EventTesting |
Bases: |
plone.testing.zca.UNIT_TESTING |
Resources: |
None |
This layer extends the zca.UNIT_TESTING layer to enable the eventtesting support from zope.component. Using this layer, you can import and use zope.component.eventtesting.getEvent to inspect events fired by the code under test.
See above for details.
Layer cleanup
Layer: |
plone.testing.zca.LAYER_CLEANUP |
Class: |
plone.testing.zca.LayerCleanup |
Bases: |
None |
Resources: |
None |
This layer calls the cleanup functions from zope.testing.cleanup on setup and tear-down (but not between each test). It is useful as a base layer for other layers that need as pristine an environment as possible.
Basic ZCML directives
Layer: |
plone.testing.zca.ZCML_DIRECTIVES |
Class: |
plone.testing.zca.ZCMLDirectives |
Bases: |
plone.testing.zca.LAYER_CLEANUP |
Resources: |
configurationContext |
This registers a minimal set of ZCML directives, principally those found in the zope.component package, and makes available a configuration context. This allows custom ZCML to be loaded as described above.
The configurationContext resource should be used when loading custom ZCML. To ensure isolation, you should stack this using the stackConfigurationContext() helper. For example, if you were writing a setUp() method in a layer that had zca.ZCML_DIRECTIVES as a base, you could do:
self['configurationContext'] = context = zca.stackConfigurationContext(self.get('configurationContext'))
xmlconfig.string(someZCMLString, context=context)
This will create a new configuration context with the state of the base layer’s context. On tear-down, you should delete the layer-specific resource:
del self['configurationContext']
Note: If you fail to do this, you may get problems if your layer is torn down and then needs to be set up again later.
See above for more details about loading custom ZCML in a layer or test.
ZCML files helper class
Class: |
plone.testing.zca.ZCMLSandbox |
Resources: |
configurationContext |
The ZCMLSandbox can be instantiated with a filename` and package arguments:
ZCML_SANDBOX = zca.ZCMLSandbox(filename="configure.zcml",
package=my.package)
That layer setUp loads the ZCML file. It avoids the need to using (and understand) configurationContext` and ``globalRegistry until you need more flexibility or modularity for your layer and tests.
See above for more details about loading custom ZCML in a layer or test.
Helper functions
The following helper functions are available in the plone.testing.zca module.
- stackConfigurationContext(context=None)
Create and return a copy of the passed-in ZCML configuration context, or a brand new context if it is None.
The purpose of this is to ensure that if a layer loads some ZCML files (using the zope.configuration API during) its setUp(), the state of the configuration registry (which includes registered directives as well as a list of already imported files, which will not be loaded again even if explicitly included) can be torn down during tearDown().
The usual pattern is to keep the configuration context in a layer resource called configurationContext. In setUp(), you would then use:
self['configurationContext'] = context = zca.stackConfigurationContext(self.get('configurationContext')) # use 'context' to load some ZCMLIn tearDown(), you can then simply do:
del self['configurationContext']
- pushGlobalRegistry(new=None)
Create or obtain a stack of global component registries, and push a new registry to the top of the stack. The net result is that zope.component.getGlobalSiteManager() and (an un-hooked) getSiteManager() will return the new registry instead of the default, module-scope one. From this point onwards, calls to provideAdapter(), provideUtility() and other functions that modify the global registry will use the new registry.
If new is not given, a new registry is created that has the previous global registry (site manager) as its sole base. This has the effect that registrations in the previous default global registry are still available, but new registrations are confined to the new registry.
Warning: If you call this function, you must reciprocally call popGlobalRegistry(). That is, if you “push” a registry during layer setUp(), you must “pop” it during tearDown(). If you “push” during testSetUp(), you must “pop” during testTearDown(). If the calls to push and pop are not balanced, you will leave your global registry in a mess, which is not pretty.
Returns the new default global site manager. Also causes the site manager hook from zope.site to be reset, clearing any local site managers as appropriate.
- popGlobalRegistry()
Pop the global site registry, restoring the previous registry to be the default.
Please heed the warning above: push and pop must be balanced.
Returns the new default global site manager. Also causes the site manager hook from zope.site to be reset, clearing any local site managers as appropriate.
Zope Security
The Zope Security layers build can be found in the module plone.testing.security.
If you depend on this, you can use the [security] extra when depending on plone.testing.
Security checker isolation
Layer: |
plone.testing.security.CHECKERS |
Class: |
plone.testing.security.Checkers |
Bases: |
None |
Resources: |
None |
This layer ensures that security checkers used by zope.security are isolated. Any checkers set up in a child layer will be removed cleanly during tear-down.
Helper functions
The security checker isolation outlined above is managed using two helper functions found in the module plone.testing.security:
- pushCheckers()
Copy the current set of security checkers for later tear-down.
- popCheckers()
Restore the set of security checkers to the state of the most recent call to pushCheckers().
You must keep calls to pushCheckers() and popCheckers() in balance. That usually means that if you call the former during layer setup, you should call the latter during layer tear-down. Ditto for calls during test setup/tear-down or within tests themselves.
Zope Publisher
The Zope Publisher layers build on the Zope Component Architecture layers. They can be found in the module plone.testing.publisher.
If you depend on this, you can use the [publisher] extra when depending on plone.testing.
Publisher directives
Layer: |
plone.testing.publisher.PUBLISHER_DIRECTIVES |
Class: |
plone.testing.publisher.PublisherDirectives |
Bases: |
plone.testing.zca.ZCML_DIRECTIVES |
Resources: |
None |
This layer extends the zca.ZCML_DIRECTIVES layer to install additional ZCML directives in the browser namespace (from zope.app.publisher.browser) as well as those from zope.security. This allows browser views, browser pages and other UI components to be registered, as well as the definition of new permissions.
As with zca.ZCML_DIRECTIVES, you should use the configurationContext resource when loading ZCML strings or files, and the stackConfigurationRegistry() helper to create a layer-specific version of this resource resource. See above.
ZODB
The ZODB layers set up a test fixture with a persistent ZODB. The ZODB instance uses DemoStorage, so it will not interfere with any “live” data.
ZODB layers can be found in the module plone.testing.zodb. If you depend on this, you can use the [zodb] extra when depending on plone.testing.
Empty ZODB sandbox
Layer: |
plone.testing.zodb.EMPTY_ZODB |
Class: |
plone.testing.zodb.EmptyZODB |
Bases: |
None |
Resources: |
zodbDB |
zodbDB (test set-up only) |
|
zodbConnection (test set-up only) |
This layer sets up a simple ZODB sandbox using DemoStorage. The ZODB root object is a simple persistent mapping, available as the resource zodbRoot. The ZODB database object is available as the resource zodbDB. The connection used in the test is available as zodbConnection.
Note that the zodbConnection and zodbRoot resources are created and destroyed for each test. You can use zodbDB (and the open() method) if you are writing a layer based on this one and need to set up a fixture during layer set up. Don’t forget to close the connection before concluding the test setup!
A new transaction is begun for each test, and rolled back (aborted) on test tear-down. This means that so long as you don’t use transaction.commit() explicitly in your code, it should be safe to add or modify items in the ZODB root.
If you want to create a test fixture with persistent data in your own layer based on EMPTY_ZODB, you can use the following pattern:
from plone.layer import Layer
from plone.layer import zodb
class MyLayer(Layer):
defaultBases = (zodb.EMPTY_ZODB,)
def setUp(self):
import transaction
self['zodbDB'] = db = zodb.stackDemoStorage(self.get('zodbDB'), name='MyLayer')
conn = db.open()
root = conn.root()
# modify the root object here
transaction.commit()
conn.close()
def tearDown(self):
self['zodbDB'].close()
del self['zodbDB']
This shadows the zodbDB resource with a new database that uses a new DemoStorage stacked on top of the underlying database storage. The fixture is added to this storage and committed during layer setup. (The base layer test set-up/tear-down will still begin and abort a new transaction for each test). On layer tear-down, the database is closed and the resource popped, leaving the original zodbDB database with the original, pristine storage.
Helper functions
One helper function is available in the plone.testing.zodb module.
- stackDemoStorage(db=None, name=None)
Create a new DemoStorage using the storage from the passed-in database as a base. If db is None, a brand new storage is created.
A name can be given to uniquely identify the storage. It is optional, but it is often useful for debugging purposes to pass the name of the layer.
The usual pattern is:
def setUp(self): self['zodbDB'] = zodb.stackDemoStorage(self.get('zodbDB'), name='MyLayer') def tearDown(self): self['zodbDB'].close() del self['zodbDB']This will shadow the zodbDB resource with an isolated DemoStorage, creating a new one if that resource does not already exist. All existing data continues to be available, but new changes are written to the stacked storage. On tear-down, the stacked database is closed and the resource removed, leaving the original data.
Zope 2
The Zope 2 layers provide test fixtures suitable for testing Zope 2 applications. They set up a Zope 2 application root, install core Zope 2 products, and manage security.
Zope 2 layers can be found in the module plone.testing.z2. If you depend on this, you can use the [z2] extra when depending on plone.testing.
Startup
Layer: |
plone.testing.z2.STARTUP |
Class: |
plone.testing.z2.Startup |
Bases: |
plone.testing.zca.LAYER_CLEANUP |
Resources: |
zodbDB |
configurationContext |
|
host |
|
port |
This layer sets up a Zope 2 environment, and is a required base for all other Zope 2 layers. You cannot run two instances of this layer in parallel, since Zope 2 depends on some module-global state to run, which is managed by this layer.
On set-up, the layer will configure a Zope environment with:
Note: The STARTUP layer is a useful base layer for your own fixtures, but should not be used directly, since it provides no test lifecycle or transaction management. See the “Integration test” and “Functional” test sections below for examples of how to create your own layers.
Debug mode enabled.
ZEO client cache disabled.
Some patches installed, which speed up Zope startup by disabling the help system and some other superfluous aspects of Zope.
One thread (this only really affects the ZSERVER and FTP_SERVER layers).
A pristine database using DemoStorage, exposed as the resource zodbDB. Zope is configured to use this database in a way that will also work if the zodbDB resource is shadowed using the pattern shown above in the description of the zodb.EMPTY_ZODB layer.
A fake hostname and port, exposed as the host and port resource, respectively.
A minimal set of products installed (Products.OFSP and Products.PluginIndexes, both required for Zope to start up).
A stacked ZCML configuration context, exposed as the resource configurationContext. As illustrated above, you should use the zca.stackConfigurationContext() helper to stack your own configuration context if you use this.
A minimal set of global Zope components configured.
Note that unlike a “real” Zope site, products in the Products.* namespace are not automatically loaded, nor is any ZCML.
Integration test
Layer: |
plone.testing.z2.INTEGRATION_TESTING |
Class: |
plone.testing.z2.IntegrationTesting |
Bases: |
plone.testing.z2.STARTUP |
Resources: |
app |
request |
This layer is intended for integration testing against the simple STARTUP fixture. If you want to create your own layer with a more advanced, shared fixture, see “Integration and functional testing with custom fixtures” below.
For each test, it exposes the Zope application root as the resource app. This is wrapped in the request container, so you can do app.REQUEST to acquire a fake request, but the request is also available as the resource request.
A new transaction is begun for each test and rolled back on test tear-down, meaning that so long as the code under test does not explicitly commit any changes, the test may modify the ZODB.
Hint: If you want to set up a persistent test fixture in a layer based on this one (or z2.FUNCTIONAL_TESTING), you can stack a new DemoStorage in a shadowing zodbDB resource, using the pattern described above for the zodb.EMPTY_ZODB layer.
Once you’ve shadowed the zodbDB resource, you can do (e.g. in your layer’s setUp() method):
... with z2.zopeApp() as app: # modify the Zope application rootThe zopeApp() context manager will open a new connection to the Zope application root, accessible here as app. Provided the code within the with block does not raise an exception, the transaction will be committed and the database closed properly upon exiting the block.
Functional testing
Layer: |
plone.testing.z2.FUNCTIONAL_TESTING |
Class: |
plone.testing.z2.FunctionalTesting |
Bases: |
plone.testing.z2.STARTUP |
Resources: |
app |
request |
This layer is intended for functional testing against the simple STARTUP fixture. If you want to create your own layer with a more advanced, shared fixture, see “Integration and functional testing with custom fixtures” below.
As its name implies, this layer is intended mainly for functional end-to-end testing using tools like zope.testbrowser. See also the Browser object as described under “Helper functions” below.
This layer is very similar to INTEGRATION_TESTING, but is not based on it. It sets up the same fixture and exposes the same resources. However, instead of using a simple transaction abort to isolate the ZODB between tests, it uses a stacked DemoStorage for each test. This is slower, but allows test code to perform and explicit commit, as will usually happen in a functional test.
Integration and functional testing with custom fixtures
If you want to extend the STARTUP fixture for use with integration or functional testing, you should use the following pattern:
Create a layer class and a “fixture” base layer instance that has z2.STARTUP (or some intermediary layer, such as z2.ZSERVER_FIXTURE or z2.FTP_SERVER_FIXTURE, shown below) as a base.
Create “end user” layers by instantiating the z2.IntegrationTesting and/or FunctionalTesting classes with this new “fixture” layer as a base.
This allows the same fixture to be used regardless of the “style” of testing, minimising the amount of set-up and tear-down. The “fixture” layers manage the fixture as part of the layer lifecycle. The layer class (IntegrationTesting or FunctionalTesting), manages the test lifecycle, and the test lifecycle only.
For example:
from plone.testing import Layer, z2, zodb
class MyLayer(Layer):
defaultBases = (z2.STARTUP,)
def setUp(self):
# Set up the fixture here
...
def tearDown(self):
# Tear down the fixture here
...
MY_FIXTURE = MyLayer()
MY_INTEGRATION_TESTING = z2.IntegrationTesting(bases=(MY_FIXTURE,), name="MyFixture:Integration")
MY_FUNCTIONAL_TESTING = z2.FunctionalTesting(bases=(MY_FIXTURE,), name="MyFixture:Functional")
(Note that we need to give an explicit, unique name to the two layers that re-use the IntegrationTesting and FunctionalTesting classes.)
In this example, other layers could extend the “MyLayer” fixture by using MY_FIXTURE as a base. Tests would use either MY_INTEGRATION_TESTING or MY_FUNCTIONAL_TESTING as appropriate. However, even if both these two layers were used, the fixture in would``MY_FIXTURE only be set up once.
Note: If you implement the testSetUp() and testTearDown() test lifecycle methods in your “fixture” layer (e.g. in the the MyLayer class above), they will execute before the corresponding methods from IntegrationTesting and FunctionalTesting. Hence, they cannot use those layers’ resources (app and request).
It may be preferable, therefore, to have your own “test lifecycle” layer classes that subclass IntegrationTesting and/or FunctionalTesting and call base class methods as appropriate. plone.app.testing takes this approach, for example.
HTTP ZServer thread (fixture only)
Layer: |
plone.testing.z2.ZSERVER_FIXTURE |
Class: |
plone.testing.z2.ZServer |
Bases: |
plone.testing.z2.STARTUP |
Resources: |
host |
port |
This layer extends the z2.STARTUP layer to start the Zope HTTP server in a separate thread. This means the test site can be accessed through a web browser, and can thus be used with tools like Windmill or Selenium.
Note: This layer is useful as a fixture base layer only, because it does not manage the test lifecycle. Use the ZSERVER layer if you want to execute functional tests against this fixture.
The ZServer’s hostname (normally localhost) is available through the resource host, whilst the port it is running on is available through the resource port.
Hint: Whilst the layer is set up, you can actually access the test Zope site through a web browser. The default URL will be http://localhost:55001.
HTTP ZServer functional testing
Layer: |
plone.testing.z2.ZSERVER |
Class: |
plone.testing.z2.FunctionalTesting |
Bases: |
plone.testing.z2.ZSERVER_FIXTURE |
Resources: |
This layer provides the functional testing lifecycle against the fixture set up by the z2.ZSERVER_FIXTURE layer.
You can use this to run “live” functional tests against a basic Zope site. You should not use it as a base. Instead, create your own “fixture” layer that extends z2.ZSERVER_FIXTURE, and then instantiate the FunctionalTesting class with this extended fixture layer as a base, as outlined above.
FTP server thread (fixture only)
Layer: |
plone.testing.z2.FTP_SERVER_FIXTURE |
Class: |
plone.testing.z2.FTPServer |
Bases: |
plone.testing.z2.STARTUP |
Resources: |
host |
port |
This layer is the FTP server equivalent of the ZSERVER_FIXTURE layer. It can be used to functionally test Zope servers.
Note: This layer is useful as a fixture base layer only, because it does not manage the test lifecycle. Use the FTP_SERVER layer if you want to execute functional tests against this fixture.
Hint: Whilst the layer is set up, you can actually access the test Zope site through an FTP client. The default URL will be ftp://localhost:55002.
Warning: Do not run the FTP_SERVER and ZSERVER layers concurrently in the same process.
If you need both ZServer and FTPServer running together, you can subclass the ZServer layer class (like the FTPServer layer class does) and implement the setUpServer() and tearDownServer() methods to set up and close down two servers on different ports. They will then share a main loop.
FTP server functional testing
Layer: |
plone.testing.z2.FTP_SERVER |
Class: |
plone.testing.z2.FunctionalTesting |
Bases: |
plone.testing.z2.FTP_SERVER_FIXTURE |
Resources: |
This layer provides the functional testing lifecycle against the fixture set up by the z2.FTP_SERVER_FIXTURE layer.
You can use this to run “live” functional tests against a basic Zope site. You should not use it as a base. Instead, create your own “fixture” layer that extends z2.FTP_SERVER_FIXTURE, and then instantiate the FunctionalTesting class with this extended fixture layer as a base, as outlined above.
Helper functions
Several helper functions are available in the plone.testing.z2 module.
- zopeApp(db=None, conn=Non, environ=None)
This function can be used as a context manager for any code that requires access to the Zope application root. By using it in a with block, the database will be opened, and the application root will be obtained and request-wrapped. When exiting the with block, the transaction will be committed and the database properly closed, unless an exception was raised:
with z2.zopeApp() as app: # do something with appIf you want to use a specific database or database connection, pass either the db or conn arguments. If the context manager opened a new connection, it will close it, but it will not close a connection passed with conn.
To set keys in the (fake) request environment, pass a dictionary of environment values as environ.
Note that zopeApp() should not normally be used in tests or test set-up/tear-down, because the INTEGRATOIN_TEST and FUNCTIONAL_TESTING layers both manage the application root (as the app resource) and close it for you. It is very useful in layer setup, however.
- installProduct(app, product, quiet=False)
Install a Zope 2 style product, ensuring that its initialize() function is called. The product name must be the full dotted name, e.g. plone.app.portlets or Products.CMFCore. If quiet is true, duplicate registrations will be ignored silently, otherwise a message is logged.
To get hold of the application root, passed as the app argument, you would normally use the zopeApp() context manager outlined above.
- uninstallProduct(app, product, quiet=False)
This is the reciprocal of installProduct(), normally used during layer tear-down. Again, you should use zopeApp() to obtain the application root.
- login(userFolder, userName)
Create a new security manager that simulates being logged in as the given user. userFolder is an acl_users object, e.g. app['acl_users'] for the root user folder.
- logout()
Simulate being the anonymous user by unsetting the security manager.
- setRoles(userFolder, userName, roles)
Set the roles of the given user in the given user folder to the given list of roles.
- makeTestRequest()
Create a fake Zope request.
- addRequestContainer(app, environ=None)
Create a fake request and wrap the given object (normally an application root) in a RequestContainer with this request. This makes acquisition of app.REQUEST possible. To initialise the request environment with non-default values, pass a dictionary as environ.
Note that this method is rarely used, because both the zopeApp() context manager and the layer set-up/tear-down for z2.INTEGRATION_TESTING and z2.FUNCTIONAL_TESTING will wrap the app object before exposing it.
- Browser(app)
Obtain a test browser client, for use with zope.testbrowser. You should use this in conjunction with the z2.FUNCTIONAL_TESTING layer or a derivative. You must pass the app root, usually obtained from the app resource of the layer, e.g.:
app = self.layer['app'] browser = z2.Browser(app)
You can then use browser as described in the zope.testbrowser documentation.
Bear in mind that the test browser runs separately from the test fixture. In particular, calls to helpers such as login() or logout() do not affect the state that the test browser sees. If you want to set up a persistent fixture (e.g. test content), you can do so before creating the test browser, but you will need to explicitly commit your changes, with:
import transaction transaction.commit()
Changelog
4.0.10 (2014-02-11)
Read ‘FTPSERVER_HOST’ and ‘FTPSERVER_PORT’ from the environment variables if possible. This allows us to run tests in parallel on CI servers. [timo]
4.0.9 (2014-01-28)
Replace deprecated Zope2VocabularyRegistry import. [timo]
4.0.8 (2013-03-05)
Factor test request creation out of addRequestContainer into makeTestRequest. [davisagli]
4.0.7 (2012-12-09)
Fix quoting of urls by the testbrowser. [do3cc]
4.0.6 (2012-10-15)
Update manifest.in to include content in src directory. [esteele]
4.0.5 (2012-10-15)
Fixed an issue where a query string would be unquoted twice; once while setting up the HTTP request and once in the handler (the publisher). [malthe]
4.0.4 (2012-08-04)
Fixed the cache reset code. In some situations the function does not have any defaults, so we shouldn’t try to clear out the app reference. [malthe]
4.0.3 (2011-11-24)
Fixed class names in documentation to match code. [icemac]
4.0.2 (2011-08-31)
The defaults of the ZPublisher.Publish.get_module_info function cache a reference to the app, so make sure that gets reset when tearing down the app. This fixes a problem where the testbrowser in the second functional layer to be set up accessed the database from the first functional layer. [davisagli]
4.0.1 - 2011-05-20
Moved readme file containing tests into the package, so tests can be run from released source distributions. Closes http://dev.plone.org/plone/ticket/11821. [hannosch]
Relicense under BSD license. See http://plone.org/foundation/materials/foundation-resolutions/plone-framework-components-relicensing-policy [davisagli]
4.0 - 2011-05-13
Release 4.0 Final. [esteele]
Add MANIFEST.in. [WouterVH]
4.0a6 - 2011-04-06
Fixed Browser cookies retrieval with Zope 2.13. [vincentfretin]
Add ZCMLSandbox layer to load a ZCML file; replaces setUpZcmlFiles and tearDownZcmlFiles helper functions. [gotcha]
4.0a5 - 2011-03-02
Handle test failures due to userFolderAddUser returning the user object in newer versions of Zope. [esteele]
Add setUpZcmlFiles and tearDownZcmlFiles helpers to enable loading of ZCML files without too much boilerplate. [gotcha]
Add some logging. [gotcha]
Add the [security] extra, to provide tear-down of security checkers. [optilude]
Let the IntegrationTesting and FunctionalTesting lifecycle layers set up request PARENTS and, if present, wire up zope.globalrequest. [optilude]
Make the test browser support IStreamIterators [optilude]
4.0a4 - 2011-01-11
Make sure ZCML doesn’t load during App startup in Zope 2.13. [davisagli]
4.0a3 - 2010-12-14
Ignore the testinghome configuration setting if present. [stefan]
Use the new API for getting the packages_to_initialize list in Zope 2.13. [davisagli]
De-duplicate _register_monkies and _meta_type_regs in the correct module on teardown of the Startup layer in Zope 2.13. [davisagli]
Allow doctest suites from zope.testing to work with plone.testing.layer.layered. Previously, only doctest suites from the stdlib would see the layer global. [nouri]
Changed documentation to advertise the coverage library for running coverage tests instead of the built-in zope.testing support. This also avoids using z3c.coverage. The coverage tests now run at the same speed as a normal test run, making it more likely to get executed frequently. [hannosch]
Correct license to GPL version 2 only. [hannosch]
Fix some user id vs name confusion. [rossp]
Add the option to specify ZServer host and port through environment variables - ZSERVER_HOST and ZSERVER_PORT). [esteele]
1.0a2 - 2010-09-05
Fix a problem that would cause <meta:redefinePermission /> to break. In particular fixes the use of the zope2.Public permission. [optilude]
Set the security implementation to “Python” for easier debugging during the z2.STARTUP layer. [optilude]
Initialize Five in the z2.Startup layer, pushing a Zope2VocabularyRegistry on layer set-up and restoring the previous one upon tear-down. [dukebody]
1.0a1 - 2010-08-01
Initial release
Detailed documentation
Layer base class
This package provides a layer base class which can be used by the test runner. It is available as a convenience import from the package root.
>>> from plone.testing import Layer
A layer may be instantiated directly, though in this case the name argument is required (see below).
>>> NULL_LAYER = Layer(name="Null layer")
This is not very useful on its own. It has an empty list of bases, and each of the layer lifecycle methods does nothing.
>>> NULL_LAYER.__bases__ () >>> NULL_LAYER.__name__ 'Null layer' >>> NULL_LAYER.__module__ 'plone.testing.layer'>>> NULL_LAYER.setUp() >>> NULL_LAYER.testSetUp() >>> NULL_LAYER.tearDown() >>> NULL_LAYER.testTearDown()
Just about the only reason to use this directly (i.e. not as a base class) is to group together other layers.
>>> SIMPLE_LAYER = Layer(bases=(NULL_LAYER,), name="Simple layer", module='plone.testing.tests')
Here, we’ve also set the module name directly. The default for all layers is to take the module name from the stack frame where the layer was instantiated. In doctests, that doesn’t work, though, so we fall back on the module name of the layer class. The two are often the same, of course.
This layer now has the bases, name and module we set:
>>> SIMPLE_LAYER.__bases__ (<Layer 'plone.testing.layer.Null layer'>,)>>> SIMPLE_LAYER.__name__ 'Simple layer'>>> SIMPLE_LAYER.__module__ 'plone.testing.tests'
The name argument is required when using Layer directly (but not when using a subclass):
>>> Layer((SIMPLE_LAYER,)) Traceback (most recent call last): ... ValueError: The `name` argument is required when instantiating `Layer` directly>>> class NullLayer(Layer): ... pass >>> NullLayer() <Layer '__builtin__.NullLayer'>
Using Layer as a base class
The usual pattern is to use Layer as a base class for a custom layer. This can then override the lifecycle methods as appropriate, as well as set a default list of bases.
>>> class BaseLayer(Layer): ... ... def setUp(self): ... print "Setting up base layer" ... ... def tearDown(self): ... print "Tearing down base layer">>> BASE_LAYER = BaseLayer()
The layer name and module are taken from the class.
>>> BASE_LAYER.__bases__ () >>> BASE_LAYER.__name__ 'BaseLayer' >>> BASE_LAYER.__module__ '__builtin__'
We can now create a new layer that has this one as a base. We can do this in the instance constructor, as shown above, but the most common pattern is to set the default bases in the class body, using the variable defaultBases.
We’ll also set the default name explicitly here by passing a name to the the super-constructor. This is mostly cosmetic, but may be desirable if the class name would be misleading in the test runner output.
>>> class ChildLayer(Layer): ... defaultBases = (BASE_LAYER,) ... ... def __init__(self, bases=None, name='Child layer', module=None): ... super(ChildLayer, self).__init__(bases, name, module) ... ... def setUp(self): ... print "Setting up child layer" ... ... def tearDown(self): ... print "Tearing down child layer">>> CHILD_LAYER = ChildLayer()
Notice how the bases have now been set using the value in defaultBases.
>>> CHILD_LAYER.__bases__ (<Layer '__builtin__.BaseLayer'>,) >>> CHILD_LAYER.__name__ 'Child layer' >>> CHILD_LAYER.__module__ '__builtin__'
Overriding the default list of bases
We can override the list of bases on a per-instance basis. This may be dangerous, i.e. the layer is likely to expect that its bases are set up. Sometimes, it may be useful to inject a new base, however, especially when re-using layers from other packages.
The new list of bases is passed to the constructor. When creating a second instance of a layer (most layers are global singletons created only once), it’s useful to give the new instance a unique name, too.
>>> NEW_CHILD_LAYER = ChildLayer(bases=(SIMPLE_LAYER, BASE_LAYER,), name='New child')>>> NEW_CHILD_LAYER.__bases__ (<Layer 'plone.testing.tests.Simple layer'>, <Layer '__builtin__.BaseLayer'>) >>> NEW_CHILD_LAYER.__name__ 'New child' >>> NEW_CHILD_LAYER.__module__ '__builtin__'
Inconsistent bases
Layer bases are maintained in an order that is semantically equivalent to the “method resolution order” Python maintains for base classes. We can get this from the baseResolutionOrder attribute:
>>> CHILD_LAYER.baseResolutionOrder (<Layer '__builtin__.Child layer'>, <Layer '__builtin__.BaseLayer'>)>>> NEW_CHILD_LAYER.baseResolutionOrder (<Layer '__builtin__.New child'>, <Layer 'plone.testing.tests.Simple layer'>, <Layer 'plone.testing.layer.Null layer'>, <Layer '__builtin__.BaseLayer'>)
As with Python classes, it is possible to construct an invalid set of bases. In this case, layer instantiation will fail.
>>> INCONSISTENT_BASE1 = Layer(name="Inconsistent 1") >>> INCONSISTENT_BASE2 = Layer((INCONSISTENT_BASE1,), name="Inconsistent 1") >>> INCONSISTENT_BASE3 = Layer((INCONSISTENT_BASE1, INCONSISTENT_BASE2,), name="Inconsistent 1") Traceback (most recent call last): ... TypeError: Inconsistent layer hierarchy!
Using the resource manager
Layers are also resource managers. Resources can be set, retrieved and deleted using dictionary syntax. Resources in base layers are available in child layers. When an item is set on a child layer, it shadows any items with the same key in any base layer (until it is deleted), but the original item still exists.
Let’s create a somewhat complex hierarchy of layers that all set resources under a key 'foo' in their setUp() methods.
>>> class Layer1(Layer): ... def setUp(self): ... self['foo'] = 1 ... def tearDown(self): ... del self['foo'] >>> LAYER1 = Layer1()>>> class Layer2(Layer): ... defaultBases = (LAYER1,) ... def setUp(self): ... self['foo'] = 2 ... def tearDown(self): ... del self['foo'] >>> LAYER2 = Layer2()>>> class Layer3(Layer): ... def setUp(self): ... self['foo'] = 3 ... def tearDown(self): ... del self['foo'] >>> LAYER3 = Layer3()>>> class Layer4(Layer): ... defaultBases = (LAYER2, LAYER3,) ... def setUp(self): ... self['foo'] = 4 ... def tearDown(self): ... del self['foo'] >>> LAYER4 = Layer4()Important: Resources that are created in setUp() must be deleted in tearDown(). Similarly, resources created in testSetUp() must be deleted in testTearDown(). This ensures resources are properly stacked and do not leak between layers.
If a test was using LAYER4, the test runner would call each setup step in turn, starting with the “deepest” layer. We’ll simulate that here, so that each of the resources is created.
>>> LAYER1.setUp() >>> LAYER2.setUp() >>> LAYER3.setUp() >>> LAYER4.setUp()
The layers are ordered in a known “resource resolution order”, which is used to determine in which order the layers shadow one another. This is based on the same algorithm as Python’s method resolution order.
>>> LAYER4.baseResolutionOrder (<Layer '__builtin__.Layer4'>, <Layer '__builtin__.Layer2'>, <Layer '__builtin__.Layer1'>, <Layer '__builtin__.Layer3'>)
When fetching and item from a layer, it will be obtained according to the resource resolution order.
>>> LAYER4['foo'] 4
This is not terribly interesting, since LAYER4 has the resource 'foo' set directly. Let’s tear down the layer (which deletes the resource) and see what happens.
>>> LAYER4.tearDown() >>> LAYER4['foo'] 2
We can continue up the chain:
>>> LAYER2.tearDown() >>> LAYER4['foo'] 1>>> LAYER1.tearDown() >>> LAYER4['foo'] 3
Once we’ve deleted the last key, we’ll get a KeyError:
>>> LAYER3.tearDown() >>> LAYER4['foo'] Traceback (most recent call last): ... KeyError: 'foo'
To guard against this, we can use the get() method.
>>> LAYER4.get('foo', -1)
-1
We can also test with ‘in’:
>>> 'foo' in LAYER4 False
To illustrate that this indeed works, let’s set the resource back on one of the bases.
>>> LAYER3['foo'] = 10
>>> LAYER4.get('foo', -1)
10
Let’s now consider a special case: a base layer sets up a resource in layer setup, and uses it in test setup. A child layer then shadows this resource in its own layer setup method. In this case, we want the base layer’s testSetUp() to use the shadowed version that the child provided.
(This is similar to how instance variables work: a base class may set an attribute on self and use it in a method. If a subclass then sets the same attribute to a different value and the base class method is called on an instance of the subclass, the base class attribute is used).
Hint: If you definitely need to access the “original” resource in your testSetUp()/testTearDown() methods, you can store a reference to the resource as a layer instance variable:
self.someResource = self['someResource'] = SomeResource()self.someResource will now be the exact resource created here, whereas self['someResource'] will retain the layer shadowing semantics. In most cases, you probably don’t want to do this, allowing child layers to supply overridden versions of resources as appropriate.
First, we’ll create some base layers. We want to demonstrate having two “branches” of bases that both happen to define the same resource.
>>> class ResourceBaseLayer1(Layer): ... def setUp(self): ... self['resource'] = "Base 1" ... def testSetUp(self): ... print self['resource'] ... def tearDown(self): ... del self['resource']>>> RESOURCE_BASE_LAYER1 = ResourceBaseLayer1()>>> class ResourceBaseLayer2(Layer): ... defaultBases = (RESOURCE_BASE_LAYER1,) ... def testSetUp(self): ... print self['resource']>>> RESOURCE_BASE_LAYER2 = ResourceBaseLayer2()>>> class ResourceBaseLayer3(Layer): ... def setUp(self): ... self['resource'] = "Base 3" ... def testSetUp(self): ... print self['resource'] ... def tearDown(self): ... del self['resource']>>> RESOURCE_BASE_LAYER3 = ResourceBaseLayer3()
We’ll then create the child layer that overrides this resource.
>>> class ResourceChildLayer(Layer): ... defaultBases = (RESOURCE_BASE_LAYER2, RESOURCE_BASE_LAYER3) ... def setUp(self): ... self['resource'] = "Child" ... def testSetUp(self): ... print self['resource'] ... def tearDown(self): ... del self['resource']>>> RESOURCE_CHILD_LAYER = ResourceChildLayer()
We’ll first set up the base layers on their own and simulate two tests.
A test with RESOURCE_BASE_LAYER1 only would look like this:
>>> RESOURCE_BASE_LAYER1.setUp()>>> RESOURCE_BASE_LAYER1.testSetUp() Base 1 >>> RESOURCE_BASE_LAYER1.testTearDown()>>> RESOURCE_BASE_LAYER1.tearDown()
A test with RESOURCE_BASE_LAYER2 would look like this:
>>> RESOURCE_BASE_LAYER1.setUp() >>> RESOURCE_BASE_LAYER2.setUp()>>> RESOURCE_BASE_LAYER1.testSetUp() Base 1 >>> RESOURCE_BASE_LAYER2.testSetUp() Base 1 >>> RESOURCE_BASE_LAYER2.testTearDown() >>> RESOURCE_BASE_LAYER1.testTearDown()>>> RESOURCE_BASE_LAYER2.tearDown() >>> RESOURCE_BASE_LAYER1.tearDown()
A test with RESOURCE_BASE_LAYER3 only would look like this:
>>> RESOURCE_BASE_LAYER3.setUp()>>> RESOURCE_BASE_LAYER3.testSetUp() Base 3 >>> RESOURCE_BASE_LAYER3.testTearDown()>>> RESOURCE_BASE_LAYER3.tearDown()
Now let’s set up the child layer and simulate another test. We should now be using the shadowed resource.
>>> RESOURCE_BASE_LAYER1.setUp() >>> RESOURCE_BASE_LAYER2.setUp() >>> RESOURCE_BASE_LAYER3.setUp() >>> RESOURCE_CHILD_LAYER.setUp()>>> RESOURCE_BASE_LAYER1.testSetUp() Child >>> RESOURCE_BASE_LAYER2.testSetUp() Child >>> RESOURCE_BASE_LAYER3.testSetUp() Child >>> RESOURCE_CHILD_LAYER.testSetUp() Child>>> RESOURCE_CHILD_LAYER.testTearDown() >>> RESOURCE_BASE_LAYER3.testTearDown() >>> RESOURCE_BASE_LAYER2.testTearDown() >>> RESOURCE_BASE_LAYER1.testTearDown()
Finally, we’ll tear down the child layer again and simulate another test. we should have the original resources back. Note that the first and third layers no longer share a resource, since they don’t have a common ancestor.
>>> RESOURCE_CHILD_LAYER.tearDown()>>> RESOURCE_BASE_LAYER1.testSetUp() Base 1 >>> RESOURCE_BASE_LAYER2.testSetUp() Base 1 >>> RESOURCE_BASE_LAYER2.testTearDown() >>> RESOURCE_BASE_LAYER1.testTearDown()>>> RESOURCE_BASE_LAYER3.testSetUp() Base 3 >>> RESOURCE_BASE_LAYER3.testTearDown()
Finally, we’ll tear down the remaining layers..
>>> RESOURCE_BASE_LAYER3.tearDown() >>> RESOURCE_BASE_LAYER2.tearDown() >>> RESOURCE_BASE_LAYER1.tearDown()
Asymmetric deletion
It is an error to create or shadow a resource in a set-up lifecycle method and not delete it again in the tear-down. It is also an error to delete a resource that was not explicitly created. These two layers break those roles:
>>> class BadLayer1(Layer): ... def setUp(self): ... pass ... def tearDown(self): ... del self['foo'] >>> BAD_LAYER1 = BadLayer1()>>> class BadLayer2(Layer): ... defaultBases = (BAD_LAYER1,) ... def setUp(self): ... self['foo'] = 1 ... self['bar'] = 2 >>> BAD_LAYER2 = BadLayer2()
Let’s simulate a test that uses BAD_LAYER2:
>>> BAD_LAYER1.setUp() >>> BAD_LAYER2.setUp()>>> BAD_LAYER1.testSetUp() >>> BAD_LAYER2.testSetUp()>>> BAD_LAYER2.testTearDown() >>> BAD_LAYER1.testTearDown()>>> BAD_LAYER2.tearDown() >>> BAD_LAYER1.tearDown() Traceback (most recent call last): ... KeyError: 'foo'
Here, we’ve got an error in the base layer. This is because the resource is actually associated with the layer that first created it, in this case BASE_LAYER2. This one remains intact and orphaned:
>>> 'foo' in BAD_LAYER2._resources True >>> 'bar' in BAD_LAYER2._resources True
Doctest layer helper
The doctest module is not aware of zope.testing’s layers concept. Therefore, the syntax for creating a doctest with a layer and adding it to a test suite is somewhat contrived: the test suite has to be created first, and then the layer attribute set on it:
>>> class DoctestLayer(Layer): ... pass >>> DOCTEST_LAYER = DoctestLayer()>>> import unittest2 as unittest >>> import doctest>>> def test_suite(): ... suite = unittest.TestSuite() ... layerDoctest = doctest.DocFileSuite('layer.txt', package='plone.testing') ... layerDoctest.layer = DOCTEST_LAYER ... suite.addTest(layerDoctest) ... return suite>>> suite = test_suite() >>> tests = list(suite) >>> len(tests) 1 >>> tests[0].layer is DOCTEST_LAYER True
To make this a little easier - especially when setting up multiple tests - a helper function called layered is provided:
>>> from plone.testing import layered>>> def test_suite(): ... suite = unittest.TestSuite() ... suite.addTests([ ... layered(doctest.DocFileSuite('layer.txt', package='plone.testing'), layer=DOCTEST_LAYER), ... # repeat with more suites if necessary ... ]) ... return suite
This does the same as the sample above.
>>> suite = test_suite() >>> tests = list(suite) >>> len(tests) 1 >>> tests[0].layer is DOCTEST_LAYER True
In addition, a ‘layer’ glob is added to each test in the suite. This allows the test to access layer resources.
>>> len(list(tests[0])) 1 >>> list(tests[0])[0]._dt_test.globs['layer'] is DOCTEST_LAYER True
Zope Component Architecture layers
The ZCA layers are found in the module plone.testing.zca:
>>> from plone.testing import zca
For testing, we need a testrunner
>>> from zope.testing.testrunner import runner
Unit testing
The UNIT_TESTING layer is used to set up a clean component registry between each test. It uses zope.testing.cleanup to clean up all global state.
It has no bases:
>>> "%s.%s" % (zca.UNIT_TESTING.__module__, zca.UNIT_TESTING.__name__,) 'plone.testing.zca.UnitTesting'>>> zca.UNIT_TESTING.__bases__ ()
The component registry is cleaned up between each test.
>>> from zope.interface import Interface >>> from zope.component import provideUtility>>> class DummyUtility(object): ... def __init__(self, name): ... self.name = name ... def __repr__(self): ... return "<%s>" % self.name>>> provideUtility(DummyUtility("Dummy"), provides=Interface, name="test-dummy")>>> from zope.component import queryUtility >>> queryUtility(Interface, name="test-dummy") <Dummy>
Layer setup does nothing.
>>> options = runner.get_options([], [])
>>> setupLayers = {}
>>> runner.setup_layer(options, zca.UNIT_TESTING, setupLayers)
Set up plone.testing.zca.UnitTesting in ... seconds.
Let’s now simulate a test. Before any test setup has happened, our previously registered utility is still there.
>>> queryUtility(Interface, name="test-dummy") <Dummy>
On test setup, it disappears.
>>> zca.UNIT_TESTING.testSetUp()>>> queryUtility(Interface, name="test-dummy") is None True
The test would now execute. It may register some components.
>>> provideUtility(DummyUtility("Dummy2"), provides=Interface, name="test-dummy")
>>> queryUtility(Interface, name="test-dummy")
<Dummy2>
On test tear-down, this disappears.
>>> zca.UNIT_TESTING.testTearDown()>>> queryUtility(Interface, name="test-dummy") is None True
Layer tear-down does nothing.
>>> runner.tear_down_unneeded(options, [], setupLayers) Tear down plone.testing.zca.UnitTesting in ... seconds.
Event testing
The EVENT_TESTING layer extends the UNIT_TESTING layer to add the necessary registrations for zope.component.eventtesting to work.
>>> "%s.%s" % (zca.EVENT_TESTING.__module__, zca.EVENT_TESTING.__name__,) 'plone.testing.zca.EventTesting'>>> zca.EVENT_TESTING.__bases__ (<Layer 'plone.testing.zca.UnitTesting'>,)
Before the test, the component registry is empty and getEvents() returns nothing, even if an event is fired.
>>> from zope.component.eventtesting import getEvents>>> class DummyEvent(object): ... def __repr__(self): ... return "<Dummy event>">>> from zope.event import notify >>> notify(DummyEvent())>>> getEvents() []
Layer setup does nothing.
>>> options = runner.get_options([], [])
>>> setupLayers = {}
>>> runner.setup_layer(options, zca.EVENT_TESTING, setupLayers)
Set up plone.testing.zca.UnitTesting in ... seconds.
Set up plone.testing.zca.EventTesting in ... seconds.
Let’s now simulate a test. On test setup, the event testing list is emptied.
>>> zca.UNIT_TESTING.testSetUp() >>> zca.EVENT_TESTING.testSetUp()>>> getEvents() []
The test would now execute. It may fire some events, which would show up in the event testing list.
>>> notify(DummyEvent()) >>> getEvents() [<Dummy event>]
On test tear-down, the list is emptied again
>>> zca.EVENT_TESTING.testTearDown() >>> zca.UNIT_TESTING.testTearDown()>>> getEvents() []
Layer tear-down does nothing.
>>> runner.tear_down_unneeded(options, [], setupLayers) Tear down plone.testing.zca.EventTesting in ... seconds. Tear down plone.testing.zca.UnitTesting in ... seconds.
Layer cleanup
The LAYER_CLEANUP layer is used to set up a clean component registry at the set-up and tear-down of a layer. It uses zope.testing.cleanup to clean up all global state.
It has no bases:
>>> "%s.%s" % (zca.LAYER_CLEANUP.__module__, zca.LAYER_CLEANUP.__name__,) 'plone.testing.zca.LayerCleanup'>>> zca.LAYER_CLEANUP.__bases__ ()
The component registry is cleaned up on layer set-up and tear-down (but not between tests).
>>> from zope.interface import Interface >>> from zope.component import provideUtility>>> class DummyUtility(object): ... def __init__(self, name): ... self.name = name ... def __repr__(self): ... return "<%s>" % self.name>>> provideUtility(DummyUtility("Dummy"), provides=Interface, name="test-dummy")>>> from zope.component import queryUtility >>> queryUtility(Interface, name="test-dummy") <Dummy>>>> options = runner.get_options([], []) >>> setupLayers = {} >>> runner.setup_layer(options, zca.LAYER_CLEANUP, setupLayers) Set up plone.testing.zca.LayerCleanup in ... seconds.>>> queryUtility(Interface, name="test-dummy") is None True
A sub-layer may register additional components:
>>> provideUtility(DummyUtility("Dummy2"), provides=Interface, name="test-dummy2")
Let’s now simulate a test. Test setup and tear-down does nothing.
>>> zca.LAYER_CLEANUP.testSetUp()>>> queryUtility(Interface, name="test-dummy") is None True >>> queryUtility(Interface, name="test-dummy2") <Dummy2>>>> zca.LAYER_CLEANUP.testTearDown()>>> queryUtility(Interface, name="test-dummy") is None True >>> queryUtility(Interface, name="test-dummy2") <Dummy2>
On tear-down, the registry is cleaned again.
>>> runner.tear_down_unneeded(options, [], setupLayers) Tear down plone.testing.zca.LayerCleanup in ... seconds.>>> queryUtility(Interface, name="test-dummy") is None True >>> queryUtility(Interface, name="test-dummy2") is None True
Basic ZCML directives
The ZCML_DIRECTIVES layer creates a ZCML configuration context with the basic zope.component directives available. It extends the LAYER_CLEANUP layer.
>>> "%s.%s" % (zca.ZCML_DIRECTIVES.__module__, zca.ZCML_DIRECTIVES.__name__,) 'plone.testing.zca.ZCMLDirectives'>>> zca.ZCML_DIRECTIVES.__bases__ (<Layer 'plone.testing.zca.LayerCleanup'>,)
Before the test, we cannot use e.g. a <utility /> directive without loading the necessary meta.zcml files.
>>> from zope.configuration import xmlconfig
>>> xmlconfig.string("""\
... <configure package="plone.testing" xmlns="http://namespaces.zope.org/zope">
... <utility factory=".tests.DummyUtility" provides="zope.interface.Interface" name="test-dummy" />
... </configure>""")
Traceback (most recent call last):
...
ZopeXMLConfigurationError: File "<string>", line 2.4
ConfigurationError: ('Unknown directive', u'http://namespaces.zope.org/zope', u'utility')
Layer setup creates a configuration context we can use to load further configuration.
>>> options = runner.get_options([], [])
>>> setupLayers = {}
>>> runner.setup_layer(options, zca.ZCML_DIRECTIVES, setupLayers)
Set up plone.testing.zca.LayerCleanup in ... seconds.
Set up plone.testing.zca.ZCMLDirectives in ... seconds.
Let’s now simulate a test that uses this configuration context to load the same ZCML string.
>>> zca.ZCML_DIRECTIVES.testSetUp()>>> context = zca.ZCML_DIRECTIVES['configurationContext'] # would normally be self.layer['configurationContext'] >>> xmlconfig.string("""\ ... <configure package="plone.testing" xmlns="http://namespaces.zope.org/zope"> ... <utility factory=".tests.DummyUtility" provides="zope.interface.Interface" name="test-dummy" /> ... </configure>""", context=context) is context True
The utility is now registered:
>>> queryUtility(Interface, name="test-dummy") <Dummy utility>>>> zca.UNIT_TESTING.testTearDown()
Note that normally, we’d combine this with the UNIT_TESTING layer to tear down the component architecture as well.
Layer tear-down deletes the configuration context.
>>> runner.tear_down_unneeded(options, [], setupLayers) Tear down plone.testing.zca.ZCMLDirectives in ... seconds.>>> zca.ZCML_DIRECTIVES.get('configurationContext', None) is None True
Configuration registry sandboxing
For simple unit tests, the full cleanup performed between each test using the UNIT_TESTING layer is undoubtedly the safest and most convenient way to ensure proper isolation of tests using the global component architecture. However, if you are writing a complex layer that sets up a lot of components, you may wish to keep some components registered at the layer level, whilst still allowing tests and sub-layers to register their own components in isolation.
This is a tricky problem, because the default ZCML directives and APIs (provideAdapter(), provideUtility() and so on) explicitly work on a single global adapter registry object. To get around this, you can use two helper methods in the zca module to push a new global component registry before registering components, and pop the registry after. Registries are stacked, so the components registered in a “lower” registry are automatically available in a “higher” registry.
Let’s illustrate this with a layer that stacks two new global registries. The first registry is specific to the layer, and is used to house the components registered at the layer level. The second registry is set up and torn down for each test, allowing tests to register their own components freely.
First, we’ll create a simple dummy utility to illustrate registrations.
>>> from zope.interface import Interface, implements>>> class IDummyUtility(Interface): ... pass >>> class DummyUtility(object): ... implements(IDummyUtility) ... def __init__(self, name): ... self.name = name ... def __repr__(self): ... return "<DummyUtility %s>" % self.name
The two key methods are:
zca.pushGlobalRegistry(), which creates a new global registry.
zca.popGlobalRegistry(), which restores the previous global registry.
Warning: You must balance your calls to these methods. If you call pushGlobalRegistry() in setUp(), call popGlobalRegistry() in tearDown(). Ditto for testSetUp() and testTearDown().
Let’s now create our layer.
>>> from zope.component import provideUtility >>> from plone.testing import Layer >>> from plone.testing import zca>>> class ComponentSandbox(Layer): ... def setUp(self): ... zca.pushGlobalRegistry() ... provideUtility(DummyUtility("layer"), name="layer") ... def tearDown(self): ... zca.popGlobalRegistry() ... def testSetUp(self): ... zca.pushGlobalRegistry() ... def testTearDown(self): ... zca.popGlobalRegistry() >>> COMPONENT_SANDBOX = ComponentSandbox()
Let’s now simulate a test using this layer.
To begin with, we have the default registry.
>>> from zope.component import getGlobalSiteManager, getSiteManager >>> getSiteManager() is getGlobalSiteManager() True>>> defaultGlobalSiteManager = getGlobalSiteManager()>>> from zope.component import queryUtility >>> queryUtility(IDummyUtility, name="layer") is None True
We’ll now simulate layer setup. This will push a new registry onto the stack:
>>> COMPONENT_SANDBOX.setUp()>>> getSiteManager() is getGlobalSiteManager() True >>> getGlobalSiteManager() is defaultGlobalSiteManager False >>> layerGlobalSiteManager = getGlobalSiteManager()>>> queryUtility(IDummyUtility, name="layer") <DummyUtility layer>
We’ll then simulate a test that registers a global component:
>>> COMPONENT_SANDBOX.testSetUp()>>> getSiteManager() is getGlobalSiteManager() True >>> getGlobalSiteManager() is defaultGlobalSiteManager False >>> getGlobalSiteManager() is layerGlobalSiteManager False
Our previously registered component is still here.
>>> queryUtility(IDummyUtility, name="layer") <DummyUtility layer>
We can also register a new one.
>>> provideUtility(DummyUtility("test"), name="test")
>>> queryUtility(IDummyUtility, name="layer")
<DummyUtility layer>
>>> queryUtility(IDummyUtility, name="test")
<DummyUtility test>
On test tear-down, only the second utility disappears:
>>> COMPONENT_SANDBOX.testTearDown()>>> getSiteManager() is getGlobalSiteManager() True >>> getGlobalSiteManager() is defaultGlobalSiteManager False >>> getGlobalSiteManager() is layerGlobalSiteManager True>>> queryUtility(IDummyUtility, name="layer") <DummyUtility layer> >>> queryUtility(IDummyUtility, name="test") is None True
If we tear down the layer too, we’re back where we started:
>>> COMPONENT_SANDBOX.tearDown()>>> getSiteManager() is getGlobalSiteManager() True >>> getGlobalSiteManager() is defaultGlobalSiteManager True>>> queryUtility(IDummyUtility, name="layer") is None True >>> queryUtility(IDummyUtility, name="test") is None True
ZCML files helper class
One of the frequent use cases is a layer that loads a ZCML file and sandbox the resulting registry.
The ZCMLSandbox can be instantiated with a filename` and package arguments.
>>> import plone.testing >>> ZCML_SANDBOX = zca.ZCMLSandbox(filename="testing_zca.zcml", ... package=plone.testing)
Before layer setup, the utility is not registered.
>>> queryUtility(Interface, name="layer") is None True
We’ll now simulate layer setup. This pushes a new registry onto the stack:
>>> ZCML_SANDBOX.setUp()>>> getSiteManager() is getGlobalSiteManager() True >>> getGlobalSiteManager() is defaultGlobalSiteManager False >>> queryUtility(Interface, name="layer") <Dummy utility>
The ZCMLSandbox class can also be used as ancestor for your own classes when you need to load more than a single ZCML file.
Your class then needs to override the setUpZCMLFiles() method. It is in charge of calling loadZCMLFile(), once for each ZCML file that the class needs to load.
>>> class OtherZCML(zca.ZCMLSandbox):
... def setUpZCMLFiles(self):
... self.loadZCMLFile("testing_zca.zcml", package=plone.testing)
... self.loadZCMLFile("testing_zca_more_specific.zcml",
... package=plone.testing)
>>> OTHER_ZCML_SANDBOX = OtherZCML()
Before layer setup, a second utility is not registered.
>>> queryUtility(Interface, name="more_specific_layer") is None True
We’ll now simulate the setup of the more specific layer.
>>> OTHER_ZCML_SANDBOX.setUp()
After setUp, the second utility is registered :
>>> queryUtility(Interface, name="more_specific_layer") <Dummy utility>
After layer teardown, the second utility is not registered anymore.
>>> OTHER_ZCML_SANDBOX.tearDown() >>> queryUtility(Interface, name="more_specific_layer") is None True
After teardown of the first layer, the first utility is not registered anymore.
>>> ZCML_SANDBOX.tearDown() >>> queryUtility(Interface, name="layer") is None True
Security
The Zope Security layers are found in the module plone.testing.security:
>>> from plone.testing import security
For testing, we need a testrunner
>>> from zope.testing.testrunner import runner
Layers
The security.CHECKERS layer makes sure that zope.security checkers are correctly set up and torn down.
>>> "%s.%s" % (security.CHECKERS.__module__, security.CHECKERS.__name__,) 'plone.testing.security.Checkers'>>> security.CHECKERS.__bases__ ()
Before the test, our custom checker is not in the registry.
>>> class DummyObject(object): ... pass>>> from zope.security.interfaces import IChecker >>> from zope.interface import implements >>> class FauxChecker(object): ... implements(IChecker) ... # we should really implement the interface here, but oh well>>> from zope.security.checker import getCheckerForInstancesOf >>> getCheckerForInstancesOf(DummyObject) is None True
Layer setup stacks the current checkers.
>>> options = runner.get_options([], [])
>>> setupLayers = {}
>>> runner.setup_layer(options, security.CHECKERS, setupLayers)
Set up plone.testing.security.Checkers in ... seconds.
We can now set up a checker. In real life, this may happen during ZCML configuration, but here will just call the API directlyMost likely, we’d do this in a child layer:
>>> from zope.security.checker import defineChecker >>> fauxChecker = FauxChecker() >>> defineChecker(DummyObject, fauxChecker)>>> getCheckerForInstancesOf(DummyObject) is fauxChecker True
Let’s now simulate a test that may use the checker.
>>> security.CHECKERS.testSetUp() >>> getCheckerForInstancesOf(DummyObject) is fauxChecker True >>> security.CHECKERS.testTearDown()
We still have the checker after test tear-down:
>>> getCheckerForInstancesOf(DummyObject) is fauxChecker True
However, when we tear down the layer, the checker is gone:
>>> runner.tear_down_unneeded(options, [], setupLayers) Tear down plone.testing.security.Checkers in ... seconds.>>> getCheckerForInstancesOf(DummyObject) is None True
Zope Publisher layers
The Zope Publisher layers are found in the module plone.testing.publisher:
>>> from plone.testing import publisher
For testing, we need a testrunner
>>> from zope.testing.testrunner import runner
ZCML directives
The publisher.PUBLISHER_DIRECTIVES layer extends the zca.ZCML_DIRECTIVES layer to extend its ZCML configuration context with the zope.app.publisher and zope.security directives available. It also extends security.CHECKERS.
>>> from plone.testing import zca, security>>> "%s.%s" % (publisher.PUBLISHER_DIRECTIVES.__module__, publisher.PUBLISHER_DIRECTIVES.__name__,) 'plone.testing.publisher.PublisherDirectives'>>> publisher.PUBLISHER_DIRECTIVES.__bases__ (<Layer 'plone.testing.zca.ZCMLDirectives'>, <Layer 'plone.testing.security.Checkers'>)
Before the test, we cannot use e.g. the <permission /> or <browser:view /> directives without loading the necessary meta.zcml files.
>>> from zope.configuration import xmlconfig
>>> xmlconfig.string("""\
... <configure package="plone.testing"
... xmlns="http://namespaces.zope.org/zope"
... xmlns:browser="http://namespaces.zope.org/browser"
... i18n_domain="plone.testing.tests">
... <permission id="plone.testing.Test" title="plone.testing: Test" />
... <browser:view
... for="*"
... name="plone.testing-test"
... class="plone.testing.tests.DummyView"
... permission="zope.Public"
... />
... </configure>""")
Traceback (most recent call last):
...
ZopeXMLConfigurationError: File "<string>", line 5.4
ConfigurationError: ('Unknown directive', u'http://namespaces.zope.org/zope', u'permission')
Layer setup creates a configuration context we can use to load further configuration.
>>> options = runner.get_options([], [])
>>> setupLayers = {}
>>> runner.setup_layer(options, publisher.PUBLISHER_DIRECTIVES, setupLayers)
Set up plone.testing.zca.LayerCleanup in ... seconds.
Set up plone.testing.zca.ZCMLDirectives in ... seconds.
Set up plone.testing.security.Checkers in ... seconds.
Set up plone.testing.publisher.PublisherDirectives in ... seconds.
Let’s now simulate a test that uses this configuration context to load the same ZCML string.
>>> zca.ZCML_DIRECTIVES.testSetUp() >>> security.CHECKERS.testSetUp() >>> publisher.PUBLISHER_DIRECTIVES.testSetUp()>>> context = zca.ZCML_DIRECTIVES['configurationContext'] # would normally be self.layer['configurationContext'] >>> xmlconfig.string("""\ ... <configure package="plone.testing" ... xmlns="http://namespaces.zope.org/zope" ... xmlns:browser="http://namespaces.zope.org/browser" ... i18n_domain="plone.testing.tests"> ... <permission id="plone.testing.Test" title="plone.testing: Test" /> ... <browser:view ... for="*" ... name="plone.testing-test" ... class="plone.testing.tests.DummyView" ... permission="zope.Public" ... /> ... </configure>""", context=context) is context True
The permission and view are now registered:
>>> from zope.component import queryUtility >>> from zope.security.interfaces import IPermission>>> queryUtility(IPermission, name=u"plone.testing.Test") <zope.security.permission.Permission object at ...>>>> from zope.interface import Interface >>> from zope.publisher.interfaces.browser import IDefaultBrowserLayer >>> from zope.component import getSiteManager >>> siteManager = getSiteManager()>>> [x.factory for x in siteManager.registeredAdapters() ... if x.provided==Interface and x.required==(Interface, IDefaultBrowserLayer) ... and x.name==u"plone.testing-test"] [<class '....plone.testing-test'>]
We can then simulate test tear-down:
>>> publisher.PUBLISHER_DIRECTIVES.testTearDown() >>> security.CHECKERS.testTearDown() >>> zca.ZCML_DIRECTIVES.testTearDown()
Note that you’d normally combine this layer with the zca.UNIT_TESTING or a similar layer to automatically tear down the component architecture between each test. Here, we need to do it manually.
>>> from zope.component.testing import tearDown >>> tearDown()
Layer tear-down does nothing.
>>> runner.tear_down_unneeded(options, [], setupLayers) Tear down plone.testing.publisher.PublisherDirectives in ... seconds. Tear down plone.testing.zca.ZCMLDirectives in ... seconds. Tear down plone.testing.zca.LayerCleanup in ... seconds. Tear down plone.testing.security.Checkers in ... seconds.>>> zca.ZCML_DIRECTIVES.get('configurationContext', None) is None True
Zope Object Database layers
The ZODB layers are found in the module plone.testing.zodb:
>>> from plone.testing import zodb
For testing, we need a testrunner
>>> from zope.testing.testrunner import runner
Empty ZODB layer
The EMPTY_ZODB layer is used to set up an empty ZODB using DemoStorage.
The storage and database are set up as layer fixtures. The database is exposed as the resource zodbDB.
A connection is opened for each test and exposed as zodbConnection. The ZODB root is also exposed, as zodbRoot. A new transaction is begun for each test. On test tear-down, the transaction is aborted, the connection is closed, and the two test-specific resources are deleted.
The layer has no bases.
>>> "%s.%s" % (zodb.EMPTY_ZODB.__module__, zodb.EMPTY_ZODB.__name__,) 'plone.testing.zodb.EmptyZODB'>>> zodb.EMPTY_ZODB.__bases__ ()
Layer setup creates the database, but not a connection.
>>> options = runner.get_options([], []) >>> setupLayers = {} >>> runner.setup_layer(options, zodb.EMPTY_ZODB, setupLayers) Set up plone.testing.zodb.EmptyZODB in ... seconds.>>> db = zodb.EMPTY_ZODB['zodbDB'] >>> db.storage EmptyZODB>>> zodb.EMPTY_ZODB.get('zodbConnection', None) is None True >>> zodb.EMPTY_ZODB.get('zodbRoot', None) is None True
Let’s now simulate a test.
>>> zodb.EMPTY_ZODB.testSetUp()
The test would then execute. It may use the ZODB root.
>>> zodb.EMPTY_ZODB['zodbConnection'] <Connection at ...>>>> zodb.EMPTY_ZODB['zodbRoot'] {}>>> zodb.EMPTY_ZODB['zodbRoot']['foo'] = 'bar'
On test tear-down, the transaction is aborted and the connection is closed.
>>> zodb.EMPTY_ZODB.testTearDown()>>> zodb.EMPTY_ZODB.get('zodbConnection', None) is None True>>> zodb.EMPTY_ZODB.get('zodbRoot', None) is None True
The transaction has been rolled back.
>>> conn = zodb.EMPTY_ZODB['zodbDB'].open()
>>> conn.root()
{}
>>> conn.close()
Layer tear-down closes and deletes the database.
>>> runner.tear_down_unneeded(options, [], setupLayers) Tear down plone.testing.zodb.EmptyZODB in ... seconds.>>> zodb.EMPTY_ZODB.get('zodbDB', None) is None True
Extending the ZODB layer
When creating a test fixture, it is often desirable to add some initial data to the database. If you want to do that once on layer setup, you can create your own layer class based on EmptyZODB and override its createStorage() and/or createDatabase() methods to return a pre-populated database.
>>> import transaction >>> from ZODB.DemoStorage import DemoStorage >>> from ZODB.DB import DB>>> class PopulatedZODB(zodb.EmptyZODB): ... ... def createStorage(self): ... return DemoStorage("My storage") ... ... def createDatabase(self, storage): ... db = DB(storage) ... conn = db.open() ... ... conn.root()['someData'] = 'a string' ... ... transaction.commit() ... conn.close() ... ... return db>>> POPULATED_ZODB = PopulatedZODB()
We’ll use this new layer in a similar manner to the test above, showing that the data is there for each test, but that other changes are rolled back.
>>> options = runner.get_options([], []) >>> setupLayers = {} >>> runner.setup_layer(options, POPULATED_ZODB, setupLayers) Set up PopulatedZODB in ... seconds.>>> db = POPULATED_ZODB['zodbDB'] >>> db.storage My storage>>> POPULATED_ZODB.get('zodbConnection', None) is None True >>> POPULATED_ZODB.get('zodbRoot', None) is None True
Let’s now simulate a test.
>>> POPULATED_ZODB.testSetUp()
The test would then execute. It may use the ZODB root.
>>> POPULATED_ZODB['zodbConnection'] <Connection at ...>>>> POPULATED_ZODB['zodbRoot'] {'someData': 'a string'}>>> POPULATED_ZODB['zodbRoot']['foo'] = 'bar'
On test tear-down, the transaction is aborted and the connection is closed.
>>> POPULATED_ZODB.testTearDown()>>> POPULATED_ZODB.get('zodbConnection', None) is None True>>> POPULATED_ZODB.get('zodbRoot', None) is None True
The transaction has been rolled back.
>>> conn = POPULATED_ZODB['zodbDB'].open()
>>> conn.root()
{'someData': 'a string'}
>>> conn.close()
Layer tear-down closes and deletes the database.
>>> runner.tear_down_unneeded(options, [], setupLayers) Tear down PopulatedZODB in ... seconds.>>> POPULATED_ZODB.get('zodbDB', None) is None True
Stacking DemoStorage storages
The example above shows how to create a simple test fixture with a custom database. It is sometimes useful to be able to stack these fixtures, so that a base layer sets up some data for one set of tests, and a child layer extends this, temporarily, with more data.
This can be achieved using layer bases and resource shadowing, combined with ZODB’s stackable DemoStorage. There is even a helper function available:
>>> from plone.testing import Layer >>> from plone.testing import zodb >>> import transaction>>> class ExpandedZODB(Layer): ... defaultBases = (POPULATED_ZODB,) ... ... def setUp(self): ... # Get the database from the base layer ... ... self['zodbDB'] = db = zodb.stackDemoStorage(self.get('zodbDB'), name='ExpandedZODB') ... ... conn = db.open() ... conn.root()['additionalData'] = "Some new data" ... transaction.commit() ... conn.close() ... ... def tearDown(self): ... # Close the database and delete the shadowed copy ... ... self['zodbDB'].close() ... del self['zodbDB']>>> EXPANDED_ZODB = ExpandedZODB()
Notice that we are using plain Layer as a base class here. We obtain the underlying database from our bases using the resource manager, and then create a shadow copy using a stacked storage. Stacked storages contain the data of the original storage, but save changes in a separate (and, in this case, temporary) storage.
Let’s simulate a test run again to show how this would work.
>>> options = runner.get_options([], []) >>> setupLayers = {} >>> runner.setup_layer(options, EXPANDED_ZODB, setupLayers) Set up PopulatedZODB in ... seconds. Set up ExpandedZODB in ... seconds.>>> db = EXPANDED_ZODB['zodbDB'] >>> db.storage ExpandedZODB>>> EXPANDED_ZODB.get('zodbConnection', None) is None True >>> EXPANDED_ZODB.get('zodbRoot', None) is None True
Let’s now simulate a test.
>>> POPULATED_ZODB.testSetUp() >>> EXPANDED_ZODB.testSetUp()
The test would then execute. It may use the ZODB root.
>>> EXPANDED_ZODB['zodbConnection'] <Connection at ...>>>> EXPANDED_ZODB['zodbRoot'] == dict(someData='a string', additionalData='Some new data') True>>> POPULATED_ZODB['zodbRoot']['foo'] = 'bar'
On test tear-down, the transaction is aborted and the connection is closed.
>>> EXPANDED_ZODB.testTearDown() >>> POPULATED_ZODB.testTearDown()>>> EXPANDED_ZODB.get('zodbConnection', None) is None True>>> EXPANDED_ZODB.get('zodbRoot', None) is None True
The transaction has been rolled back.
>>> conn = EXPANDED_ZODB['zodbDB'].open() >>> conn.root() == dict(someData='a string', additionalData='Some new data') True >>> conn.close()
We’ll now tear down the expanded layer and inspect the database again.
>>> runner.tear_down_unneeded(options, [POPULATED_ZODB], setupLayers) Tear down ExpandedZODB in ... seconds.>>> conn = EXPANDED_ZODB['zodbDB'].open() >>> conn.root() {'someData': 'a string'}>>> conn.close()
Finally, we’ll tear down the rest of the layers.
>>> runner.tear_down_unneeded(options, [], setupLayers) Tear down PopulatedZODB in ... seconds.>>> EXPANDED_ZODB.get('zodbDB', None) is None True >>> POPULATED_ZODB.get('zodbDB', None) is None True
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
