Skip to main content
Python Software Foundation 20th Year Anniversary Fundraiser  Donate today!

Smart random-generation state persistence for unittest

Project description


Randomized unit tests are great. The inputs fed to your algorithm are not consistent across executions, and so each time the tests run successfully, your confidence in the algorithm grows. Unfortunately, when a test fails, this inconsistency makes it hard to duplicate the input until the problem is solved.

This library provides a simple metaclass for dropping into unittest.TestCase that addresses this. As long as a test fails, it will continue making the same random choices (via random or numpy.random) each execution. Any test that passed the previous execution, will get a fresh set of random choices.


Consider the following simplified example of a test case with three functions testing some algorithms. The first and third algorithms are OK, but the second has a bug when given a number that divides by 20 (the simplified example asserts this directly).

import random

class TestMyAlgs(unittest.TestCase):
    # This "algorithm" is OK.
    def test_ok(self):
        i = random.randint(100, 200)
        self.assertLess(i, i + 1)

    # This "algorithm" fails for inputs that divide by 20.
    def test_bad(self):
        i = random.randint(100, 200)
        self.assertNotEqual(i % 20, 0, i)

    # This "algorithm" is OK too.
    def test_another_ok(self):
        i = random.randint(100, 200)
        self.assertLess(i - 1, i)

The second function will fail approximately 5% of the executions. When it does so, it might be hard (especially in a real-life case) to reproduce the problem, and eventually solve it.

Using this library, a simple change makes things easier. Simply use unittest_rand_gen_state.Saver as the metaclass:

import random

import unittest_rand_gen_state

class TestMyAlgs(unittest.TestCase, metaclass = unittest_rand_gen_state.Saver):

or, using an older Python version:

import random

import unittest_rand_gen_state

class TestMyAlgs(unittest.TestCase):
    __metaclass__ = unittest_rand_gen_state.Saver

Nothing else needs to be changed.

Once TestMyAlgs.test_bad first fails in some execution, then it will consistently fail in following executions. That is, say it fails the first time with

FAIL: test_bad (__main__.TestMyAlgs)
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/unittest_rand_gen_state/", line 75, in wrapped
    fn(*args, **kwargs)
  File "", line 21, in test_bad
    self.assertNotEqual(i % 20, 0, i)
AssertionError: 140


then every following execution will fail exactly the same way, until it is fixed.

The two other tests, meanwhile, will run afresh all the while.

Download, Installation, And Bugtracking

  • The package is at PyPI.

  • The usual setup for Python libraries is used. Type:

    $ pip install UnittestRandGenState


    $ sudo pip install UnittestRandGenState

  • Bugtracking is on Google Code.


The metaclass supplied by this library deals with the states of random and numpy.random, which should be sufficient for almost all tests. However, the library may be extended by dropping as a metaclass, a subclass of unittest_rand_gen_state.Saver supplying three class methods:

  • reset_state: A method specifying the meaning of resetting the random-generation state (e.g., random.seed)
  • get_state: A method specifying the query to call for getting the current random-generation state (e.g., random.getstate)
  • set_state: A method specifying the function to call to restore the random-generation state using the value returned by get_state (e.g., random.setstate)

E.g., suppose a random-generation library, groovy_rng has the functions groovy_rng.seed (setting the state to some function determined by the current time), groovy_rng.get_state, returning an int, and groovy_rng.set_state taking an int, then here is a metaclass for working with groovy_rng:

class GroovyRNGSaver(unittest_rand_gen_state.Saver):
    reset_state = groovy_rng.seed
    get_state = groovy_rng.get_state
    set_state = groovy_rng.set_state

This metaclass can be used just like in the previous example.


Version Date Description
0.1.4. 15/03/2013 Duplicate class-name bugfix, Python 2/3 coexistance bugfix
0.1.3 23/02/2013 Better test hashes, better support for extension
0.1.2 22/02/2013 Doc bugfix
0.1.1 20/02/2013 Less stateful design
0.1.0 18/02/2013 Initial Release

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for UnittestRandGenState, version 0.1.4
Filename, size File type Python version Upload date Hashes
Filename, size UnittestRandGenState-0.1.4.tar.gz (6.6 kB) File type Source Python version None Upload date Hashes View
Filename, size (12.7 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page