This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Nose plugin to randomly order tests and control random.seed. (Also available for pytest).

Features

All of these features are on by default but can be disabled with flags.

  • Randomly shuffles the submodules, TestCase classes + test functions when loading a module of tests.
  • Randomly shuffles the test functions inside a TestCase when loading it.
  • Resets random.seed() at the start of every test case and test to a fixed number - this defaults to time.time() from the start of your test run, but you can pass in --randomly-seed to repeat a randomness-induced failure.
  • If factory boy is installed, its random state is reset at the start of every test. This allows for repeatable use of its random ‘fuzzy’ features.
  • If faker is installed, its random state is reset at the start of every test. This is also for repeatable fuzzy data in tests - factory boy uses faker for lots of data.

About

Randomness in testing can be quite powerful to discover hidden flaws in the tests themselves, as well as giving a little more coverage to your system.

By randomly ordering the tests, the risk of surprising inter-test dependencies is reduced - a technique used in many places, for example Google’s C++ test runner googletest.

By resetting the random seed to a repeatable number for each test, tests can create data based on random numbers and yet remain repeatable, for example factory boy’s fuzzy values. This is good for ensuring that tests specify the data they need and that the tested system is not affected by any data that is filled in randomly due to not being specified.

Requirements

Tested with:

  • Python 2.7, 3.4, 3.5
  • The latest version of Nose

Usage

Install from pip with:

pip install nose-randomly

Nose will automatically find the plugin.

To activate it on your test run, use the --with-randomly flag, for example:

nosetests -v --with-randomly

The output will start with an extra line that tells you the random seed that is being used:

Using --randomly-seed=1234
test_D (abcd_tests.Tests) ... ok
...

If the tests then fail due to ordering or randomly created data, you can then restart them with that seed:

nosetests -v --with-randomly --randomly-seed=1234

You can disable behaviours you don’t like with the following flags:

  • --randomly-dont-shuffle-modules - turn off the shuffling of the contents of modules
  • --randomly-dont-shuffle-cases - turn off the shuffling of test functions inside TestCase classes
  • --randomly-dont-reset-seed - turn off the reset of random.seed() at the start of every test

Background

nose has an unmerged pull request from 2009 to add random ordering functionality. This is available in plugin format in the nose-randomize package. It works quite well but I found that since it replaces all of the test loading machinery inside nose, it can interact badly with other plugins. This plugin was developed as a thinner layer to achieve the same thing, plus the random seed resetting which was not available before.

License

  • BSD licensed, see LICENSE file
  • Logo by Christian Mohr from the Noun Project (link).

History

Pending release

  • New release notes here

1.2.5 (2016-10-28)

  • Set a high plugin score to ensure that nose-randomly is loaded before other plugins. This fixes a bug where randomization would disapper when using the doctests plugin that is included with Nose.

1.2.4 (2016-10-27)

  • Reset the random state for NumPy too.

1.2.3 (2016-08-19)

  • Fixed output so the random seed is always output when the plugin is enabled, not just when resetting random.seed() at the start of tests. Thanks @amygdalama.

1.2.2 (2016-07-06)

  • Fixed to work with python setup.py nosetests on Python 2 due to issue with unicode not working with distutils.fancy_getopt.

1.2.1 (2016-06-01)

  • Support test generators.

1.2.0 (2015-12-10)

  • Reset the random state for Faker (pip package fake-factory) too

1.1.0 (2015-08-27)

  • Reset the random seed at the start of nose test contexts (TestCases etc.) too
  • Slight performance improvement by always using random.setstate() for reseeding

1.0.0 (2015-07-23)

  • First release on PyPI.
Release History

Release History

1.2.5

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.2.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.2.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.2.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.2.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.2.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

1.0.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
nose_randomly-1.2.5-py2.py3-none-any.whl (11.1 kB) Copy SHA256 Checksum SHA256 2.7 Wheel Oct 28, 2016
nose-randomly-1.2.5.tar.gz (9.6 kB) Copy SHA256 Checksum SHA256 Source Oct 28, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting