Skip to main content

nose-parallel

A plugin to help split up your tests runs across multiple machines.

In an ideal world, test suites should be fast enough that they can be run locally on a single machine without extra engineering. This plugin is there to help when you can’t make that to happen.

This won’t help you run tests in parallel on one machine in different threads; that’s what the built-in multiprocess plugin is for.

This will help you split up your test suites so that you can run the suites on multiple machines and not have the same test run twice - think Jenkins with the multijob plugin, or CI services like CircleCI.

Install

pip install nose-parallel

Usage

On each machine:

  1. Export environment variables NODE_TOTAL (the number of machines on which the suite will be run) and and NODE_INDEX (the 0-based index of the current machine)

  2. Run nosetests with the --with-parallel flag

  3. Do something to join the results from all the machines back together

For example, this is how we’d run nosetests on the second machine in a four-machine testing cluster:

NODE_TOTAL=4 NODE_INDEX=1 nosetests --with-parallel

If you don’t set those variables, nose-parallel will do the right thing and run all your tests. It also supports natively the default environment variables from:

  • CircleCI

  • Buildkite

  • GitLab CI

We welcome PRs that add native support for other CI alternatives.

If you want to further randomize the distribution of tests, so that the same test cases don’t always run together on the same node, you may use the --parallel-salt option or NOSE_PARALLEL_SALT environment variable to specify a salt value for the randomization (e.g. a build number, the day of the month). The salt must be the same on each node during a parallel build, otherwise some test cases might be missed.

For example, on CircleCI you could do:

nosetests --with-parallel --parallel-salt-env CIRCLE_BUILD_NUM

By default nose-parallel will split tests based on method and function names. You can customize this strategy by setting the --parallel-strategy option. Supported strategies are: ‘CLASS’, ‘DIRECTORY’, ‘FILE’, ‘METHOD’, ‘FUNCTION’ and ‘MODULE’.

For example, to split tests based on the file names:

nosetests --with-parallel --parallel-strategy=FILE

License

nose-parallel is released under the MIT license.

Contribute

  • Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug.

  • Fork the repository on GitHub to start making your changes to the master branch (or branch off of it).

  • Send a pull request and bug the maintainer until it gets merged and published.

Thanks To

Metadata

Release files for nose-parallel 0.4.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for nose-parallel 0.4.0
File Size Uploaded
nose-parallel-0.4.0.tar.gz 4.6 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for nose-parallel 0.4.0
File Interpreter ABI Platform
nose_parallel-0.4.0-py2.py3-none-any.whl Python 2, Python 3 none any Details

Total release size: 9.7 kB

Release files / nose-parallel-0.4.0.tar.gz

Download URL nose-parallel-0.4.0.tar.gz
Size 4.6 kB
Tags Source
SHA-256 checksum
How to use checksums
e4e6d6891a4bb417f09abf1f400554f557f0a9df3fdaed6517d4236e87ed0f51
BLAKE2b-256 checksum
How to use checksums
7060039898d68516d2086db9b8ec8e318f84f44a15a8584b2abdfc2d97980a68
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.45.0 CPython/3.7.7

Release files / nose_parallel-0.4.0-py2.py3-none-any.whl

Download URL nose_parallel-0.4.0-py2.py3-none-any.whl
Size 5.1 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
3f7c48d7aa5030fe872211cbffc6852e64e4e356ddbac5958a35796bf15dc225
BLAKE2b-256 checksum
How to use checksums
64089968ef72b3ee8f3243e1f300219afdfb753a3432b203afe9bef56e34ce95
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.45.0 CPython/3.7.7

Release history Release notifications | RSS feed

This release

0.4.0 This release

2 release files

0.3.1

2 release files

0.3.0

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.4

2 release files

0.1.3

1 release file

0.1.2

2 release files

0.1.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page