spiny 0.3

Spiny will run your Python tests under multiple versions of Python

Spiny

Spiny Norman is a Hedgehog.

It’s also a package that will run your Python tests under multiple versions of Python.

“Hey hang on!” I hear you say, “There’s already Tox!” And you are right. Spiny is my attempt to look into the problem space tox covers to make something that is less complex, and fixes a couple of problems with Tox.

Most notably, with Spiny, Python does not have to be on the path, it can install versions of Python other than the one your installed version of virtualenv happens to support, and it does not always call setup.py.

There is no guarantee that Spiny’s features will not end up in Tox.

Command line parameters

You run Spiny from the root of the Python project that you want to test, ie the direectory that has the setup.py file.

The command line parameters are:

usage: spiny [-h] [–version] [-c <filename>] [-e <environments>] [-v] [-q]

[<configvar> [<configvar> …]]

Run tests under several Python versions.

positional arguments:

<configvar> Override a config variable by “section:variable=value”

Example: “spiny:venv-dir=.venv”

optional arguments:

-h, --help

Show this help message and exit.

--version

Show the version and exit.

-c <filename>, --config <filename>

The config file to use. Defaults “to spiny.cfg”.

-e <environments>, --envlist <environments>

A list of environments to run, separated by commas.

-v, --verbose

Increases the output, -vv increases it even more.

-q, --quiet

Reduces output to only the run summary, -qq removes also that.

Version support

Spiny can be run under Python 2.6, 2.7, 3.3 and 3.4. It can also be run under PyPy, PyPy3 and Jython.

It can run tests under a much wider range of Python versions, this has been tested with Python 2.4, 2.5, 3.1 and 3.2 in addition to the above Python versions.

IronPython is supported in theory, but I can’t get virtualenv working with IronPython. Other Python implementations are not tested. If one doesn’t work, you are welcome to raise an issue and I can look into it.

Configuration files

Spiny does not typically need a configuration file. It will instead look at your setup.py and find out what Python versions you module supports, and run the tests with those versions.

You declare version support with classifiers, like this:

setup(
    ...
    classifiers=[
        ...
        "Programming Language :: Python",
        "Programming Language :: Python :: 2.6",
        "Programming Language :: Python :: 2.7",
        "Programming Language :: Python :: 3.3",
        "Programming Language :: Python :: 3.4",
        "Programming Language :: Python :: Implementation :: PyPy",
        ...
        ],
    ...
)

With these classifiers, Spiny will run the tests under Python 2.6, 2.7, 3.3, 3.4 and also under PyPy.

You can configure Spiny explicitly on a configuration basis. You can add a spiny.cfg located in the project root, or you can add the configuration to the projects setup.cfg.

You can also have a personal configuration file in ~/.config/spiny.cfg whose configuration options will be valid for all projects.

Under the [spiny] section the following configuration options are supported:

• environments: A whitespace separated list of Python and Python versions.

• venv-dir: The name of the directory to install virtualenvs in. Defaults to .venv.

• max-processes: The maxiumum of concurrent processes to run tests with. Defaults to the number of CPU’s you have.

• test-commands: The commands used to run the tests. You can have several lines of commands. Defaults to {python} setup.py test. There are a few variables that you can use in the commands that will be replaced:

  • {python} will be replaced with the full path to the Python executable

  • {envpath} will be replaced with the full path to the virtualenv directory.

  • {project_dir} will be replaced with the full path to the directory of the Python project (ie, the current directory)

Example:

[spiny]
environments = python2.7
               python3.4
               pypy2.4

venv-dir = .venv

test-commands = {python} something.py magic
                {python} setup.py test

max-processes = 3

There is also a [pythons] section, which defines up the paths to the various executables, per environment:

[pythons]
python2.6 = /pythons/python26/bin/python
python2.7 = /pythons/python27/bin/python
python3.3 = /pythons/python33/bin/python3
python3.4 = /pythons/python34/bin/python3

If you don’t configure this, the executables that are on the PATH will be used.

This doesn’t make much sense to have in the projects spiny.cfg, as each person who runs the tests are likely to have differing Python installs. However, this does make a lot of sense to have in the personal configuration file.

You can add the [pythons] section in your projects spiny.cfg, but the usecase for that is very limited. Possibly if you are using custom Pythons in your project.

max-processes also is reasonable in your personal file, if you for example have very long-running tests, and you want to keep a CPU free, for example for browing the web while the tests run. It also makes sense in a project file if your tests use a lot of memory, to avoid running out of memory.

environments and test-commands only make sense per configuration and not in the personal file. However, no checks for this are done, so you can add them there if you want to, but the results are unlikely to be practical.

Todo

Things that needs doing:

• Make the tests run under itself.

• Figure out how to run coverage on things run by subprocesses.

• Add commands per environment, enabling things like a pep8 environment that checks for pep8 compliance, etc.

• Add an override also for the test setup commands, ie virtualenv etc.

• Add an explicit requirements setting instead of looking in setup.py.

CHANGES

0.3 (2014-10-30)

• Now runs under Python 3, PyPy, PyPy3, Jython.

• All output is now through logging, which enables you to select verbosity.

• Added –version, –verbose and –quiet options.

• Now use subprocess32 under Python 2.

• Will now run tests in parallell with multiprocessing. A ‘max-processes’ option in the [spiny] section will allow you to cap the processes if you don’t want to max your computer. Otherwise it will use as many processes as multiprocessing will report cpu’s.

0.2 (2014-10-27)

• Now prints a summary in the end.

• Regard more virtualenv tests as failures.

• Return 1 if not all tests succeeded.

• Fall back to setup.cfg is there is no spiny config file.

• Renamed spiny.conf to spiny.cfg for consistency.

• Made the configuration file optional.

• Spiny will now grep setup.py for Python version classifiers if no environment configuration is found.

• If both Python X and Python X.Y is given as environments, it will skip the tests for Python X, seeing as it is fulfilled by Python X.Y.

• The version specified in the [pythons] section now have preference.

• Setting a config option from the command line now works even if the section doesn’t exist.

• Spiny will now attempt to extract the test requirements and install them into the virtualenv before running the tests, to avoid polluting the project directory with .egg directories.

0.1.1 (2014-10-18)

• Initial release