Skip to main content

A pytest plugin for testing Jupyter Notebooks

Project description

pytest-notebook

CI Status Coverage RTD PyPI Conda Code style: black

A pytest plugin for regression testing and regenerating Jupyter Notebooks.

Example Test


Purpose

The purpose of the plugin is to ensure that changes to the python environment (e.g. updating packages), have not affected the outputs of the notebook. If the notebook has changed, this plugin can also regenerate the notebooks, saving the new outputs.

Features

  • Recognise, collect, execute (optionally output) then diff input vs. output Jupyter Notebooks.

  • Provides clear and colorized diffs of the notebooks (using nbdime)

  • Regenerate failing notebooks.

  • Integration with coverage and pytest_cov.

  • A well defined API allows notebook regression tests to be run:

    1. Using the pytest test collection architecture.
    2. As a pytest fixtures (nb_regression.check(filename)).
    3. Using the pytest_notebook python package.
  • All stages are highly configurable via:

    1. The pytest command-line interface.
    2. The pytest configuration file.
    3. The notebook and cell level metadata.
  • Post-processor plugin entry-points, allow for customisable modifications of the notebook, including source code formatting with black

Configuration Examples

Installation

To install from Conda (recommended):

>> conda install -c conda-forge pytest-notebook

To install via pip from PyPI:

>> pip install pytest-notebook

To install the development version:

>> git clone https://github.com/chrisjsewell/pytest-notebook .
>> cd pytest-notebook
>> pip install -e .
>> # pip install -e .[code_style,testing,docs] # install extras for more features

Usage

See the documentation at: http://pytest-notebook.readthedocs.io/

Contributing

Contributions are very welcome.

The following will discover and run all unit test:

>> pip install -e .[testing]
>> pytest -v

Coding Style Requirements

The code style is tested using flake8, with the configuration set in .flake8, and code should be formatted with black.

Installing with pytest-notebook[code_style] makes the pre-commit package available, which will ensure these tests are passed by reformatting the code and testing for lint errors before submitting a commit. It can be setup by:

>> cd pytest-notebook
>> pre-commit install

Optionally you can run black and flake8 separately:

>> black .
>> flake8 .

Editors like VS Code also have automatic code reformat utilities, which can adhere to this standard.

License

Distributed under the terms of the BSD-3 license, pytest-notebook is free and open source software.

Issues

If you encounter any problems, please file an issue along with a detailed description.

Acknowledgements

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for pytest-notebook, version 0.6.0
Filename, size & hash File type Python version Upload date
pytest_notebook-0.6.0-py3-none-any.whl (37.4 kB) View hashes Wheel py3
pytest-notebook-0.6.0.tar.gz (177.3 kB) View hashes Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page