Pytest plugin for testing notebooks
Project description
nbmake
What? Pytest plugin for testing and releasing notebook documentation
Why? To raise the quality of scientific material through better automation
Who is this for? Research/Machine Learning Software Engineers who maintain packages/teaching materials with documentation written in notebooks.
Functionality
- Executes notebooks using pytest and nbclient, allowing parallel notebook testing
- Optionally writes back to the repo, allowing faster building of nbsphinx or jupyter book docs
- Optionally builds an HTML report using jupyter-book of the test run which can be uploaded to hosting providers such as Netlify.
Quick Start
pip install pytest nbmake
pytest --nbmake **/*ipynb
Allow errors and Configure Cell Timeouts
nbmake is built on top of nbclient and designed to compatible with nbsphinx and jupyter-book.
The jupyter book docs have a good description of how to ignore errors -- note that nbmake does not work with the external _config.yml
configuration file. You must use the notebook JSON metadata field.
Parallelisation
Parallelisation with xdist is experimental upon initial release, but you can try it out:
pip install pytest-xdist
pytest --nbmake -n=auto
It is also possible to parallelise at a CI-level using strategies, see example
Build Jupyter Books Faster
Using xdist and the --overwrite
flag let you build a large jupyter book repo faster:
pytest --nbmake --overwrite -n=auto examples
jb build examples
Advice on Usage
nbmake is best used in a scenario where you use the ipynb files only for development. Consumption of notebooks is primarily done via a docs site, built through jupyter book, nbsphinx, or some other means. If using one of these tools, you are able to write assertion code in cells which will be hidden from readers.
Pre-commit
Treating notebooks like source files lets you keep your repo minimal. Some tools, such as plotly may drop several megabytes of javascript in your output cells, as a result, stripping out notebooks on pre-commit is advisable:
# .pre-commit-config.yaml
repos:
- repo: https://github.com/kynan/nbstripout
rev: master
hooks:
- id: nbstripout
See https://pre-commit.com/ for more...
Disable Nbmake
Implicitly:
pytest
Explicitly:
pytest -p no:nbmake
See Also:
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.