Basic notebook checks. Do they run? Do they contain lint?
Basic notebook smoke tests: Do they run ok? Do they contain lint?
WARNING: early stage proof of concept; work in progress. Use at your own risk.
In particular, this extension is supposed to handle ipython magics as far as possible, but has not yet been widely tested.
$ pip install nbsmoke
$ conda install -c pyviz/label/dev -c conda-forge nbsmoke
Check all notebooks run without errors:
$ pytest --nbsmoke-run
Check all notebooks run without errors, and store html to look at afterwards:
$ pytest --nbsmoke-run --store-html=/scratch
Lint check notebooks:
$ pytest --nbsmoke-lint
Instead of all files in a directory, you can specify a list e.g.:
$ pytest --nbsmoke-run notebooks/Untitled*.ipynb
If you want to restrict pytest to running only your notebook tests, use -k, e.g.:
$ pytest --nbsmoke-run -k ".ipynb"
Additional options are available by standard pytest ‘ini’ configuration in setup.cfg, pytest.ini, or tox.ini:
[pytest] # when running, seconds allowed per cell (see nbconvert timeout) nbsmoke_cell_timeout = 600 # notebooks to skip running; one case insensitive re to match per line nbsmoke_skip_run = ^.*skipme\.ipynb$ ^.*skipmetoo.*$ # case insensitive re to match for file to be considered notebook; # defaults to ``^.*\.ipynb`` it_is_nb_file = ^.*\.something$
nbsmoke supports # noqa comments to mark that something should be ignored during lint checking.
The nbsmoke_skip_run list in a project’s config can be ignored by passing --ignore-nbsmoke-skip-run (useful if sometimes you want to run all notebooks).
What’s the point?
Although more sophisticated testing of notebooks is possible (e.g. see nbval), just checking that notebooks run from start to finish without error in a fresh kernel (or on a neutral CI service) can be useful during development. Practical experience of working on several projects with notebooks confirms this, but that’s all the evidence I have.
Checking notebooks for lint might seem trivial/pointless, but it frequently uncovers unused names (typically unused imports). It’s also quite common to find python 2 vs 3 problems, and sometimes undefined names - in a way that’s faster than running the notebook (over multiple versions of python).
Unused imports/names themselves might seem trivial, but they can hinder understanding of a notebook by readers, or add dependencies that are not required.
Using # noqa: explanation in a notebook might seem like overkill, but the intention is to at least force ‘mysterious imports’ to be clarified (if they are necessary at all, which ideally they shouldn’t be). E.g. if you’re importing something for its side effects, it’s very helpful to inform the reader of that, and the ugly/strange # noqa should help remind you to fix the underlying problem…
Pyflakes is used as the underlying linter because “Pyflakes makes a simple promise: it will never complain about style, and it will try very, very hard to never emit false positives.”
First, install using pip install -e .. Then run the tests using tox or pytest -v tests/.
New release to PyPI and anaconda.org: git tag -a vX.Y.Z -m "Something about release" && git push --tags.
Distributed under the terms of the BSD-3 license, “nbsmoke” is free and open source software.
If you encounter any problems, please file an issue (ideally including a copy of any problematic notebook).
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size & hash SHA256 hash help||File type||Python version||Upload date|
|nbsmoke-0.2.8-py2.py3-none-any.whl (10.4 kB) Copy SHA256 hash SHA256||Wheel||py2.py3|
|nbsmoke-0.2.8.tar.gz (19.1 kB) Copy SHA256 hash SHA256||Source||None|