Skip to main content

Run (and time) Jupyter Notebooks for command-line and makefile

Project description

Run Jupyter notebooks quietly from command-line

PyPI GitHub release (latest by date)
PyPI - Python Version
DOI

runPyNB is a quick and dirty utility to run (and time) Jupyter notebooks from command-line and makefiles.

Quickstart

Install from PyPI

pip install runpynb

General usage: runpynb <notebook(s)> [options] (".ipynb" not required)

  • runpynb: Run all notebooks in directory.

      $ runpynb

(back to top)

Usage

  • runpynb <notebook(s)> -q: Run quietly (-q).

      $ runpynb hello.ipynb -q

  • runpynb <notebook(s)> -qs: Run quietly (-q) as a sequence of workflow (-s). Errors (eg in error.ipynb) will break the workflow.

      $ runpynb error.ipynb hello.ipynb -qs

  • runpynb <notebook(s)> -o: Save output as separate notebook (-o), instead of overwriting existing notebook(s).

      $ runpynb hello.ipynb -o

(back to top)

Options

usage: runpynb [-h] [-t TIMEOUT] [-s] [-o] [-v VERSION] [-q] [notebooks ...]

Run (and time) Jupyter notebooks silently in command-line.

positional arguments:
  notebooks             List of Jupyter notebooks (*.ipynb) to be run
                        (default=all notebooks in path).

optional arguments:
  -h, --help            show this help message and exit
  -t TIMEOUT, --timeout TIMEOUT
                        Seconds until a cell in the notebook timesout, which
                        raises a Timeouterror exception (default is 3000=5
                        mins).
  -s, --sequence        Sequence implicit in notebook lists. If error occurs
                        somewhere, stop entire pipeline.
  -o, --output          Save output as a separate notebook with "-out"-suffix
                        (e.g. *-out.ipynb) instead of overwriting existing
                        file.
  -v VERSION, --version VERSION
                        Version of notebook to return (Default=No conversion).
                        Notebook will be converted if necessary.
  -q, --quiet           Be quiet and don't print messages (including run
                        time). Caution: Does not suppress error messages.

(back to top)

Status

Documentation Status
Build Status
Tests codecov
CI
CLI
Doclinks

(back to top)


More on this package

This is a lightweight package that wraps around the official Jupyter nbformat and nbconvert modules.

My workflow involves using Jupyter notebooks to clean, and analyze data. I use this utility to run notebooks silently from the command-line and Makefiles (without converting from .ipynb files to .py files).

Related packages are guoquan/runnb and vinayak-mehta/nbcommands with a planned enhancement nbtime to run Jupyter notebooks from command-line.

(back to top)

Usage with Makefiles

A minimal workflow where get-data.ipynb takes 5000 seconds to prepare data.csv. And where analyze.ipynb uses data.csv to produce output.png.

.DEFAULT_GOAL := output.png

data.csv: get-data.ipynb
	runpynb $^ -t 5000
	
output.png: analyze.ipynb data.csv
	runpynb $< 

(back to top)

Known Issues

  • Build fails with Python 3.6 in Windows OS.
  • Notebooks with long execution time will require the timeout option (eg runpynb notebook.ipynb -t 10000).

(back to top)

License

This package is licensed under the MIT License.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

runpynb-0.1.4.tar.gz (6.4 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page