Skip to main content

Map Reduce for Notebooks

Project description

Papermill is map reduce for Jupyter notebooks.

Stepping away from the hyperbole, our goals for Papermill include simplifying and streamlining:

  • Parametrizing notebooks
  • Executing and collecting metrics across the notebooks
  • Summarize collections of notebooks

Installation

pip install papermill

Usage

Executing a parametrized notebook

import papermill as pm

pm.execute_notebook(
    notebook="template.ipynb",
    output="output.ipynb",
    params=dict(alpha=0.1, ratio=0.001)
)

nb = pm.read_notebook("output.ipynb")

Creating a parametrized notebook and record metrics

### template.ipynb
import papermill as pm

rmse = metrics.mean_squared_error(...)
pm.record_value("rmse", rmse)
plot() # Tag this cell as "results" for extraction later
### run_and_summarize.ipynb
pm.execute_notebook(
    notebook="template.ipynb",
    output="output.ipynb",
    params=dict(alpha=0.1, ratio=0.001)
)

nb = pm.read_notebook("output.ipynb")
result_cell = pm.get_tagged_cell(nb, "results")

rmse = pm.fetch_record(result_cell, "rmse")
plot = pm.get_image_from_cell(result_cell)
print("rmse", rmse)
pm.display_image(plot)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
papermill-0.3-py2-none-any.whl (8.3 kB) Copy SHA256 hash SHA256 Wheel py2 Jul 10, 2017
papermill-0.3.tar.gz (22.4 kB) Copy SHA256 hash SHA256 Source None Jul 10, 2017

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page