Skip to main content

A Jupyter Notebook server extension manages Dask clusters.

Project description

Dask JupyterLab Extension

Build Status Version Downloads Dependencies

This package provides a JupyterLab extension to manage Dask clusters, as well as embed Dask's dashboard plots directly into JupyterLab panes.

Dask Extension

Explanatory Video (5 minutes)

Dask + JupyterLab Screencast


JupyterLab >= 1.0 distributed >= 1.24.1


To install the Dask JupyterLab extension you will need both JupyterLab, and Node.js. These are available through a variety of sources. One source common to Python users is the conda package manager.

conda install jupyterlab nodejs

This extension includes both a client-side JupyterLab extension and a server-side Jupyter notebook extension. Install via pip or conda-forge:

pip install dask_labextension
conda install -c conda-forge dask-labextension

and build the extension as follows:

jupyter labextension install dask-labextension
jupyter serverextension enable dask_labextension

If you are running Notebook 5.2 or earlier, enable the server extension by running

jupyter serverextension enable --py --sys-prefix dask_labextension

Configuration of Dask cluster management

This extension has the ability to launch and manage several kinds of Dask clusters, including local clusters and kubernetes clusters. Options for how to launch these clusters are set via the dask configuration system, typically a .yml file on disk.

By default the extension launches a LocalCluster, for which the configuration is:

    module: 'dask.distributed'
    class: 'LocalCluster'
    args: []
    kwargs: {}
    workers: null
      # minimum: 0
      # maximum: 10
    # - name: "My Big Cluster"
    #   workers: 100
    # - name: "Adaptive Cluster"
    #   adapt:
    #     minimum: 0
    #     maximum: 50

In this configuration, factory gives the module, class name, and arguments needed to create the cluster. The default key describes the initial number of workers for the cluster, as well as whether it is adaptive. The initial key gives a list of initial clusters to start upon launch of the notebook server.

In addition to LocalCluster, this extension has been used to launch several other Dask cluster objects, a few examples of which are:

  • A SLURM cluster, using
      module: 'dask_jobqueue'
       class: 'SLURMCluster'
       args: []
       kwargs: {}
  • A PBS cluster, using
    module: 'dask_jobqueue'
    class: 'PBSCluster'
    args: []
    kwargs: {}
    module: dask_kubernetes
    class: KubeCluster
    args: []
    kwargs: {}

Development install

As described in the JupyterLab documentation for a development install of the labextension you can run the following in this directory:

jlpm install   # Install npm package dependencies
jlpm run build  # Compile the TypeScript sources to Javascript
jupyter labextension install  # Install the current directory as an extension

To rebuild the extension:

jlpm run build

If you run JupyterLab in watch mode (jupyter lab --watch) it will automatically pick up changes to the built extension and rebundle itself.

To run an editable install of the server extension, run

pip install -e .
jupyter serverextension enable --sys-prefix dask_labextension


This application is distributed as two subpackages.

The JupyterLab frontend part is published to npm, and the server-side part to PyPI.

Releases for both packages are done with the jlpm tool, git and Travis CI.

Note: Package versions are not prefixed with the letter v. You will need to disable this.

$ jlpm config set version-tag-prefix ""

Making a release

$ jlpm version [--major|--minor|--patch]  # updates package.json and creates git commit and tag
$ git push upstream master && git push upstream master --tags  # pushes tags to GitHub which triggers Travis CI to build and deploy

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dask_labextension-4.0.0.tar.gz (33.5 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page