Skip to main content
Help the Python Software Foundation raise $60,000 USD by December 31st!  Building the PSF Q4 Fundraiser

Minimal task scheduling abstraction

Project description

Build Status Documentation Status

A minimal task scheduling abstraction and parallel arrays.

  • dask is a specification to describe task dependency graphs.
  • dask.array is a drop-in NumPy replacement (for a subset of NumPy) that encodes blocked algorithms in dask dependency graphs.
  • dask.async is a shared-memory asynchronous scheduler that efficiently executes dask dependency graphs on multiple cores.

See full dask documentation at


python install

Dask Graphs

Consider the following simple program:

def inc(i):
    return i + 1

def add(a, b):
    return a + b

x = 1
y = inc(x)
z = add(y, 10)

We encode this as a dictionary in the following way:

d = {'x': 1,
     'y': (inc, 'x'),
     'z': (add, 'y', 10)}

While less aesthetically pleasing this dictionary may now be analyzed, optimized, and computed on by other Python code, not just the Python interpreter.

A simple dask dictionary

Dask Arrays

The dask.array module creates these graphs from NumPy-like operations

>>> import dask.array as da
>>> x = da.random.random((4, 4), blockshape=(2, 2))
>>> x.T[0, 3].dask
{('x', 0, 0): (np.random.random, (2, 2)),
 ('x', 0, 1): (np.random.random, (2, 2)),
 ('x', 1, 0): (np.random.random, (2, 2)),
 ('x', 1, 1): (np.random.random, (2, 2)),
 ('y', 0, 0): (np.transpose, ('x', 0, 0)),
 ('y', 0, 1): (np.transpose, ('x', 1, 0)),
 ('y', 1, 0): (np.transpose, ('x', 0, 1)),
 ('y', 1, 1): (np.transpose, ('x', 1, 1)),
 ('z',): (getitem, ('y', 0, 1), (0, 1))}

Finally, a scheduler executes these graphs to achieve the intended result. The dask.async module contains a shared memory scheduler that efficiently leverages multiple cores.


dask.core supports Python 2.6+ and Python 3.3+ with a common codebase. It is pure Python and requires no dependencies beyond the standard library. It is a light weight dependency.

dask.array depends on numpy.

dask.bag depends on toolz and dill.


New BSD. See License File.

Project details

Release history Release notifications | RSS feed

This version


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for dask, version 0.3.0
Filename, size File type Python version Upload date Hashes
Filename, size dask-0.3.0.tar.gz (53.3 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page