A 2D progress matrix for visualizing Dask computations by chunk.
Project description
Visualize Dask computations by chunk.
Install
pip install dask-progress-matrix
Quick-start
API
Use ProgressMatrix as a context manager to track Dask computations:
import dask.array as da
from dask_progress_matrix import ProgressMatrix
with ProgressMatrix(cmap="inferno"):
da.random.random((2, 128, 256), chunks=(1, 16, 16)).compute()
CLI
Track Dask computations in any Python file using the CLI. For example, using uv:
$ uvx dask-progress-matrix compute_something.py --cmap=inferno
Features
-
Terminal or Jupyter - Progress matrixes can be displayed in both terminal environments and Jupyter notebooks.
-
Modes - When a computation is complete, the
ProgressMatrixdisplays a summary with either the completed index or the elapsed time for each chunk, depending on themodeparameter. -
Data structures - Computing any Dask-backed object will display a progress matrix, including Xarray objects.
-
Dimensionality - You can track the computation of any Dask array, regardless of dimensionality. For visualization, arrays are truncated to the last two dimensions, so e.g. an array with chunks
(3, 16, 16)will be rendered as a 16 x 16 matrix where each chunk tracks the progress of 3 different computations.
Limitations
-
Distributed schedulers - Support for distributed schedulers like
dask.distributed.Clientisn't currently implemented. -
Character width - Each computation chunk is rendered with a minimum width of 2 characters, so arrays with huge numbers of chunks may render slowly or poorly.
-
Chunk shapes - All chunks are represented by squares, regardles of their shape.
FAQ
Why use a progress matrix?
This was mostly developed out of curiosity, but it has some practical debugging and tuning applications, like identifying chunks that are slow to compute.
Why does it take a long time to start doing anything?
The progress matrix only tracks terminal tasks that correspond directly to chunks in the computed output array. If your computation has a lot of intermediate tasks, you won't see any progress until those are completed.
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file dask_progress_matrix-0.0.1.tar.gz.
File metadata
- Download URL: dask_progress_matrix-0.0.1.tar.gz
- Upload date:
- Size: 7.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.12.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
de192f37c7699e6170d3f5e46fe72d9b320c5c3288e359eac661812fa940f3de
|
|
| MD5 |
eb214fb9d0ae3a861a05b72fa30db015
|
|
| BLAKE2b-256 |
ff1d6239b36ef816aabcb36c6d67ea9942a99ab516741943ba0c10a5cb71c503
|
Provenance
The following attestation bundles were made for dask_progress_matrix-0.0.1.tar.gz:
Publisher:
publish.yaml on aazuspan/dask-progress-matrix
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
dask_progress_matrix-0.0.1.tar.gz -
Subject digest:
de192f37c7699e6170d3f5e46fe72d9b320c5c3288e359eac661812fa940f3de - Sigstore transparency entry: 381847627
- Sigstore integration time:
-
Permalink:
aazuspan/dask-progress-matrix@016354830e598961a2db79f6e4e0b3770760fe13 -
Branch / Tag:
refs/tags/v0.0.1 - Owner: https://github.com/aazuspan
-
Access:
private
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yaml@016354830e598961a2db79f6e4e0b3770760fe13 -
Trigger Event:
push
-
Statement type:
File details
Details for the file dask_progress_matrix-0.0.1-py3-none-any.whl.
File metadata
- Download URL: dask_progress_matrix-0.0.1-py3-none-any.whl
- Upload date:
- Size: 11.2 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/6.1.0 CPython/3.12.9
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
595f373fc0d50710a74685ce1731a1371e85d8a3b25d00926c6b74183b03ca86
|
|
| MD5 |
20b90e2464a4efd4a840a6eacb503067
|
|
| BLAKE2b-256 |
321b4a4e467dfa1ab60cd1d0c24c72ba7da7cea90761513dc82290c1f038940b
|
Provenance
The following attestation bundles were made for dask_progress_matrix-0.0.1-py3-none-any.whl:
Publisher:
publish.yaml on aazuspan/dask-progress-matrix
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
dask_progress_matrix-0.0.1-py3-none-any.whl -
Subject digest:
595f373fc0d50710a74685ce1731a1371e85d8a3b25d00926c6b74183b03ca86 - Sigstore transparency entry: 381847648
- Sigstore integration time:
-
Permalink:
aazuspan/dask-progress-matrix@016354830e598961a2db79f6e4e0b3770760fe13 -
Branch / Tag:
refs/tags/v0.0.1 - Owner: https://github.com/aazuspan
-
Access:
private
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
publish.yaml@016354830e598961a2db79f6e4e0b3770760fe13 -
Trigger Event:
push
-
Statement type:
