Skip to main content

Contributor Covenant PyPI badge Anaconda-Server Badge License: BSD 3-Clause

Cyclebane

About

Transform directed acyclic graphs using map-reduce and groupby operations

This library is an attempt to merge the concepts of directed acyclic graphs (DAG) with array-like objects such as NumPy arrays, Pandas DataFrames, or Xarray/Scipp DataArrays. This could be useful for describing tasks graphs, e.g., when a series of tasks is applied to chunks of an array. These tasks also have an array structure. After an reduction operation of chunks, the graph loses this structure, i.e., only a subset of the graph's nodes has array structure. What if we could work with this structure, even though only parts of the graph follows it? And what if we could use the power of array slicing with named dimensions, or select by label? This is what Cyclebane tries to do.

Our initial goal is to support:

  • map operations of a DAG's source nodes over an array-like (https://docs.dask.org/en/latest/high-level-graphs.html). Cyclebane will effectively copy all descendants of those nodes, once for each array element. Cyclebane will support joint mappings of multiple source nodes by mapping over, e.g., a DataFrame with multiple columns, as well as chaining independent map operations at different source nodes. In the latter case this will effectively broadcast at descendant nodes that depend on multiple such source nodes.
  • reduce operations at descendants of mapped nodes. This will add a new node with edges to all copies of the mapped node being reduced. Cyclebane will support reducing only individual axes or all axes, similar to Numpy.
  • groupby operations similar to Pandas and Xarray (albeit more limited).
  • Positional and label-based indexing. Cyclebane will support selecting branches that were creating during map (or groupby) operations based on their indices. The graph structure will be left untouched, i.e., nodes after a reduce operation will be preserved, but fewer edges will lead to the reduce node.

See also Dask's High Level Graphs for a related concept (without the direct support for any such operations).

Installation

python -m pip install cyclebane

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

cyclebane-26.9.0.tar.gz (54.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

cyclebane-26.9.0-py3-none-any.whl (17.3 kB view details)

Uploaded Python 3

File details

Details for the file cyclebane-26.9.0.tar.gz.

File metadata

  • Download URL: cyclebane-26.9.0.tar.gz
  • Upload date:
  • Size: 54.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for cyclebane-26.9.0.tar.gz
Algorithm Hash digest
SHA256 60ad6bf20cf0e4fd0217c3f56aaf83ed8e90e14497dca5875f84a4b81cab4c1a
MD5 a8e68ad4c1ed3f1a5615cdbc43d6372c
BLAKE2b-256 b085ee7312ffdce098b26afa03b54a81b2ecaddd53fb2149208f790fed6c1e79

See more details on using hashes here.

Provenance

The following attestation bundles were made for cyclebane-26.9.0.tar.gz:

Publisher: release.yml on scipp/cyclebane

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file cyclebane-26.9.0-py3-none-any.whl.

File metadata

  • Download URL: cyclebane-26.9.0-py3-none-any.whl
  • Upload date:
  • Size: 17.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.13

File hashes

Hashes for cyclebane-26.9.0-py3-none-any.whl
Algorithm Hash digest
SHA256 cbbbe12f92be969a9e46147416a494ad118ed3e2b7d9870a33eb84140d4d92dd
MD5 bcedb96354546a008ff3b7327610b035
BLAKE2b-256 858def78d56d41eb1bfa61350bb955038072966f8abf2ac01749368a1f6ee7d5

See more details on using hashes here.

Provenance

The following attestation bundles were made for cyclebane-26.9.0-py3-none-any.whl:

Publisher: release.yml on scipp/cyclebane

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Release history Release notifications | RSS feed

This release

26.9.0 This release

2 files

24.10.0

2 files

24.6.0

2 files

24.5.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page