Skip to main content

Run code on a dask worker via a context manager

Project description

Afar

Python Version Version License Code style

One man's magic is another man's engineering
Robert A. Heinlein


afar explores new syntax around context managers. For example:

import afar
with afar.run() as results, locally:
    x = 1
    y = x + 1

>>> results.x
1
>>> results.y
2

Soon, we will be able to run code on a remote dask worker with syntax like:

with afar.run() as result, remotely:
    import dask_cudf
    df = dask_cudf.read_parquet("s3://...")
    result = df.sum().compute()

For motivation, see https://github.com/dask/distributed/issues/4003

This code is highly experimental and magical!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

afar-0.0.1.tar.gz (21.3 kB view hashes)

Uploaded Source

Built Distribution

afar-0.0.1-py3-none-any.whl (5.3 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page