powerfunc 0.3.0

Make functions run however you like — CLI, pipeline, or remote compute

powerfunc

Add superpowers to your functions. Run them from the CLI, on cloud data, or on cloud compute.

Features

• Load and save function arguments and results to file paths, local or in cloud storage.
• Invoke functions on the command line.
• Remote execution on cloud compute.
• YAML configuration file support.
• Supports pandas, polars, PyArrow, Dask, and the Python csv module out of the box
• Supports cloud providers Google Cloud and Modal.
• Extensible for new data types and cloud providers.

Installation

pip install powerfunc

With Google Cloud remote execution:

pip install 'powerfunc[gcp]'

With Modal remote execution:

pip install 'powerfunc[modal]'

Example usage

Decorate a normal function with @powerfunc:

import pandas as pd
from powerfunc import powerfunc

@powerfunc
def sum_col(df: pd.DataFrame) -> float:
    return float(df["value"].sum())

df = pd.DataFrame({"value": [1, 2, 3]})
print(sum_col(df))

Then use it on files, local or in the cloud:

print(sum_col("data.csv"))
print(sum_col("https://storage.googleapis.com/powerfunc/data.csv"))

Add one line at the end of your file to make it runnable on the command line:

from powerfunc import powerfunc

...

powerfunc.enable_cli()

python myfile.py gs://powerfunc/data.csv

Support

This project is in early development. Breaking changes may happen at any time.

Documentation

For complete information on using powerfunc, including executing on cloud compute, please see the documentation.

License

Released under the MIT license.