distributed processing for eppyy
Project description
zeppy
distributed processing for eppyy
Free software: Mozilla Public License 2.0 (MPL-2.0)
Documentation: https://zeppy.readthedocs.io.
Vision
To run eppy on multiple nodes in parallel and collect the results.
So what is a node and why would you want to do this ?
A node can be any or all of the following:
- a process (such E+ running on a single core on a multi-core computer)
so we can do multi-processing and run it on many cores on a single computer
- a computer
so we can run it on multiple computers that are on the same network
- a group of group of computer in a local network
So we can run multiple groups of machines that may be at different locations on different local networks
This can also be computers at different cloud locations
a single computer in the local network may act as an access node
Features
Do the distributed processing with a single function call and get all the results back.
Sample code
import zeppy result = zeppy.zmq_parallelpipe(runfunction, args_list, nworkers=None) # runfunction is a function you will write, # that may run idf.run(), # gather the total energy use and retrn it # args_list = {args: [idf1, idf2, idf3, ...]} # list of files to run # if nworkers=None: # it will start up as many nodes as there are items in args_list # if you don't have enough nodes avaliable, you can set nworkers=n. # it will start up n nodes and queue up the runs evenly on the nodes
For example the above code can do the following:
runfunction will run the idf file, and return the total energy usage
result will be a list total energy usage in the same order as the items in args_list
see the comments in the code for greater clarity
Why is the function called zmq_parallelpipe ?
This package uses ZeroMQ Library
Parallel Pipeline is a fundamental pattern in it
This is the first pass at the API. It doesn’t work yet - Ha!. But there is hope :-)
Credits
This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template.
History
0.1.0 (2020-04-19)
First release on PyPI.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for zeppy-0.1.0-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 541420b0388f03ac805009cc6534536af70b401d8966b02ad53a3fce3edae527 |
|
MD5 | 306f320f52ab290378c7b9ab63833e59 |
|
BLAKE2b-256 | 00037f73f29575069df40f1c0882b2105d6be3e889ea0684247fb80f0490850c |