rapid development of embarrassingly parallelizable applications
Project description
Poly provides a framework for rapidly developing distributed applications where the number of work units to be performed is known in advance. It causes many of the hassles of distributed computing to be handled automatically, such as:
assigning work units to jobs
reassembling one output file from the standard outputs of many jobs
copying files between servers and execution hosts
cleaning up temporary files on execution hosts
job and I/O throttling to reduce load on servers
Poly currently includes drivers for running under Platform LSF and in standalone mode, but drivers for other distributed computing environments can be added easily.
Poly also includes applications to distribute processing directly from the shell, with no additional programming needed.
