A library for parallel execution of Python code in the Ufora runtime
pyfora - Compiled, parallel python
pyfora is the client package for Ufora - a compiled, automatically parallel Python for data science and numerical computing.
Ufora achieves speed and scale by reasoning about your python code to compile it to machine code (so it’s fast) and find parallelism in it (so that it scales). The Ufora runtime is fully fault tolerant, and handles all the details of data management and task scheduling transparently to the user.
The Ufora runtime is invoked by enclosing code in a “ufora.remote” block. Code and objects are shipped to a Ufora cluster and executed in parallel across those machines. Results are then injected back into the host python environment either as native python objects, or as handles (in case the objects are very large). This allows you to pick the subset of your code that will benefit from running in Ufora - the remainder can run in your regular python environment.
For all of this to work properly, Ufora places one major restriction on the code that it runs: it must be “pure”, meaning that it cannot modify data structures or have side effects. This restriction allows the Ufora runtime to agressively reorder calculations, which is crucial for parallelism, and allows it to perform compile-time optimizations than would not be possible otherwise. For more on the subset of python that Ufora supports, see python restrictions.
The pyfora client is a pure python package and can be installed by running:
pip install pyfora
Getting Started with Ufora
The ufora backend is available as a docker image that can be run locally on your machine, or in a cluster of machines on a local network or in the cloud.
Release date: UNRELEASED
- Initial pre-release of pyfora!
- Includes support for core language features and builtin types.
- Some support for builtin functions like all, any, sum, etc.
- pyfora.aws module and pyfora_aws script help setup a Ufora cluster in EC2.
Release history Release notifications
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size pyfora-0.1a2-py2-none-any.whl (84.6 kB)||File type Wheel||Python version 2.7||Upload date||Hashes View hashes|
|Filename, size pyfora-0.1a2.tar.gz (64.1 kB)||File type Source||Python version None||Upload date||Hashes View hashes|