buckshot

The buckshot library contains multiprocessing functions designed to help developers create distributed applications quickly.

Installation

The easiest way to install buckshot is via pip:

$ pip install buckshot

Usage

Currently, there are two ways of leveraging buckshot to distribute work across processes: the @distribute decorator and with distributed(...) context manager.

The following example shows how we can distibute the work of calculating many harmonic sum values across processes.

Serial Approach

import fractions

def harmonic_sum(x):
    F = fractions.Fraction
    return sum(F(1, d) for d in xrange(1, x + 1))

for value in range(1, 100):
    result = harmonic_sum(value)
    print result

Using @distribute

import fractions

from buckshot import distribute

@distribute(processes=4)
def harmonic_sum(x):
    F = fractions.Fraction
    return sum(F(1, d) for d in xrange(1, x + 1))

for result in harmonic_sum(range(1, 100)):  # Pass in values list.
    print result

Using with distributed(...)

import fractions

from buckshot import distributed

def harmonic_sum(x):
    F = fractions.Fraction
    return sum(F(1, d) for d in xrange(1, x + 1))

with distributed(harmonic_sum, processes=4) as distributed_harmonic_sum:
    for result in distributed_harmonic_sum(range(1, 100)):
        print result

All processes are destroyed when inputs are exhausted and/or the context is exited.

Known Issues

Function inputs must be picklable.
Function outputs must be picklable.
If a child process is killed externally, buckshot will block forever waiting for results.
This uses os.fork() under the hood, so there is a risk of rapidly exhausting memory.

LICENSE

Read the LICENSE file for details.

Metadata

Release files for buckshot 0.0.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for buckshot 0.0.10
File	Size	Uploaded
buckshot-0.0.10.tar.gz	10.0 kB	Sep 7, 2016	Details

Release files / buckshot-0.0.10.tar.gz

Download URL	buckshot-0.0.10.tar.gz
Size	10.0 kB
Tags	Source
SHA-256 checksum How to use checksums	`d34355e3710bb07e489646f8340ce29ac612db86acd810908d966834808e5311`
BLAKE2b-256 checksum How to use checksums	`9ad3bf9c89a88dcaaf654703818106f99723328f809a1af35d39ae4f271dab76`
Upload date	Sep 7, 2016
Uploaded using Trusted Publishing? What is trusted publishing?	No