This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Help us improve Python packaging - Donate today!
Project Description

Steam factory

Replacement for multiprocessing’s Pool, offering more powerful features. Allow running a generic Python function asynchronously.

Example usage

First, we are going to need a function that does something. In this case, it does nothing more than waiting one second.

import time

def do_nothing():
    time.sleep(1)
    print('Sleeping done')  # If you want some feedback..

To run this function in parallel, we’re going to need a Factory instance.

from steamfactory import Factory

# Create a factory, running up to 4 tasks concurrently
factory = Factory(size=4)

All set, we can schedule some async function executions:

for _ in range(4):
    factory.run(do_nothing)

After a second, you should see the four “Sleeping done” messages being printed at once.

In case you’re using this inside a script, and you need the main process to wait for all tasks to be executed before terminating (meaning that tasks will be lost), remember to call the shutdown() method:

factory.shutdown()

Getting feedback

How to get “feedback” from the tasks usually greatly depends on the application. Many times you don’t even bother with the function return value, you just need something to be done. Other times values might be large, or the required retention time might vary.

The library doesn’t currently offer any way to return results to the caller, but you can easily do something like this:

import time
from multiprocessing import Manager

from steamfactory import Factory

_mgr = Manager()
results = _mgr.dict()  # Shared between processes


def addup(a, b):
    time.sleep(1)
    results[(a, b)] = a + b

# Create a factory, running up to 4 tasks concurrently
factory = Factory(size=4)

# Let's schedule some tasks
factory.run(addup, 1, 2)
factory.run(addup, 3, 4)
factory.run(addup, 5, 6)
factory.run(addup, 7, 8)

factory.shutdown()

# Now, results contains all the results (after a 1s processing
# time)

Changelog

0.1

Initial release, implementing functionality to run functions asynchronously.

Release History

Release History

0.1

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
SteamFactory-0.1-py2-none-any.whl (5.7 kB) Copy SHA256 Checksum SHA256 2.7 Wheel Jun 12, 2016
SteamFactory-0.1-py3-none-any.whl (5.7 kB) Copy SHA256 Checksum SHA256 3.5 Wheel Jun 12, 2016
SteamFactory-0.1.tar.gz (3.9 kB) Copy SHA256 Checksum SHA256 Source Jun 12, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting