This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

The pynisher is a little module intended to limit a functions resources. It starts a new process, sets the desired limits, and executes the function inside it. In the end, it returns the function return value. If, for any reason, the function call is not successful, None is returned.

Currently, the total memory usage(*), wall and cpu time, and the number of subprocesses can be limited.

(*) As the subprocess also includes the Python interpreter, the actual memory available to your function is less than the specified value.

To show the basic usage, consider the following script

import pynisher
import time

# using it as a decorator for every call to this function
@pynisher.enforce_limits(wall_time_in_s=2)
def my_function (t):
        time.sleep(t)
        return(t)

for t in range(5):
        print(my_function(t))

The full list of argments to enforce_limits reads:

pynisher.enforce_limits(mem_in_mb=None, cpu_time_in_s=None,\
                        wall_time_in_s=None, num_processes=None,\
                        grace_period_in_s = None, logger = None)

The first four are actual constraints on the memory, the CPU time, the wall time, and the number of subprocesses of the function. All values should be integers or None, which means no restriction.

The grace period allows the function to properly end. More technically, the subprocess receives a SIGXCPU/SIGALARM signal if the CPU/wall clock limit is reached. After the grace period a SIGKILL is send terminating the process immediately. Without a grace period, pynisher might not be able to correctly determine the cause of the shutdown, as the subprocess might die without any notice (more on that below).

The logger is used to display additional information about the status of the pynisher module (mostly debug level). This might come in handy while debugging.

If you need to know what happend to the function call or why it was aborted, you can use the object returned from pynisher.enforce_limits. Consider this slight variation of the above example:

import pynisher
import time

def my_function (t):
        time.sleep(t)
        return(t)

for t in range(5):
        obj = pynisher.enforce_limits(wall_time_in_s=2)(my_function)
        result = obj(t)
        print(result, obj.result, obj.exit_status, obj.wall_clock_time)

# see all the available information
print(vars(f))

The object `obj` can be used as the original function. After calling it, it contains the actual result, but also an indicator of what happend. The `exit_status` attribute is either zero (function returned properly) or one of the following exceptions:

pynisher.CpuTimeoutException    # CPU time limit was reached
pynisher.TimeoutException       # Wall clock time limit exceeded
pynisher.MemorylimitException   # function hit the memory constraint
pynisher.SubprocessException    # function tried to spawn too many subprocesses
pynisher.AnythingException      # Something else went wrong, e.g., your function received a signal and just died.

Here, the above issue about the grace period becomes interesting. Without it, it is likely that a AnythingException is returned where a Cpu-/TimeoutException would be appropriate.

Release History

Release History

0.4.2

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.3.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
pynisher-0.4.2.tar.gz (4.9 kB) Copy SHA256 Checksum SHA256 Source Apr 18, 2016

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting