A wrapper that runs a function inside a new process, that is not a child of the main process.
Project description
Burner Process
Burner Process is a wrapper that runs a function inside a newly spawned independent process that is not child to the main process, handling the function's arguments and returns (via pickle
) as well as all its dependencies.
Other solutions (such as this one) rely on subprocess
and thus spawn a process that is child to the main process, sharing memory and resources with it. Burner Process comes in handy when you need to run a function in an isolated environment, as a 'hard' workaround to memory leaks or inside parallel sub-processes, to monitor a specific function's resources usage.
Limitations
- The function has to be defined in a module different from the main, and the functions's module must not include importing
burner_process
(see example below), which would cause circular imports. - Both arguments and returns must be pickleable
Features
- Supporting overlapping/parallel calls
- Not relying on
subprocess
normultiprocessing
- Not relying on a queue for the returns
- Tested on Windows and Linux
Usage
Installation
pip install burner-process
Example
my_module.py
def my_function(x):
return x * 2
my_script.py
from burner_process import processify
from my_module import my_function
my_function_p = processify(my_function)
my_function_p(5)
>> 10
Verbose example
my_module.py
def my_function(x):
return x * 2
my_script.py
from burner_process import processify
from my_module import my_function
my_function_p = processify(my_function, verbose=True)
my_function_p(5)
>> Launching from main process (pid: 20124, parent pid: 10568)
>> Serializing arguments to C:\...\io\3
>> Running inside spawned process (pid: 18976, parent pid: 14584)
>> Serializing outputs to C:\...\io\3
>> 10
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for burner_process-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d45da372b84feaaa7dabec74c739c5425f0ece61a2b77b7cdccb87782763e583 |
|
MD5 | ac6f6dcf6e6d0e9904cb3b325b3205a2 |
|
BLAKE2b-256 | 0b902e02660adce4f21ff8cbf84e6c6417d4db4744263c2e95e50c3796c99e28 |