A wrapper that runs a function inside a new process, that is not a child of the main process.
Project description
burner_process
burner_process
is a wrapper that runs a function inside a newly spawned independent process that is not child to the main process, handling the function's arguments and returns (via pickle
) as well as all its dependencies.
Other solutions (such as this one) rely on subprocess
and thus spawn a process that is child to the main process, sharing memory and resources with it. burner_process
comes in handy when you need to run a function in an isolated environment, as a 'hard' workaround to memory leaks or inside parallel sub-processes, to monitor a specific function's resources usage.
Limitations
- The function has to be defined in a module different from the main, and the functions's module must not include importing
burner_process
(see example below), which would cause circular imports. - Both arguments and returns must be pickleable
Features
- Supporting overlapping/parallel calls
- Not relying on
subprocess
normultiprocessing
- Not relying on a queue for the returns
- Tested on Windows and Linux
Usage
Installation
pip install burner_process
Example
my_module.py
def my_function(x):
return x * 2
my_script.py
from burner_process import processify
from my_module import my_function
my_processified_function = processify(my_function)
my_processified_function(5)
>> 10
Verbose example
my_module.py
def my_function(x):
return x * 2
my_script.py
from burner_process import processify
from my_module import my_function
my_processified_function = processify(my_function, verbose=True)
my_processified_function(5)
>> Launching from main process (pid: 20124, parent pid: 10568)
>> Serializing arguments to C:\...\io\3
>> Running inside spawned process (pid: 18976, parent pid: 14584)
>> Serializing outputs to C:\...\io\3
>> 10
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for burner_process-1.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 99b0590fb675dc8e0fc0b05ef72b4441c2124ff6926c8760a0f940e91e7bee88 |
|
MD5 | 0397fd97a4cd746ae3ee617d261a43bb |
|
BLAKE2b-256 | 1f9591c321226ffc86a9377a7e0c2f573c0442a5d604077afa26e380850efffe |