Shared generator between process
Project description
Python library : cm_shared_generator
Resume
In python, it exist a generator type which can be really useful.
The cm_shared_generator
library provide a Shared_Generator which share generator between thread and process safely.
Install
Pre-required
Global requirements
- Python 3 (
sudo apt update && sudo apt install python3 -y
)
Install requirements
- Setuptools (
sudo apt update && sudo apt install python3-pip -y && pip3 install setuptools
)
Installation
python3 setup.py install
Utilization
The utilization of the Shared_Generator is quite simple. Its prototype is like :
Shared_Generator(self, arg_generator, DEFAULT_GENERATOR_TRAITMENT_FUNCTION=None, FORCE=False, MAX_OBJECT_IN_QUEUE=len(os.sched_getaffinity(0)), TIMEOUT=None)
Its parameters are defined as :
- Required :
arg_generator
-> The generator to shared
- Optional :
DEFAULT_GENERATOR_TRAITMENT_FUNCTION
(Default : None) -> Function to treat default type of value from the generator. Possible value :Shared_Generator.DEFAULT_GENERATOR_TRAITMENT_CLASS
(Create a deep copy of the class before the insertion into the Queue).- Your own function !
FORCE
(Default : False) -> If True, the generator will be defined even his type is not a proper generator.MAX_OBJECT_IN_QUEUE
(Default : Nb of logical processors) : Defined the maximum of elements in the queue at the same time. ( > 0 )NUMBER_PUT_RETRY
(Default : None) -> Number of retry to acquire the semaphore which define if the generator process can continue to fill the Generator list before quitting timeout.TIMEOUT
(Default :Shared_Generator.DEFAULT_TIMEOUT
) : Defined the timeout of Semaphores (The Shared_Generator will be stopped after (1+NUMBER_PUT_RETRY
)*TIMEOUT
if no process has taken a value from the generator.
Example :
from cm_shared_generator import Shared_Generator
# Simple integer generator
def int_generator():
i = 0
while i<10:
i+=1
yield i
return
my_shared_generator = Shared_Generator(int_generator, MAX_OBJECT_IN_QUEUE=2) # Creation of the generator with a Maximum lenght queue of 2
my_shared_generator.start(NEW_PROCESS=True) # Launch the generator in a new process
val = my_shared_generator.next() # Get the next value of the generator
my_shared_generator.stop() # Stop the generator
For more examples, check unit tests in tests directory.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for cm_shared_generator-0.1.4.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 898b7abebf138a14d871da5e76b749e73bcc561d1f6ceb331d1aed0049408a7f |
|
MD5 | 8b07c169aa55b5a0e4ca89974e4d3ad0 |
|
BLAKE2b-256 | e4177edbd5a4f3f7500dc1157c264c569fbdaa0b0b2164289257a60d8674f4f9 |
Close
Hashes for cm_shared_generator-0.1.4-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0a1fa31ad5e205ddc446565c4fea55921f9b659648cfb52c478a5934e49d28d9 |
|
MD5 | abd223c4760dbd67566998441d9d9c00 |
|
BLAKE2b-256 | 34631f456a6384ef219152c6569ecbf8b1b05910faefa4bae7f4177a80e69051 |