Connor's Process Managerial (PM)

Connor's Process Managerial (PM) is a utility that processes and enques functions in a separate single thread. This is useful to handle multiple multiprocessed functions, and allows them all to run over a period of time without causing complex issues with multiprocessing.

To install: pip install process-managerial

Example code of usage:

from process_managerial import QueueSystem
import time
from pprint import pprint

# Sample function that succeeds by adding two numbers.
def task_success(x, y):
    time.sleep(2)
    return x + y

# Sample function that fails by raising an exception.
def task_failure():
    time.sleep(1)
    raise ValueError("Intentional error for demonstration.")

# Sample function that prints a message.
def task_print(message):
    time.sleep(1)
    print(message)
    return message

def main():
    # Instantiate the QueueSystem with a process directory and log file.
    qs = QueueSystem(process_dir="processes", log_path="queue_log.txt")
    
    # Start the background worker thread.
    qs.start_queuesystem()
    
    # Queue a task that will successfully complete.
    hex_success = qs.queue_function(task_success, 5, 10)
    print(f"Queued task_success with hex: {hex_success}")
    
    # Queue a task that is designed to fail.
    hex_failure = qs.queue_function(task_failure)
    print(f"Queued task_failure with hex: {hex_failure}")
    
    # Queue a task that prints a message.
    hex_print = qs.queue_function(task_print, "Hello, QueueSystem!")
    print(f"Queued task_print with hex: {hex_print}")
    
    # Wait individually for each task to finish.
    qs.wait_until_hex_finished(hex_success)
    qs.wait_until_hex_finished(hex_failure)
    qs.wait_until_hex_finished(hex_print)
    
    # Wait until the entire queue has been processed.
    qs.wait_until_finished()
    
    # Optionally, requeue the failed task for another attempt.
    print("Requeuing the failed task...")
    qs.requeue_hex(hex_failure)
    qs.wait_until_hex_finished(hex_failure)
    
    # Stop the background worker thread.
    qs.stop_queuesystem()
    
    # Retrieve and display final task statuses.
    print("\nFinal task statuses:")
    for hex_val in qs.get_hexes():
        props = qs.get_properties(hex_val)
        pprint({
            'hex': hex_val,
            'status': props.status.name,
            'result': props.result,
            'output': props.output,
            'start_time': props.start_time,
            'end_time': props.end_time
        })
    
    # Optional: Clear tasks older than a certain datetime.
    # Uncomment the following lines to remove pickle files for tasks with a start time before the cutoff.
    # import datetime
    # cutoff_time = datetime.datetime.now() - datetime.timedelta(seconds=5)
    # qs.clear_hexes(before_date=cutoff_time)
    # print("\nCleared tasks before:", cutoff_time)

if __name__ == "__main__":
    main()

.venv/bin/python -m build

.venv/bin/python3 -m twine upload --repository process_managerial dist/*

Metadata

Release files for process-managerial 0.1.38

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for process-managerial 0.1.38
File	Size	Uploaded
process_managerial-0.1.38.tar.gz	10.4 kB	Apr 22, 2025	Details

Built distribution (wheel)

Table of built distributions (wheels) for process-managerial 0.1.38
File	Interpreter	ABI	Platform	Reset
process_managerial-0.1.38-py3-none-any.whl 10.1 kB Apr 22, 2025	Python 3	none	any	Details

Total release size: 20.5 kB

Release files / process_managerial-0.1.38.tar.gz

Download URL	process_managerial-0.1.38.tar.gz
Size	10.4 kB
Tags	Source
SHA-256 checksum How to use checksums	`f031ecaa17543f6426f595c1457dee2f240720f85fa396add1c5361d13e058d0`
BLAKE2b-256 checksum How to use checksums	`4d3a1934e9be0e407a3c1ed399bc5882456778d20908d67e4ca694d129f2ae2a`
Upload date	Apr 22, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.1.0 CPython/3.12.3`

Release files / process_managerial-0.1.38-py3-none-any.whl

Download URL	process_managerial-0.1.38-py3-none-any.whl
Size	10.1 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`d7033a1a06bd6847f3a437ad2f94c407f85ffe664e1c5389046af1f2b28b9437`
BLAKE2b-256 checksum How to use checksums	`3c8d156a0ad1f0e7474b3959dd3d2e195094f518c532f60e6eebad5b519f6435`
Upload date	Apr 22, 2025
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/6.1.0 CPython/3.12.3`