Skip to main content

Python producer-customer model based on redis

Installment

use pip:

pip install woodenwaiter

from source code:

python3 setup.py install

Run Example

python3 -m woodenwaiter.woodenwaiter

Example Usage

A simple program that showing how to use WoodenWaiter as a customer and producer

customer example:

import time

from woodenwaiter.woodenwaiter import WoodenWaiter
from woodenwaiter.woodenwaiter import WoodenCustomer
from woodenwaiter.woodenwaiter import WoodenManager

if __name__ == '__main__':
    table1 = 'cmdb'
    table2 = 'rbac'
    dish1 = 'custom_sync'
    dish2 = 'some_task'
    waiter = WoodenWaiter()

    def print_foods(foods):
        print('custom foods')
        print('foods is: {}'.format(foods))

    customer1 = WoodenCustomer(
            table=table1, dish=dish1, waiter=waiter,
            process=print_foods, seconds=1)
    customer2 = WoodenCustomer(
            table=table2, dish=dish2, waiter=waiter,
            process=print_foods, seconds=3)

    manager = WoodenManager()
    manager.add_customer(customer1)
    manager.add_customer(customer2)
    manager.launch()

    while True:
        time.sleep(1)

producer example:

import time
import random

from woodenwaiter.woodenwaiter import WoodenMenu
from woodenwaiter.woodenwaiter import WoodenWaiter
from woodenwaiter.woodenwaiter import WoodenCooker

if __name__ == '__main__':
    table1 = 'cmdb'
    table2 = 'rbac'
    dish1 = 'custom_sync'
    dish2 = 'some_task'
    foods1 = {
        "action": "sync_custom_data",
        "paras": ""
    }
    foods2 = {
        "action": "some_action",
        "paras": {
            "para1": "value1",
            "para2": "value2"
        }
    }
    menu1 = WoodenMenu(table=table1, dish=dish1, foods=foods1)
    menu2 = WoodenMenu(table=table2, dish=dish2, foods=foods2)
    waiter = WoodenWaiter()
    cooker1 = WoodenCooker(menu=menu1, waiter=waiter)
    cooker2 = WoodenCooker(menu=menu2, waiter=waiter)

    cooker_running = True
    def cook_sometime():
        while cooker_running:
            seconds = random.randint(3, 10)
            time.sleep(seconds)
            print('cookone after {} seconds'.format(seconds))
            cooker1.cookone()
            cooker2.cookone()

    cooker_thread = threading.Thread(target=cook_sometime)
    cooker_thread.start()

    while True:
        try:
            time.sleep(1)
        except KeyboardInterrupt:
            cooker_running = False
            manager.terminate_all()
            break

classes introduce

Interoduce Classes of WoodenWaiter

WoodenMenu

Each WoodenCooker cook ONE kind of food according to ONE WoodenMenu instance. We create WoodenMenu instance by offer a table(maybe your program model), a dish(maybe one kinds of your task) and foods(some informations the customer need)

suggested format of parameter ‘foods’:

foods = {
    "action": "要执行的任务字符串",
    "paras": {"para1": "value1", "para2": "value2"}
}

WoodenWaiter

Each WoodenWater connect ONE redis database. We use WoodenWaiter to create WoodenCooker instance and WoodenCustomer instance.

WoodenWater take dish from WoodenCooker(push task to redis list) and serve dish to WoodenCustomer(pop task from redis).

WoodenCooker

Creating an WoodenCooker need a WoodenMenu and a WoodenWaiter.

Methods:

set_menu(self, menu) - set WoodenMenu

set_waiter(self, waiter) - set WoodenWaiter

cookone(self, menu=None) - push a task to redis

WoodenCustomer

WoodenCustomer take four parameters:

  • table: maybe your program model

  • dish: maybe one kind of your task

  • waiter: a WoodenWaiter instance

  • process: a function that accept a dictionary parameter. This function will be call when waiter serve a dish of food(when pop a task from redis).

  • seconds: the seconds of the cycil that waiter check the redis.

WoodenCustomer is inherented from threading.Thread. But start the thread yourself is not suggested. Please use WoodenManager instand.

Methods:

call_waiter(self) - call waiter to check if foods is OK(if there is a task in redis)

call_waiter_cyclic(self, seconds) - call waiter cyclic

terminate(self) - stop customer thread genteely

WoodenManager

WoodenManager is used for WoodenCustomer centralized management. We add woodenCustomer instance to it, and then launch the threads

Methods:

add_customer(self, customer) - add WoodenCustomer instance

launch(self) - launch all WoodenCustomer Threads

terminate_all(self) - stop all customers thread genteely

Release files for woodenwaiter 0.0.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for woodenwaiter 0.0.4
File Size Uploaded
woodenwaiter-0.0.4.tar.gz 4.7 kB Details

Built distributions (wheels)

Table of built distributions (wheels) for woodenwaiter 0.0.4
File Interpreter ABI Platform
woodenwaiter-0.0.4-py3.5.egg Legacy Egg format - - Details
woodenwaiter-0.0.4-py3-none-any.whl Python 3 none any Details

Total release size: 21.0 kB

Release files / woodenwaiter-0.0.4.tar.gz

Download URL woodenwaiter-0.0.4.tar.gz
Size 4.7 kB
Tags Source
SHA-256 checksum
How to use checksums
96450aebf8fa494d8414d2e17e29baf1cc7b1bd752cacbf294509ecf6c242da0
BLAKE2b-256 checksum
How to use checksums
224171e71d8c4486cddda9c2bad555cfb7a9d13e037897635cf7e95663d0ea3c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release files / woodenwaiter-0.0.4-py3.5.egg

Download URL woodenwaiter-0.0.4-py3.5.egg
Size 8.9 kB
Tags Egg
SHA-256 checksum
How to use checksums
9a3bc9ac0b221488e6bd63d8cb7c3bb7a400e8d4c940684ade20ac5db93f3f43
BLAKE2b-256 checksum
How to use checksums
462e5cb15a8c8fbe7f8e0510bf30c192bfc7da217f50dcbb7b50ea73d4c17d23
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release files / woodenwaiter-0.0.4-py3-none-any.whl

Download URL woodenwaiter-0.0.4-py3-none-any.whl
Size 7.4 kB
Tags Python 3
SHA-256 checksum
How to use checksums
369ef5f3e84ced55f2209f99bd534dd5cd1bb6e2f4e419bd2d1aa8f672fe360a
BLAKE2b-256 checksum
How to use checksums
ced9759eca05d16162eeabbbc95351926490d511adce09b3b54ff2b1d2d9dc9a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No

Release history Release notifications | RSS feed

This release

0.0.4 This release

3 release files

0.0.3

3 release files

0.0.2

1 release file

0.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page