gcl-looper 1.2.0

Looper is a daemonizer library, it can help you with lifecycle of your daemon.

GenesisCoreLibs Looper Documentation

Overview

GCL Looper is a Python library designed to create daemon-like services that can run indefinitely, performing tasks at regular intervals or on demand.

Usage Examples

Basic Service

• Iterate infinitely
• There should be at least 5 seconds between start of previous and next iteration (iter_min_period)
• pause for 1 second between iterations (iter_pause)

from gcl_looper.services import basic

class MyService(basic.BasicService):
    def __init__(self, iter_min_period=5, iter_pause=1):
        super(MyService, self).__init__(iter_min_period, iter_pause)

    def _iteration(self):
        print("Iteration", self._iteration_number)

service = MyService()
service.start()

Finite Service without any pauses in-between

from gcl_looper.services import basic

class MyFiniteService(basic.BasicService):
    def __init__(self, iter_min_period=0, iter_pause=0):
        super(MyFiniteService, self).__init__(iter_min_period, iter_pause)
        self.countdown = 3

    def _iteration(self):
        if self.countdown > 1:
            self.countdown -= 1
        else:
            self.stop()

service = MyFiniteService()
service.start()

API service with database (restalchemy)

from gcl_looper.services import bjoern_service
from gcl_looper.services import hub
from oslo_config import cfg
from restalchemy.storage.sql import engines
from restalchemy.common import config_opts as db_config_opts

from MY_PACKAGE.user_api import app

api_cli_opts = [
    cfg.StrOpt(
        "bind-host", default="127.0.0.1", help="The host IP to bind to"
    ),
    cfg.IntOpt("bind-port", default=8080, help="The port to bind to"),
    cfg.IntOpt(
        "workers", default=1, help="How many http servers should be started"
    ),
]

DOMAIN = "user_api"

CONF = cfg.CONF
CONF.register_cli_opts(api_cli_opts, DOMAIN)
db_config_opts.register_posgresql_db_opts(conf=CONF)


def main():

    serv_hub = hub.ProcessHubService()

    for _ in range(CONF[DOMAIN].workers):
        service = bjoern_service.BjoernService(
            wsgi_app=app.build_wsgi_application(),
            host=CONF[DOMAIN].bind_host,
            port=CONF[DOMAIN].bind_port,
            bjoern_kwargs=dict(reuse_port=True),
        )

        service.add_setup(
            lambda: engines.engine_factory.configure_postgresql_factory(
                conf=CONF
            )
        )

        serv_hub.add_service(service)

    serv_hub.start()


if __name__ == "__main__":
    main()

Public interface:

• start(): Starts the service.
• stop(): Stop the service.
• _loop_iteration(): Performs one iteration of the service loop.

Implement these methods to get usable service:

• _iteration(): This method must be implemented by subclasses to perform the actual work at each iteration.

Process Hub service

Process Hub allows running multiple services in separate processes. It's useful when you want to run multiple instances of a service (e.g., multiple API workers) or different services that should be isolated.

Security Feature: Privilege Downgrade

When using ProcessHubService, you can set __mp_downgrade_user__ on a child service to automatically downgrade process privileges after the fork. This is a security best practice to minimize attack surface - start as root (if needed to bind to privileged ports), then downgrade to an unprivileged user.

• __mp_downgrade_user__: Class attribute set to a username (e.g., "nobody"). When set, the child process will downgrade to this user after forking. Default is None (no downgrade).

from gcl_looper.services import hub
from gcl_looper.services import bjoern_service

serv_hub = hub.ProcessHubService()

# BjoernService has __mp_downgrade_user__ = "nobody" by default
for _ in range(4):  # 4 workers
    service = bjoern_service.BjoernService(
        wsgi_app=my_app,
        host="0.0.0.0",
        port=80,  # Privileged port, needs root to bind
        bjoern_kwargs=dict(reuse_port=True),
    )
    serv_hub.add_service(service)

serv_hub.start()
# Each worker starts as root to bind port 80, then downgrades to 'nobody'

Note: This feature only works on Linux and requires the process to start as root. The target user must exist on the system.

Manual Privilege Downgrade

You can also manually downgrade privileges using the utility function:

from gcl_looper import utils

# Downgrade to 'nobody' user
utils.downgrade_user_group_privileges("nobody")

Launchpad Service

Launchpad service is a service that can run multiple services and execute them sequentially. It's convenient when you have multiple services that need to be run in a specific order or the services aren't heavy and you don't want to use multiprocessing. Also it simplifies the configuration of the services.

Basic usage:

from gcl_looper.services import launchpad

services = [
    MyService(),
    MyFiniteService(),
]

service = launchpad.LaunchpadService(services)
service.start()

The most important part in the launchpad service is its configuration. In the configuration you specify how to run inner services, how to configure them and how to initialize them.

Configuration options:

• services: List of services to run. Each service can be specified as a string in the format module.path:ServiceName::count where count is optional and defaults to 1.
• common_registrator_opts: Common options for all services. These options are passed to the service constructor.
• common_initializer: Common initializer for all services. This initializer is called after the service is created and before it is started.
• iter_min_period: Minimum period between iterations of the service loop.
• iter_pause: Pause between iterations of the service loop.

Example:

[DEFAULT]
verbose = True
debug = True

[launchpad]
services =
    my_package.service_foo:FooService,
    my_package.service_bar:BarService,
    my_package.service_baz:BazService
common_registrator_opts = my_package.service_common:common_opts
common_initializer = my_package.service_common:common_init

[my_package.service_foo:FooService]
name = foo

[my_package.service_bar:BarService]
name = bar
project_id = 123

[my_package.service_baz:BazService]
param1 = value1
param2 = value2

Example with multiple instances of the same service:

[DEFAULT]
verbose = True
debug = True

[launchpad]
services =
    my_package.service_foo:FooService,
    my_package.service_bar:BarService::2

[my_package.service_foo:FooService]
name = foo

[my_package.service_bar:BarService::0]
name = bar0
project_id = 123

[my_package.service_bar:BarService::1]
name = bar1
project_id = 456