Skip to main content

Asyncio plugins, components, dependency injection and configs

Project description

This is heavily inspired by Pyramid and my daily needs to fastly create and maintain microservice like applications.

a plugin mechanic

  • plugin may depend on other plugins

  • plugins yield tasks to run

  • a context registry serves as a store for application components created by plugins

  • a dependency injection creates intermediate components

  • a config source is mapped to plugin specific configuration and may be fully overridden by environment vars

  • structlog boilerplate for json/tty logging

  • fork the process and share bound sockets

  • pytest fixtures to reduce testing boilerplate

You bootstrap like following:

from buvar import plugin

plugin.stage("some.module.with.prepare")
# some.module.with.prepare
from buvar import context, plugin

class Foo:
    ...


async def task():
    asyncio.sleep(1)


async def server():
    my_component = context.get(Foo)
    await asyncio.Future()


# there is also plugin.Teardown and plugin.Cancel
async def prepare(load: plugin.Loader):
    await load('.another.plugin')

    # create some long lasting components
    my_component = context.add(Foo())

    # you may run simple tasks
    yield task()

    # you may run server tasks
    yield server()

a components and dependency injection solution

Dependency injection relies on registered adapters, which may be a function, a method, a class, a classmethod or a generic classmthod.

Dependencies are looked up in components or may be provided via kwargs.

from buvar import di

class Bar:
    pass

class Foo:
    def __init__(self, bar: Bar = None):
        self.bar = bar

    @classmethod
    async def adapt(cls, baz: str) -> Foo:
        return Foo()

async def adapt(bar: Bar) -> Foo
    foo = Foo(bar)
    return foo


async def task():
    foo = await di.nject(Foo, baz="baz")
    assert foo.bar is None

    bar = Bar()
    foo = await di.nject(Foo, bar=bar)
    assert foo.bar is bar

async def prepare():
    di.register(Foo.adapt)
    di.register(adapt)

    yield task()

a config source

buvar.config.ConfigSource is just a dict, which merges arbitrary dicts into one. It serves as the single source of truth for application variability.

You can load a section of config values into your custom attrs class instance. ConfigSource will override values by environment variables if present.

config.toml

log_level = "DEBUG"
show_warnings = "yes"

[foobar]
some = "value"
export APP_FOOBAR_SOME=thing
import attr
import toml

from buvar import config

@attr.s(auto_attribs=True)
class GeneralConfig:
    log_level: str = "INFO"
    show_warnings: bool = config.bool_var(False)


@attr.s(auto_attribs=True)
class FoobarConfig:
   some: str


source = config.ConfigSource(toml.load('config.toml'), env_prefix="APP")

general_config = source.load(GeneralConfig)
assert general_config == GeneralConfig(log_level="DEBUG", show_warnings=True)

foobar_config = source.load(FoobarConfig, 'foobar')
assert foobar_config.some == "thing"

There is a shortcut to the above approach provided by buvar.config.Config, which requires to be subclassed from it with a distinct section attribute. If one adds a buvar.config.ConfigSource component, he will receive the mapped config in one call.

from buvar import config, plugin


@attr.s(auto_attribs=True)
class GeneralConfig(config.Config):
    log_level: str = "INFO"
    show_warnings: bool = config.bool_var(False)


@attr.s(auto_attribs=True)
class FoobarConfig(config.Config, section="foobar"):
    some: str


async def prepare(load: plugin.Loader):
    # this would by typically placed in the main CLI entry point
    source = context.add(config.ConfigSource(toml.load('config.toml'), env_prefix="APP"))

    # to provide the adapter to di, which could also be done in the main entry point
    await load(config)
    foobar_config = await di.nject(FoobarConfig)

a structlog

Just structlog boilerplate.

import sys

from buvar import log

log_config = log.LogConfig(tty=sys.stdout.isatty(), level="DEBUG")
log_config.setup()

forked process and shared sockets

You may fork your process and bind and share sockets, to leverage available CPUs e.g. for serving an aiohttp microservice.

Signals like INT, TERM, HUP are forwarded to the child processes.

import aiohttp.web
from buvar import fork, plugin, di, context
from buvar_aiohttp import AioHttpConfig


async def hello(request):
    return aiohttp.web.Response(body=b"Hello, world")


async def prepare_aiohttp(load: plugin.Loader):
    await load("buvar_aiohttp")

    app = await di.nject(aiohttp.web.Application)
    app.router.add_route("GET", "/", hello)


context.add(AioHttpConfig(host="0.0.0.0", port=5678))

fork.stage(prepare_aiohttp, forks=0, sockets=["tcp://:5678"])

pytest

There are a couple of pytest fixtures provided to get your context into a reasonable state:

buvar_config_source

A dict with arbitrary application settings.

buvar_context

The basic context staging operates on.

buvar_stage

The actual stage processing all plugins.

buvar_load

The loader to add plugins to the stage.

buvar_plugin_context

The context all plugins share, when they are prepared.

Following markers may be applied to a test

buvar_plugins(plugin, ...)

Load all plugins into plugin context.

import pytest


async def prepare():
    from buvar import context

    context.add("foobar")


@pytest.mark.asyncio
@pytest.mark.buvar_plugins("tests.test_testing")
async def test_wrapped_stage_context():
    from buvar import context, plugin

    assert context.get(str) == "foobar"
    assert context.get(plugin.Cancel)


@pytest.mark.asyncio
@pytest.mark.buvar_plugins()
async def test_wrapped_stage_context_load(buvar_load):
    from buvar import context, plugin

    await buvar_load(prepare)
    assert context.get(str) == "foobar"
    assert context.get(plugin.Cancel)

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

buvar-0.43.25.tar.gz (33.6 kB view details)

Uploaded Source

Built Distributions

buvar-0.43.25-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (758.6 kB view details)

Uploaded CPython 3.12manylinux: glibc 2.28+ x86-64manylinux: glibc 2.5+ x86-64

buvar-0.43.25-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (730.7 kB view details)

Uploaded CPython 3.11manylinux: glibc 2.28+ x86-64manylinux: glibc 2.5+ x86-64

buvar-0.43.25-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl (699.8 kB view details)

Uploaded CPython 3.10manylinux: glibc 2.28+ x86-64manylinux: glibc 2.5+ x86-64

File details

Details for the file buvar-0.43.25.tar.gz.

File metadata

  • Download URL: buvar-0.43.25.tar.gz
  • Upload date:
  • Size: 33.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.12.9

File hashes

Hashes for buvar-0.43.25.tar.gz
Algorithm Hash digest
SHA256 992511be855492d387e53ed4ad8fc41adda99e5f1613fc013e81986d006c1168
MD5 8879610cf41313983ec7b120a619b157
BLAKE2b-256 a47f8645b901d6d0248ca5497fc10e8aa90a59af75909cf46b6f337303619892

See more details on using hashes here.

Provenance

The following attestation bundles were made for buvar-0.43.25.tar.gz:

Publisher: package.yaml on diefans/buvar

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file buvar-0.43.25-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.

File metadata

File hashes

Hashes for buvar-0.43.25-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
Algorithm Hash digest
SHA256 7c92894cc3832e939cdc49b8b96ed5870789e987fdde04f2bf14d8b237d6dd2f
MD5 fe4f9608a7c482fde9473554406d52cc
BLAKE2b-256 2ed472ecd62de7a092894ff90df75d929b9e5a31246526b8c6e5d063a99760a3

See more details on using hashes here.

Provenance

The following attestation bundles were made for buvar-0.43.25-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl:

Publisher: package.yaml on diefans/buvar

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file buvar-0.43.25-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.

File metadata

File hashes

Hashes for buvar-0.43.25-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
Algorithm Hash digest
SHA256 243f749cd9446fe2c7b635309efaca1ef8c675aa5d3068612f081a34001092b6
MD5 4612e334b41e1dc65dc736321f5b52ac
BLAKE2b-256 a3b6fa96db60042b5088ddee5ea018d28b835e5e9bc36cb154d702823afaa5b6

See more details on using hashes here.

Provenance

The following attestation bundles were made for buvar-0.43.25-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl:

Publisher: package.yaml on diefans/buvar

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file buvar-0.43.25-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl.

File metadata

File hashes

Hashes for buvar-0.43.25-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl
Algorithm Hash digest
SHA256 2e979bfc3435b61d017765a01f725d0114b6826547535d05d55d288edc2ddd8c
MD5 2ed8859768cc0d0bd29acb0d7c88d7a5
BLAKE2b-256 741935761de6cf244b92197f5a20e61682bc21a6e836196da92ef99fb816ba39

See more details on using hashes here.

Provenance

The following attestation bundles were made for buvar-0.43.25-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl:

Publisher: package.yaml on diefans/buvar

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page