A simple plugin system for async applications
aiohook – simple plugins for asyncio, trio and curio
aiohooks is a small library meant to help implementing a plugin system in Python 3.7+ asynchronous applications developed with an anyio-compatible library. It can also be used to set up synchronous function hooks.
- Declare the signature of the hook coroutine by decorating a dummy
async def(without implementation, or with a default one) with
awaitthe dummy coroutine where you would want the plugin one to be called.
- Implement the dummy coros in a separate module or class and decorate them with
aiohook.register(reference)to register the decorated implementations in
reference(object instance or module).
- Non-implemented hooks result in the default implementation being called.
- Registering multiple implementations of the same hook raises an exception.
In this story, your name is Wallace, you are implementing an awesome async application accepting plugins. Your user, Gromit, wants to make use of your plugin system to inject his custom parsing logic.
You must first define the signature of each hook function, for instance in a separate pluginspecs.py file:
from typing import AsyncIterator, Optional import aiohook @aiohook.spec async def tokenize(sentence: str) -> AsyncIterator[str]: # Describe the purpose of the hook, arguments etc in the docstring """Split string of characters into individual tokens.""" # workaround for python's type system inability to declare a generator # function without a body with a 'yield' somewhere yield @aiohook.spec async def transform_token(word: str) -> Optional[str]: """Preprocess each raw token, for instance as a normalization step."""
In your application code, you then call your spec'ed out functions as if they had already been implemented:
import sys, asyncio, importlib, aiohook from pluginspecs import tokenize, transform_token async def main(): with open(sys.argv, 'r') as f: source = f.read() async for token in tokenize(source): transformed = await transform_token(token) print(transformed, end='') if __name__ == '__main__': aiohook.register(importlib.import_module(sys.argv)) asyncio.run(main())
Gromit wants to use Wallace's application to transform text files. He creates a pip-installable module in which the gromify.py file looks as follows:
import asyncio, aiohook, random from typing import AsyncIterator, Optional @aiohook.impl('pluginspecs.tokenize') async def give_word(text: str) -> AsyncIterator[str]: for w in text.split(): yield w @aiohook.impl('pluginspecs.transform_token') async def bark_and_pause(word: str) -> Optional[str]: await asyncio.sleep(random.uniform(0, 2)) return 'woo' + 'o'*(max(len(word) - 2, 0)) + 'f'
Gromit pip-installs his plugin and Wallace's text processing app, and then calls the latter, providing as first argument the absolute import path to the gromify module.
Note that I haven't actually tried the above example, and also that it makes little sense to use async functions in this case, but I think it illustrates the basic usage os aiohook nicely.
For less silly examples, please refer to the sample applications used as functional tests in tests/functional
Two requirements files are used to describe the development setup:
- The requirements.txt file describes a working development environment with all pinned dependencies.
- The requirements-base.txt file contains the direct unpinned dependencies only necessary for a full development environment.
Step-by-step guide to set up a development environment
This assumes that pyenv and pyenv-virtualenv are installed, and the OS is somehow Unix-like. Commands are executed from the root of the repo.
pyenv install 3.7.4 pyenv virtualenv 3.7.4 aiohook-dev-3.7.4 pyenv activate aiohook-dev-3.7.4
To work on the main library, it is best to just pip install it as an editable package, and install the rest of the dev requirements:
pip install -e . pip install -r requirements.txt
To work on one of the functional test applications under [tests/functional(tests/functional), it is useful to also install it in editable mode in the same environment:
pip install -e tests/functional/text_processing_app # Try it out process_text textproc_plugin requirements.txt requirements-transformed.txt
Testing with coverage
pytest --cov=aiohook tests/
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.