aio.app 0.1.0

Aio application runner

Detailed documentation
**********************

aio.app
=======

Application runner for the aio_ asyncio framework

.. _aio: https://github.com/phlax/aio


Build status
------------

.. image:: https://travis-ci.org/phlax/aio.app.svg?branch=master
:target: https://travis-ci.org/phlax/aio.app


Installation
------------

Requires python >= 3.4

Install with:

.. code:: bash

pip install aio.app


Quick start - hello world scheduler
-----------------------------------

Save the following into a file "hello.conf"

.. code:: ini

[schedule/EXAMPLE]
every = 2
func = my_example.schedule_handler

And save the following into a file named "my_example.py"

.. code:: python

import asyncio

def schedule_handler(name):
yield from asyncio.sleep(1)
print ("Received scheduled: %s" % name)

Run with the aio run command

.. code:: bash

aio run -c hello.conf



The *aio config* command
------------------------

To dump the system configuration you can run

.. code:: bash

aio config

To dump a configuration section you can use -g or --get with the section name

.. code:: bash

aio config -g aio

aio config --get aio/commands

To get a configuration option, you can use -g with the section name and option

.. code:: bash

aio config -g aio:log_level

aio config --get listen/example:example-signal

You can set a configuration option with -s or --set

Multi-line options should be enclosed in " and separated with "\\n"

Options containing interpolation should be enclosed in single quotes

.. code:: bash

aio config --set aio:log_level DEBUG

aio config -s aio/otherapp:log_level '${aio:log_level}'

aio config -s listen/example:example-signal "my.listener\nmy.listener2"

When saving or reading configuration options, configuration files are searched for in order from the following locations

- aio.conf
- etc/aio.conf
- /etc/aio/aio.conf

If none are present aio will attempt to save in "aio.conf" in the current working directory

To get or set an option in a particular file you can use the -f flag

.. code:: bash

aio config -g aio:modules -f custom.conf

aio config -s aio:log_level DEBUG -f custom.conf

When getting config values with the -f flag, ExtendedInterpolation_ is not used, and you therefore see the raw values



the *aio run* command
---------------------

You can run an aio app as follows:

.. code:: bash

aio run

Or with a custom configuration file

.. code:: bash

aio -c custom.conf run


On startup aio run sets up the following

- Configuration - system-wide configuration
- Modules - initialization and configuration of modules
- Logging - system logging policies
- Schedulers - functions called at set times
- Servers - listening on tcp/udp or other type of socket
- Signals - functions called in response to events


Configuration
~~~~~~~~~~~~~

Configuration is in ini syntax

.. code:: ini

[aio]
modules = aio.app
aio.signals

While the app is running the system configuration is importable from aio.app

.. code:: python

from aio.app import config

Configuration is parsed using ExtendedInterpolation_ as follows

- aio.app defaults read
- user configuration read to initialize modules
- "aio.conf" read from initialized modules where present
- user configuration read again to ensure for precedence


Logging
~~~~~~~

Logging policies can be placed in the configuration file, following pythons fileConfig_ format

.. _fileConfig: https://docs.python.org/3/library/logging.config.html#logging-config-fileformat

As the configuration is parsed with ExtendedInterpolation_ you can use options from other sections

.. code:: ini

[logger_root]
level=${aio:log_level}
handlers=consoleHandler
qualname=aio

The default aio:log_level is INFO

Any sections that begin with handler_, logger_, or formatter_ will automattically be added to the relevant logging section

So by adding a section such as

.. code:: ini

[logger_custom]
level=${aio:log_level}
handlers=consoleHandler
qualname=custom

"logger_custom" will automatically be added to the logger keys:

.. code:: ini

[loggers]
keys=root,custom


Modules
~~~~~~~

You can list any modules that should be imported at runtime in the configuration

Configuration for each module is read from a file named aio.conf in the module's path, if it exists.

The system modules can be accessed from aio.app

.. code:: python

from aio.app import modules


Schedulers
~~~~~~~~~~

Any sections in the configuration that start with "schedule/" will create a scheduler.

Specify the frequency and the function to call. The function will be wrapped in a coroutine if it isnt one already

.. code:: ini

[schedule/example]
every = 2
func = my.scheduler.example_scheduler

The scheduler function takes 1 argument the name of the scheduler

.. code:: python

def example_scheduler(name):
yield from asyncio.sleep(2)
# do something
pass

Servers
~~~~~~~

Any sections in the configuration that start with "server/" will create a server

The server requires either a factory or a protocol to start

Protocol configuration example:

.. code:: ini

[server/example]
protocol = my.example.ServerProtocol
port = 8888

Protocol example code:

.. code:: python

class ServerProtocol(asyncio.Protocol):

def connection_made(self, transport):
self.transport = transport

def data_received(self, data):
# do stuff
self.transport.close()

If you need further control over how the protocol is created and attached you can specify a factory method

Factory configuration example:

.. code:: ini

[server/example]
factory = my.example.server_factory
port = 8080

Factory code example:

.. code:: python

@asyncio.coroutine
def server_factory(name, protocol, address, port):
loop = asyncio.get_event_loop()
return (
yield from loop.create_server(
ServerProtocol, address, port))


Signals
~~~~~~~

Any section in the configuration that starts with "listen/" will subscribe listed functions to given events

An example listen configuration section

.. code:: ini

[listen/example]
example-signal = my.example.listener

And an example listener function. The listener function will be called as a coroutine

.. code:: python

def listener(signal, message):
yield from asyncio.sleep(2)
print(message)

Signals are emitted in a coroutine

.. code:: python

yield from app.signals.emit(
'example-signal', "BOOM!")

You can add multiple subscriptions within the section

.. code:: ini

[listen/example]
example-signal = my.example.listener
example-signal-2 = my.example.listener2

You can also subscribe multiple functions to a signal

.. code:: ini

[listen/example]
example-signal = my.example.listener
my.example.listener2


And you can have multiple "listen/" sections

.. code:: ini

[listen/example]
example-signal = my.example.listener
my.example.listener2

[listen/example2]
example-signal2 = my.example.listener2

The *aio test* command
----------------------

You can test the modules set in the aio:modules configuration option

.. code:: ini

[aio]
modules = aio.config
aio.core
aio.signals

By default the aio test command will test all of your test modules

.. code:: bash

aio test

You can also specify a module

.. code:: bash

aio test aio.app

If you want to specify a different set of modules for testing than for your app environment, you can set them in aio/testing:modules

.. code:: ini

[aio/testing]
modules = aio.config
aio.core



Dependencies
------------

aio.app depends on the following packages

- aio.core_
- aio.signals_
- aio.config_


Related software
----------------

- aio.testing_
- aio.http.server_
- aio.web.server_

.. _aio.testing: https://github.com/phlax/aio.testing
.. _aio.core: https://github.com/phlax/aio.core
.. _aio.signals: https://github.com/phlax/aio.signals
.. _aio.config: https://github.com/phlax/aio.config

.. _aio.http.server: https://github.com/phlax/aio.http.server
.. _aio.web.server: https://github.com/phlax/aio.web.server

.. _ExtendedInterpolation: https://docs.python.org/3/library/configparser.html#interpolation-of-values





aio.app usage
-------------

The aio command can be run with any commands listed in the [aio/commands] section of its configuration

There are also 3 builtin commands - run, config and test

Initially aio.app does not have any config, signals, modules or servers

>>> import aio.app

>>> print(aio.app.signals, aio.app.config, aio.app.modules, aio.app.servers)
None None () {}


Lets start the app runner in a test loop with the default configuration and print out the signals and config objects

>>> import aio.testing
>>> from aio.app.runner import runner

>>> @aio.testing.run_until_complete
... def run_app():
... yield from runner(['run'])
...
... print(aio.app.signals)
... print(aio.app.config)
... print(aio.app.modules)
... print(aio.app.servers)


>>> run_app()
<aio.signals.Signals object ...>
<configparser.ConfigParser ...>
(<module 'aio.app' from ...>,)
{}


Clear the app
-------------

We can clear the app vars.

This will also close any socket servers that are currently running

>>> aio.app.clear()

>>> print(aio.app.signals, aio.app.config, aio.app.modules, aio.app.servers)
None None () {}


Adding a signal listener
------------------------

We can add a signal listener in the app config

>>> config = """
... [listen/testlistener]
... test-signal = aio.app.tests._example_listener
... """

Lets create a test listener and make it importable

The listener needs to be a coroutine

>>> import asyncio

>>> @asyncio.coroutine
... def listener(signal, message):
... print("Listener received: %s" % message)

>>> aio.app.tests._example_listener = listener

Running the test...

>>> @aio.testing.run_until_complete
... def run_app(message):
... yield from runner(['run'], config_string=config)
... yield from aio.app.signals.emit('test-signal', message)
... aio.app.clear()

>>> run_app('BOOM!')
Listener received: BOOM!


We can also add listeners programatically

>>> @aio.testing.run_until_complete
... def run_app(message):
... yield from runner(['run'])
...
... aio.app.signals.listen('test-signal-2', asyncio.coroutine(listener))
... yield from aio.app.signals.emit('test-signal-2', message)
... aio.app.clear()

>>> run_app('BOOM AGAIN!')
Listener received: BOOM AGAIN!


Adding app modules
------------------

When you run the app with the default configuration, the only module listed is aio.app

>>> @aio.testing.run_until_complete
... def run_app(config_string=None):
... yield from runner(['run'], config_string=config_string)
... print(aio.app.modules)
... aio.app.clear()

>>> run_app()
(<module 'aio.app' from ...>,)

We can make the app runner aware of any modules that we want to include, these are imported at runtime

>>> config = """
... [aio]
... modules = aio.app
... aio.core
... """

>>> run_app(config_string=config)
(<module 'aio.app' from ...>, <module 'aio.core' from ...>)


Running a scheduler
-------------------

A basic configuration for a scheduler

>>> config = """
... [schedule/test-scheduler]
... every: 2
... func: aio.app.tests._example_scheduler
... """

Lets create a scheduler function and make it importable.

The scheduler function should be a coroutine

>>> @asyncio.coroutine
... def scheduler(name):
... print('HIT: %s' % name)

>>> aio.app.tests._example_scheduler = scheduler

We need to use a aio.testing.run_forever to wait for the scheduled events to occur

>>> @aio.testing.run_forever(timeout=5)
... def run_app():
... yield from runner(['run'], config_string=config)
...
... return aio.app.clear

Running the test for 5 seconds we get 3 hits

>>> run_app()
HIT: test-scheduler
HIT: test-scheduler
HIT: test-scheduler


Running a server
----------------

Lets set up and run an addition server

At a minimum we should provide a protocol and a port to listen on

>>> config_server_protocol = """
... [server/additiontest]
... protocol: aio.app.tests._example_AdditionServerProtocol
... port: 8888
... """

Lets create the server protocol and make it importable

>>> class AdditionServerProtocol(asyncio.Protocol):
...
... def connection_made(self, transport):
... self.transport = transport
...
... def data_received(self, data):
... nums = [
... int(x.strip())
... for x in
... data.decode("utf-8").split("+")]
... self.transport.write(str(sum(nums)).encode())
... self.transport.close()

>>> aio.app.tests._example_AdditionServerProtocol = AdditionServerProtocol

After the server is set up, let's call it with a simple addition

>>> @aio.testing.run_forever
... def run_addition_server(config_string, addition):
... yield from runner(['run'], config_string=config_string)
...
... def call_addition_server():
... reader, writer = yield from asyncio.open_connection(
... '127.0.0.1', 8888)
... writer.write(addition.encode())
... yield from writer.drain()
... result = yield from reader.read()
... aio.app.clear()
...
... print(int(result))
...
... return call_addition_server

>>> run_addition_server(
... config_server_protocol,
... '2 + 2 + 3')
7

If you need more control over how the server protocol is created you can specify a factory instead

>>> config_server_factory = """
... [server/additiontest]
... factory = aio.app.tests._example_addition_server_factory
... port: 8888
... """

The factory method must be a coroutine

>>> @asyncio.coroutine
... def addition_server_factory(name, protocol, address, port):
... loop = asyncio.get_event_loop()
... return (
... yield from loop.create_server(
... AdditionServerProtocol,
... address, port))

>>> aio.app.tests._example_addition_server_factory = addition_server_factory

>>> run_addition_server(
... config_server_protocol,
... '17 + 5 + 1')
23