tomodachi 0.18.0

Python 3 microservice library on asyncio with HTTP, websockets, RabbitMQ and AWS SNS+SQS.

tomodachi - a lightweight microservices library on Python asyncio

A Python 3 microservice library / framework using asyncio (async / await) with HTTP, websockets, RabbitMQ / AMQP and AWS SNS+SQS built-in support for event based messaging and intra-service communication.

Tomodachi is a tiny framework designed to build fast microservices listening on HTTP or communicating over event driven message buses like RabbitMQ, AMQP, AWS (Amazon Web Services) SNS+SQS, etc. It’s designed to be extendable to make use of any type of transport layer available.

Tomodachi [友達] means friends – a suitable name for microservices working together. 😻 👬 👭 👫 😻

Usage

tomodachi is invoked via command line interface.

Usage: tomodachi <subcommand> [options] [args]

Options:
  -h, --help             show this help message and exit
  -v, --version          print tomodachi version
  --dependency-versions  print versions of dependencies

Available subcommands:
  run <service ...> [-c <config-file ...>] [--production]
  -c, --config <files>   use json configuration files
  -l, --log <level>      specify log level
  --production           disable restart on file changes

• Installation

• Getting started / example services

• Running microservices in Docker

• Defining endpoints

  • HTTP endpoints

  • AWS SNS+SQS messaging

  • AMQP / RabbitMQ messaging

  • Scheduled functions

• Requirements

• Questions

• Contributions

Please note: this is a work in progress.

tomodachi is still a highly experimental project with an unregular release schedule.

How do I use this? (simple install using pip)

Preferrably installation should be done via pip to get the cli alias set up automatically. Locally it is recommended to install tomodachi into a virtualenv to avoid random packages into your base site-packages.

local ~$ pip install tomodachi

Getting started 🏃

Start off with import tomodachi and add a class decorated with @tomodachi.service and/or extended from the tomodachi.Service class. Name your service class and then just add functions and triggers for how to invoke them, either by HTTP requests, event messages or by timestamps / intervals.

Basic HTTP based service 🌟

Code for a simple service which would service data over HTTP.

import tomodachi


@tomodachi.service
class Service(tomodachi.Service):
    name = 'example'

    # Request paths are specified as regex for full flexibility
    @tomodachi.http('GET', r'/resource/(?P<id>[^/]+?)/?')
    async def resource(self, request, id):
        # Returning a string value normally means 200 OK
        return 'id = {}'.format(id)

    @tomodachi.http('GET', r'/health')
    async def health_check(self, request):
        # Return can also be a tuple, dict or even an aiohttp.web.Response
        # object for more complex responses - for example if you need to
        # send byte data, set your own status code or define own headers
        return {
            'body': 'Healthy',
            'status': 200
        }

    # Specify custom 404 catch-all response
    @tomodachi.http_error(status_code=404)
    async def error_404(self, request):
        return 'error 404'

RabbitMQ or AWS SNS/SQS event based messaging service 📡

Example of a service that would invoke a function when messages are published on a topic exchange.

import tomodachi

@tomodachi.service
class Service(tomodachi.Service):
    name = 'example'

    # A route / topic on which the service will subscribe to via AMQP (or AWS SNS/SQS)
    @tomodachi.amqp('example.topic')
    async def example_topic_func(self, message):
        # Received message, sending same message as response on another route / topic
        await tomodachi.amqp_publish(self, message, routing_key='example.response')

Scheduling, inter-communication between services, etc. ⚡️

There are other examples available with examples of how to use services with self-invoking methods called on a specified interval or at specific times / days. Inter-communication between different services may be established using a pub-sub type with messages over AMQP or AWS SNS+SQS which is natively supported.

See a more comprehensive example involving multiple services publishing and subcribing on topics using AWS SNS+SQS in the pubsub-examples folder.

Run the service 😎

# cli alias is set up if installed via pip
local ~/code/service$ tomodachi run service.py

# example if cloned from repo
local ~/code/tomodachi$ python tomodachi.py run example/http_simple_service.py

Defaults to output information on stdout.

local ~/code/service$ tomodachi run service.py

tomodachi/X.X.XX
October 02, 2017 - 13:38:00,481516
Quit services with <ctrl+c>.
2017-10-02 13:38:01,234 (services.service): Initializing service "example" [id: <uuid>]
2017-10-02 13:38:01,248 (transport.http): Listening [http] on http://127.0.0.1:9700/
2017-10-02 13:38:01,248 (services.service): Started service "example" [id: <uuid>]

HTTP service acts like a normal web server.

local ~$ curl -v "http://127.0.0.1:9700/resource/1234"

< HTTP/1.1 200 OK
< Content-Type: text/plain; charset=utf-8
< Server: tomodachi
< Content-Length: 9
< Date: Mon, 02 Oct 2017 13:38:02 GMT
id = 1234

Example of tomodachi service containerized in Docker 🐳

Great ways to run microservices are either to run them in Docker or running them serverless. Here’s an example of getting a tomodachi service up and running in Docker in no-time. The base-image (kalaspuff/python-nginx-proxy) also sets up nginx and proxies requests from port 80 to the service backend on 8080.

We’re building a container using just two small files, the Dockerfile and the actual code for the microservice, service.py.

Dockerfile

FROM kalaspuff/python-nginx-proxy:1.3.0
WORKDIR /
RUN apt-get -y update \
    && apt-get install -y build-essential=12.3 \
    && pip install tomodachi \
    && apt-get purge -y --auto-remove build-essential \
    && apt-get clean autoclean \
    && apt-get autoremove -y \
    && rm -rf /var/lib/{apt,dpkg,cache,log}/
RUN mkdir /app
WORKDIR /app
ADD service.py .
CMD tomodachi run service.py --production

service.py

import tomodachi

@tomodachi.service
class Service(tomodachi.Service):
    name = 'example'
    options = {
        'http': {
            'port': 8080
        }
    }

    @tomodachi.http('GET', r'/')
    async def index_endpoint(self, request):
        return 'friends forever!'

Building and running the container, forwarding host’s port 31337 to port 80.

local ~/code/service$ docker build . -t tomodachi-microservice

local ~/code/service$ docker run -ti -p 31337:80 tomodachi-microservice
2017-10-02 13:38:01,234 (services.service): Initializing service "example" [id: <uuid>]
2017-10-02 13:38:01,248 (transport.http): Listening [http] on http://127.0.0.1:8080/
2017-10-02 13:38:01,248 (services.service): Started service "example" [id: <uuid>]

Making requests to the running container.

local ~$ curl http://127.0.0.1:31337/
friends forever!

Nothing more nothing less. It’s actually as easy as that.

Available built-ins used as endpoints 🚀

There are several built-in ways to invoke your microservice methods in which the most common ones are either directly via HTTP or via event based messaging (for example AMQP or AWS SNS+SQS). Here’s a list of the currently available built-ins you may use to decorate your service functions. Here’s a short run-down of the available decorators.

HTTP endpoints:

@tomodachi.http(method, url, ignore_logging=[200])

Sets up an HTTP endpoint for the specified method (GET, PUT, POST, DELETE) on the regexp url. Optionally specify ignore_logging as a dict or tuple containing the status codes you do not wish to log the access of. Can also be set to True to ignore everything except status code 500.

@tomodachi.http_static(path, url)

Sets up an HTTP endpoint for static content available as GET / HEAD from the path on disk on the base regexp url.

@tomodachi.websocket(url)

Sets up a websocket endpoint on the regexp url. The invoked function is called upon websocket connection and should return a two value tuple containing callables for a function receiving frames (first callable) and a function called on websocket close (second callable). The passed arguments to the function beside the class object is first the websocket response connection which can be used to send frames to the client, and optionally also the request object.

@tomodachi.http_error(status_code)

A function which will be called if the HTTP request would result in a 4XX status_code. You may use this for example to set up a custom handler on “404 Not Found” or “403 Forbidden” responses.

AWS SNS+SQS messaging:

@tomodachi.aws_sns_sqs(topic, competing=None, queue_name=None, **kwargs)

This would set up an AWS SQS queue, subscribing to messages on the AWS SNS topic topic, whereafter it will start consuming messages from the queue.

The competing value is used when the same queue name should be used for several services of the same type and thus “compete” for who should consume the message.

Unless queue_name is specified an auto generated queue name will be used. Additional prefixes to both topic and queue_name can be assigned by setting the options.aws_sns_sqs.topic_prefix and options.aws_sns_sqs.queue_name_prefix dict values.

Depending on the service message_protocol used, parts of the enveloped data would be distribbuted to different keyword arguments of the decorated function. It’s usually safe to just use data as an argument. You can also specify a specific message_protocol value as a keyword argument to the decorator for specifying a specific enveloping method to use instead of the global one set for the service.

If you’re utilizing from tomodachi.protocol import ProtobufBase and using ProtobufBase as the specified service message_protocol you may also pass a keyword argument proto_class into the decorator, describing the protobuf (Protocol Buffers) generated Python class to use for decoding incoming messages.

AMQP messaging (RabbitMQ):

@tomodachi.amqp(routing_key, exchange_name='amq.topic', competing=None, queue_name=None, **kwargs)

Sets up the method to be called whenever a AMQP / RabbitMQ message is received for the specified routing_key. By default the 'amq.topic' topic exchange would be used, it may also be overridden by setting the options.amqp.exchange_name dict value for the service class.

The competing value is used when the same queue name should be used for several services of the same type and thus “compete” for who should consume the message.

Unless queue_name is specified an auto generated queue name will be used. Additional prefixes to both routing_key and queue_name can be assigned by setting the options.amqp.routing_key_prefix and options.amqp.queue_name_prefix dict values.

Depending on the service message_protocol used, parts of the enveloped data would be distribbuted to different keyword arguments of the decorated function. It’s usually safe to just use data as an argument. You can also specify a specific message_protocol value as a keyword argument to the decorator for specifying a specific enveloping method to use instead of the global one set for the service.

If you’re utilizing from tomodachi.protocol import ProtobufBase and using ProtobufBase as the specified service message_protocol you may also pass a keyword argument proto_class into the decorator, describing the protobuf (Protocol Buffers) generated Python class to use for decoding incoming messages.

Scheduled functions / cron:

@tomodachi.schedule(interval=None, timestamp=None, timezone=None, immediately=False)

A scheduled function invoked on either a specified interval (you may use the popular cron notation as a str for fine-grained interval or specify an integer value of seconds) or a specific timestamp. The timezone will default to your local time unless explicitly stated.

When using an integer interval you may also specify wether the function should be called immediately on service start or wait the full interval seconds before its first invokation.

@tomodachi.heartbeat

A function which will be invoked every second.

@tomodachi.minutely, @tomodachi.hourly, @tomodachi.daily, @tomodachi.monthly

A scheduled function which will be invoked once every minute / hour / day / month.

You may also extend the functionality by building your own transports for your endpoints. The invokers themselves should extend the class tomodachi.invoker.Invoker.

Decorated functions using @tomodachi.decorator 🎄

Invoker functions can of course be decorated using custom functionality. For ease of use you can then in turn decorate your decorator with the the built-in @tomodachi.decorator to ease development. If the decorator would return anything else than True or None (or not specifying any return statement) the invoked function will not be called and instead the returned value will be used, for example as an HTTP response.

import tomodachi

@tomodachi.decorator
async def require_csrf(instance, request):
    token = request.headers.get("X-CSRF-Token")
    if not token or token != request.cookies.get('csrftoken'):
        return {
            'body': 'Invalid CSRF token',
            'status': 403
        }

@tomodachi.service
class Service(tomodachi.Service):
    name = 'example'

    @tomodachi.http('POST', r'/create')
    @require_csrf
    async def create_data(self, request):
        # Do magic here!
        return 'OK'

Requirements 👍

• Python (3.6+, 3.7+, 3.8+)

• aiohttp

• aiobotocore

• aioamqp

• uvloop

License 🙋

Offered under the MIT license

Source code 🦄

The latest developer version of tomodachi is available at the GitHub repo https://github.com/kalaspuff/tomodachi

Any questions?

What is the best way to run a tomodachi service?

There is no way to tell you how to orchestrate your infrastructure. Some people may run it containerized in a Docker environment, deployed via Terraform / Nomad / Kubernetes and some may run several services on the same environment, on the same machine. There may be best practices but theres no way telling you how to orchestrate your application environment.

Personally I would currently go for a Dockerized environment with nginx proxy in front of the service to handle all the weirdness of the web, TLS, black magic and improved upgrades for WebSockets. Take a look at my kalaspuff/docker-python-nginx-proxy base-image to get your code up and running within minutes.

Are there any more example services?

There are a few examples in the examples folder, including using tomodachi in an example Docker environment with or without docker-compose, there are examples to publish events/messages to an AWS SNS topic and subscribe to an AWS SQS queue. There’s also a similar example of how to work with pub-sub for RabbitMQ via AMQP transport protocol.

Why should I use this?

tomodachi is a perfect place to start when experimenting with your architecture or trying out a concept for a new service. It may not have all the features you desire and it may never do, but I believe it’s great for bootstrapping microservices in async Python.

I have some great additions!

Sweet! Please send me a PR with your ideas. Get started at the short contribution guide.

Should I run this in production?

Yes? No? There are some projects that already have live versions in production. The library is provided as is with an unregular release schedule. It’s all still highly experimental and it depends on other experimental projects, so you have to be in charge here and decide for yourself. Let me know if you do however!

Another good idea is to drop in Sentry or other exception debugging solutions, for if your invoked functions would raise unhandled exceptions.

Who built this and why?

My name is Carl Oscar Aaro [@kalaspuff] and I’m a coder from Sweden. I simply wanted to learn more about asyncio and needed a constructive off-work project to experiment with – and here we are. Nowadays I use tomodachi as a base for many smaller projects where I just want to be able to focus on the application itself, while still having the power of building distributed systems. 🎉

• https://github.com/kalaspuff

• https://www.linkedin.com/in/carloscaraaro/

Contributions

Please help out to add features that you deem are missing and/or fix bugs in the repo.

To add a PR, for the repository, commit your changes to your own clone and make a PR on GitHub for your clone against master branch.

Read more in the contribution guide.

Changes

0.18.0 (2020-09-15)

• Changed the order of when to execute the service’s own _stop_service() function, to always run after active HTTP requests has finished executing, as well as awaiting ongoing AMQP before finally running the user defined function.

0.17.1 (2020-06-16)

• Updated generated proto class using protoc 3.12.2 for messages using proto envelope, which should solve some deprecation warnings.

0.17.0 (2020-06-16)

• Proper support for Python 3.8. Now correctly handles CancelledError exceptions that previously sent a lot of unwanted output on service shutdown or restart.

• Updated dependencies across the board, utilizing package versions that supports Python 3.8.

• Dropped support for Python 3.5.

• Now gracefully handles shutdown for HTTP based services, by awaiting active requests and giving them time to finish. By default the ongoing HTTP requests will have 30 seconds to complete their work, which can also be configured via options.http.termination_grace_period_seconds.

• Taking steps into making the codebase following more modern patterns. Additional updates to be followed in a later release.

0.16.6 (2020-02-25)

• Removes the dependency on ujson.

0.16.5 (2020-02-12)

• Bugfix for context reference mismatch when using custom invocation decorators which could cause the provided context variable to not include the correct information.

0.16.4 (2019-08-28)

• Fix for the the race condition causing delete_message to raise an exception, when draining the SQS receive messages call, while stopping the service.

0.16.3 (2019-08-23)

• It’s now possible to get the request object for websocket handlers by adding a third argument to the invoker function. (self, websocket, request) or by specifying request as a keyword argument in the function signature. Using the request object it’s now possible to parse browser headers and other data sent when first opening the websocket connction.

• Updated packages for automated tests to verify that newer dependencies still works correctly.

• Updated the dependency on aioamqp to allow aioamqp==0.13.x.

0.16.2 (2019-03-27)

• Added keyword arguments for overriding the topic_prefix and routing_key_prefix when publishing messages. Useful by for example intermediaries that needs to publishing messages to external services running on other environments, or services that are otherwise confined to a prefix / environment but needs to contact a core service, i.e. data collection, emails, etc.

0.16.1 (2019-03-21)

• Bug fix for websocket handler functions signature inspection in middlewares, which caused the function signature to return a non-wrapped internal function.

0.16.0 (2019-03-07)

• Refactored all internal middleware functionality to use the same base function for executing middlewares.

• A middleware context will be passed into the middlewares as the optional fifth argument, a dict that will live within the middleware excecution and may pass data along from middleware to middleware.

0.15.1 (2019-03-07)

• Middlewares first argument func: Callable will now be wrapped with the endpoint function, using @functools.wraps, so that signatures and keywords may be inspected and applied accordingly. (github: 0x1EE7)

0.15.0 (2019-02-27)

• message_middleware will now receive four arguments instead of the earlier three. func: Callable, service: Any, message: Any, topic: str for SNS and func: Callable, service: Any, message: Any, route_key: str for AMQP. If you are using middlewares for messaging you will most likely need to update these.

• Additional kwargs may be passed into the aws_sns_sqs_publish and amqp_publish functions and will be forwarded to the message_protocol build_message function.

0.14.8 (2019-01-28)

• Fixes an issue when websockets were initiated together with an HTTP middleware applying additional arguments and keywords.

• Sets the request._cache['is_websocket'] value before handing the processing off to the middleware.

• Fixes a bug causing aiohttp.web.FileResponse return values to not show any content.

0.14.7 (2019-01-21)

• Added helper functions to be able to get the status code of a HTTP response or on a raised exception during a HTTP request. await tomodachi.get_http_response_status(value, request=request) or await tomodachi.get_http_response_status(exception, request=request)

0.14.6 (2019-01-14)

• Extended middleware functionality to also be available for event based messaging (AMQP and SNS+SQS) as message_middleware.

0.14.5 (2019-01-09)

• Added the support of middlewares to inject additional arguments and keywords arguments or overriding existing keyword arguments of the invoked function.

0.14.4 (2019-01-06)

• Service classes may now use http_middleware which is a list of functions to be run on all HTTP calls and may change the behaviour before or after the invoked function is called, either preventing the function from being called or modifying the response values. An example has been added to the examples directory.

• The auto-reloader on code changes will now only reload if a the files content has actually changed and not when the file was written to disk without changes.

0.14.3 (2018-12-26)

• Added support for aiohttp 3.5.x.

0.14.2 (2018-12-19)

• Solves an issue which caused SNS / SQS invoked functions to never resume the ReceiveMessage API calls on connection failure, resulting in log output saying “Session closed” and requiring the service to be restarted.

• Added support for aiobotocore 0.10.x.

0.14.1 (2018-12-04)

• Fixes an issue which caused scheduled functions to spam output on computer sleep when developing locally.

0.14.0 (2018-12-04)

• Added the possibility of specifying message_protocol for AMQP / SNS+SQS enveloping per function, so that it’s possible to use both (for example) raw data and enveloped data within the same function without having to build fallback enveloping functionality.

• Added documentation for @tomodachi.decorator, describing how to easily write decorators to use with service invoker functions.

• Added ignore_logging keyword argument to HTTP invoker decorator, which may ignore access logging for either specific status codes or everything (except 500 statuses). (github: justcallmelarry)

• New function tomodachi.get_service() or tomodachi.get_service(service_name) available to get the service instance object from wherever in the running service, much like asyncio.get_event_loop().

• Updated examples.

• Fixes issue which caused aiohttp FileResponse responses to raise an internal exception.

• Added support for aiohttp 3.4.x.

0.13.7 (2018-08-10)

• Correction for non-defined exception in Python 3.5.

0.13.6 (2018-08-10)

• Improved error handling if strict tomodachi dependencies fail to load, for example if an installed dependency is corrupt or missing.

• Added additional examples to repository with a demo of pub-sub communication.

0.13.5 (2018-08-08)

• Fixes an issue which caused HTTP invoker functions to be accessible before the bootstrapper function _start_service() had been completed. Now _start_service() is called first, followed by activation of the invoker functions (@http, @schedule, @aws_sns_sqs, @amqp, etc.) and then lastly the _started_service() function will be called, announcing that the service is now up and running.

0.13.4 (2018-08-06)

• Added type hinting stubs for ProtoBuf _pb2.py file to allow mypy to validate functions utilizing the generated protobuf files.

0.13.3 (2018-08-03)

• RST correction from last release.

0.13.2 (2018-08-03)

• Correction regarding type hinting as to where a bytes value could be used as the HTTP body in Response objects.

0.13.1 (2018-08-01)

• Fixes bug with type hinting reporting ‘error: Module has no attribute “decorator”’ when applying a @tomodachi.decorator decorator.

0.13.0 (2018-07-25)

• Restructured base message protocols for both JSON and ProtoBuf. JSON protocol is now called tomodachi-json-base--1.0.0 (earlier json_base-wip) and the ProtoBuf protocol is now referred to as tomodachi-protobuf-base--1.0.0. Updated proto files are not compatible with earlier protocol protobuf_base-wip.

0.12.7 (2018-07-04)

• Fixed an issue for using ProtoBuf in development as hot-reloading didn’t work as expected. (github: smaaland)

0.12.6 (2018-07-02)

• Additional compatibility for Python 3.7 support including CI testing for Python 3.7.

• Improved linting for type hinted functions.

0.12.5 (2018-06-27)

• Messages via SNS+SQS or AMQP over 60000 bytes as ProtoBuf will now be sent in a gzipped base64 encoded format to allow for larger limits and lower potential SNS costs due to multiplexed messaging. (github: smaaland)

0.12.4 (2018-06-24)

• Updated aioamqp to the latest version with support for Python 3.7.

• Updated service imports for improved Python 3.7 compatibility.

0.12.3 (2018-06-12)

• Improved type hinting support.

0.12.2 (2018-06-12)

• Added stubs for type hinting via tools like mypy.

0.12.1 (2018-06-07)

• Added complete support for aiohttp 3.3.x release and aiobotocore 0.9.x releases.

0.12.0 (2018-05-31)

• Improved handling of imports to allow relative imports in services and to use better error messages if the parent package is using a reserved name.

• Preparations for aiohttp 3.3.x release which deprecates some uses for custom router.

• Preparations for upcoming Python 3.7 release.

0.11.3 (2018-05-25)

• Added additional function for message validation functionality. (github: smaaland)

• Updated documentation and examples.

0.11.2 (2018-05-19)

• Improved base documentation.

• Improved and updated examples.

• Type hinting corrections for examples.

0.11.1 (2018-05-18)

• Decorators for invoker functions already decorated with for example @tomodachi.http or @tomodachi.aws_sns_sqs is now easier to implement using the @tomodachi.decorator decorator.

• Added improved exception logging from HTTP and schedule invokers also to the AWS SNS+SQS and AMQP endpoints. Unhandled exceptions are now logged as logging.exception() to the 'exception' logger.

0.11.0 (2018-05-15)

• Propagation of exceptions in invoked functions to be able to hook in exception handlers into logging. (github: 0x1EE7)

0.10.2 (2018-05-15)

• Encoding issue for Protocol Buffers messages solved. (github: smaaland).

• Support for aiobotocore 0.8.X+.

0.10.1 (2018-04-26)

• Fixes a bug for optional dependency protobuf. message_protocol imports would break unless the google.protobuf package was installed.

0.10.0 (2018-04-20)

• Base example message protocol class for using Protocol Buffers over AMQP or AWS SNS+SQS. (github: smaaland).

• Validation of event based messages via validation function specified during decoration. (github: smaaland)

• Updates to work with aiohttp 3.1.X+.

• Improved logging functionality.

• Better type hinting and linting.

0.9.5 (2018-03-16)

• More robust handling of invoking service files that aren’t a part of a Python package.

0.9.4 (2018-03-06)

• Fixes an issue affecting websocket connections where the receive function was invalidly called twice of which one time were without error handling.

0.9.3 (2018-03-06)

• Solves an error with functions for AMQP / AWS SNS+SQS functions that are used without a message_protocol class.

• Improved disconnect and reconnect to AWS SNS+SQS via aiobotocore on hot-reload and during testing.

• Improved README with event based messaging example using AMQP.

• Added the option of running schedule tasks immediately on service start. For example a function decorated by @schedule(interval=20, immediately=True) would be run immediately on service start and then every 20 seconds.

0.9.2 (2018-03-05)

• Improved error handling for bad requests (error 400) on HTTP calls.

• File watcher for hot-reload now excludes ignored directories in a more effective way to ease CPU load and for faster boot time for projects with thousands of files which should’ve been ignored.

0.9.1 (2018-03-05)

• schedule functions limits to 20 running tasks of the same function to prevent overflows in development.

• Fixes an issue where schedule tasks stopped executing if a service was hot-reloaded on code change.

• Handles websocket cancellations better if the client would close the connection before the request had been upgraded.

0.9.0 (2018-03-04)

• Updated to use aiohttp 3.X.X+ and aiobotocore 0.6.X+.

• Dropped support for Python versions below 3.5.3 as new aiohttp requires at least Python 3.5.3. Last version with support for Python 3.5.0, 3.5.1 and 3.5.2 is tomodachi 0.8.X series.

0.8.3 (2018-03-02)

• Print stack trace for outputs from schedule invoker functions tasks instead of silently catching exceptions.

• Handle close and receive errors for websockets and cleanly close already opened websockets on service exit.

0.8.2 (2018-02-28)

• Fixed broken HTTP transports due to missing colorama import.

0.8.1 (2018-02-27)

• Correction for README in 0.8.X release.

0.8.0 (2018-02-27)

• It’s now possible to specify queue_name on AWS SNS+SQS and AMQP decorators for competing queues. If not specified an automatically generated hash will be used as queue name as it worked previously.

• Fixes an issue with relative imports from within service files, which resulted in “SystemParent module ‘’ not loaded, cannot perform relative import” or “ImportError: attempted relative import with no known parent package”. (github: 0x1EE7)

• Exceptions that are subclasses of AmqpInternalServiceError and AWSSNSSQSInternalServiceError will now also work in the same way, resulting in the messages to be retried when raised.

• Service classes now have built in log functions for setting up logging to file as well as logging. They are self.log_setup('logname', level, filename) and self.log('logname', level, message).

• HTTP services will have their access log color coded when outputting to nothing else than stdout, which should be helpful in an overview during development.

0.7.0 (2018-01-27)

• Added @websocket as a decorator type for handling websockets. A function call should return two callables which will be used for receiving messages through the socket and as a way to notify about the closure of the socket.

0.6.5 (2018-01-16)

• Updated aiohttp to latest version which solves incompabilities with yarl.

0.6.4 (2018-01-15)

• Added a stricter dependency check for yarl.

0.6.3 (2018-01-12)

• Gracefully handle exceptions thrown when receiving messages from AWS SNS+SQS. For example when invalid XML data in response which causes botocore to throw a botocore.parsers.ResponseParserError.

• Updated dependencies to allow for newer version of aiohttp 2.3.X.

• Improved type hinting.

0.6.2 (2017-11-15)

• Recreate queues and resubscribe to topics if queue is removed during runtime.

0.6.1 (2017-11-15)

• Introduced new options for AWS SNS/SQS transport to use aws_endpoint_urls for sns and sqs if the user wishes to connect to other endpoints and the actual AWS endpoints, which could be useful for development and testing. The AWS SNS/SQS examples has been updated with values to reflect these options.

• Reworked timeouts and reconnects and fixed an issue in the recreate_client method which was called on server disconnects.

0.6.0 (2017-11-15)

• Stricter version control of required packages to not break installation on major/minor related updates.

• Updates to support aiohttp 2.3.X and aiobotocore 0.5.X.

0.5.3 (2017-11-08)

• Corrects issues on timeouts and server disconnects.

• Specify fixed version for aiohttp to not break installation.

• Code cleanup to conform with pycodestyle.

0.5.2 (2017-10-08)

• Add argument option for log level as ‘-l’ or ‘–log’. (github: djKooks)

• Better matching of imported modules on hot-reload which will cause reloading into code with syntax errors or indentation errors much harder.

0.5.1 (2017-10-03)

• More improvements regarding hot-reloading of code that may have syntax errors, indentation errors or issues when the service is being initiated.

0.5.0 (2017-10-02)

• Solves the issue where hot-loading into a state where the code errors due to syntax errors would crash the application, making the user need to manually restart the process.

0.4.10 (2017-10-02)

• Fixes for failing tests on hot-reloading during test phase.

0.4.9 (2017-10-02)

• Solves issue with Segmentation fault in Python 3.6 during hot-reload on Linux.

0.4.8 (2017-10-02)

• Fixes type hinting issues with Python 3.5.1.

0.4.7 (2017-09-30)

• Reworked watcher since it ended up using 90% CPU of the running core due to constant re-indexing (mstat) of every file every 0.5s. Full re-index will now only run every 10 seconds, since it’s more rare that new files are added than existing files edited. Watcher for edited existing files will still run at the same intervals.

• Watched file types may now be specified via configuration via options.watcher.watched_file_endings.

0.4.6 (2017-09-29)

• Messages via SNS+SQS or AMQP over 60000 bytes as JSON will now be sent in a gzipped base64 encoded format to allow for larger limits and lower potential SNS costs due to multiplexed messaging.

• Fixes an issue with multidict 3.2.0 on hot-reload which made the tomodachi application crash.

0.4.5 (2017-09-07)

• Possibility to requeue messages that result in specific exceptions. Exceptions that will nack the message (for AMQP transport) is called AmqpInternalServiceError. Exceptions that won’t delete the message from the queue and in turn will result in it to “reappear” unless configured non-default (for AWS SNS+SQS transport) is called AWSSNSSQSInternalServiceError.

0.4.4 (2017-08-25)

• Corrected an issue regarding crontab notation for scheduling function calls where it didn’t parse the upcoming date correctly if both isoweekday and day part were given.

0.4.3 (2017-08-09)

• Catches unintended HTTP exceptions and prints a useful stacktrace if log_level is set to DEBUG.

0.4.2 (2017-08-07)

• Fixes an issue where Content-Type header couldn’t be specified without charset in HTTP transports.

• Cleared some old debug code.

0.4.1 (2017-08-05)

• Corrects and issue with AMQP transport which caused invoked functions to not be able to declare scope variables without crashes.

0.4.0 (2017-08-05)

• Release fixes a major issue which caused invoked functions to not be able to declare any scope variables.

• @http_static decorator for serving static files from a folder on disk. Takes to values; 1. the path to the folder, either relative to the service file or absolute; 2. the base URL path for static files as a regexp.

0.3.0 (2017-07-25)

• Changed format of access log for HTTP requests - now logging user agent and login name (if authorization via Basic Auth).

• Support for X-Forwarded-For headers via real_ip_from and real_ip_header options which will log the forwarded IP instead of the one from the load balancer / proxy.

• Access log for HTTP can now be specified as a filename to which the service will log all requests.

• Fixes issue with schedule invoker which would crash if invoked at second 0.

• Updated dependencies to latest available versions.

0.2.17 (2017-07-05)

• Timezone support for schedule invoker functions.

• Added more decorator invoker functions as aliases for common scheduler use cases - @minutely, @hourly, @daily and @heartbeat (every second)

• Updated example services and better test cases.

• Updated aiohttp / aiobotocore / botocore dependencies.

0.2.16 (2017-07-02)

• Solved issues with aiobotocore / aiohttp dependencies.

• Refactored loader functions.

0.2.15 (2017-07-02)

• Corrected issue with configuration values for AWS and AWS SNS+SQS settings.

• Improved testing suite and more code coverage for integration tests.

0.2.14 (2017-06-30)

• New “transport” invoker for service functions: schedule. It works like cron type scheduling where specific functions will be run on the specified interval. For example a function can be specified to run once per day at a specific time or every second minute, or the last Tuesday of January and March at 05:30 AM.

• Values for keyword arguments invoked by transport decorators were earlier always set to None, despite having other default values. This is now corrected.

0.2.13 (2017-06-20)

• Type hinted examples and test cases.

• Shielded function calls for AMQP and SNS+SQS transports to avoid unexpected execution stop.

• Added version output to tomodachi CLI tool.

• Additional test cases.

0.2.12 (2017-06-18)

• Type hinted code base and minor bug fixes for internal functions.

0.2.11 (2017-06-09)

• Invoker methods can now be called directly without the need to mock the invoker decorator function.

0.2.10 (2017-06-08)

• Added @functools.wraps decorator to invoked functions of service classes.

0.2.9 (2017-06-06)

• Added a list of safe modules that may never be removed from the list of already loaded modules. Removing the module ‘typing’ from the list would cause a RecursionError exception since Python 3.6.1.

0.2.8 (2017-05-23)

• Additional improvements to network connectivity issues to not get stuck in waiting state.

0.2.7 (2017-05-23)

• Improved SNS+SQS draining / restart when network connectivity has been lost or temporarily suspended. Would improve situations when development machine has been in hibernation.

• Replaced deprecated logging functions to rid warnings.

0.2.6 (2017-05-22)

• Support for a “generic” aws dictonary in options that can hold region, access key id and secret to be shared among other AWS resources/services.

• Updated aiobotocore / botocore dependencies.

• Gracefully handle and discard invalid SNS/SQS messages not in JSON format.

• Corrected issue where watched directories with “similar” names as settings would be ignored.

0.2.5 (2017-05-16)

• Updated issues with function caching due to keepalive when hot reloading in development. Currently disables keepalive entirely.

• Fixed issue with updated file logging for watcher.

0.2.4 (2017-05-12)

• Downgraded botocore to meet requirements and to make the installed tomodachi script runnable again.

0.2.3 (2017-05-10)

• Watcher is now configurable to ignore specific directories dependant on the service. (github: smaaland)

• Fixed issue where using --config instead of -c would result in a raised exception. (github: smaaland)

0.2.2 (2017-05-04)

• tomodachi.transport.http has its own Response object that works better with default content types and charsets - examples/http_service.py updated.

• No automatic conversion will be tried if the returned response of an http method is of bytes type.

0.2.1 (2017-05-03)

• Improved handling of how charsets and encodings work with aiohttp.

• Fixed an issue where Content-Type header would always be included twice for aiohttp.web.Response objects.

0.2.0 (2017-05-02)

• Watcher now only reacts to files with file endings .py, .json, .yml, .html or .html and ignores to look at paths __pycache__, .git, .svn, __ignored__, __temporary__ and __tmp__.

• HTTP transport may now respond with an aiohttp.web.Response object for more complex responses.

• HTTP transport response headers can now use the multidict library.

0.1.11 (2017-04-02)

• Working PyPI release.

• Added unit tests.

• Works with aiohttp 2 and aiobotocore 0.3.

• Service classes must be decorated with @tomodachi.service.