Skip to main content

Broadcast and receive log messages over network.

Project description


A suite with a variety of tools for handling log messages.

The name *logsweet* is a word play by combining *sweet logging* and a
*suite of logging tools*.


- Python ≥3.4


Python Package ``logsweet``


pip install logsweet



python ./ install

Docker Image ``mastersign/logsweet``

There is a public Docker image for *logsweet* available.

.. code:: sh

docker run --rm -it mastersign/logsweet [command] [OPTIONS] [ARGUMENTS]

Remember to mount log files as volumes into the container, when using
the ``watch`` or ``mock`` command.


Logsweet supports the following commands:

- ``watch`` watches for new lines in text files (e.g. log files) and
broadcasts them via a ∅MQ PUB socket and/or sends them via a ∅MQ PUSH
- ``listen`` listens to new lines broadcasted via ∅MQ PUB socket and/or
collects new lines from ∅MQ PUSH sockets and prints them on the
- ``proxy`` can connect to watchers at the backend and to listeners at
the frontend. It supports both transmission modes (PUB/SUB and
PUSH/PULL) at both backend and frontend. It can be used as a single
point of knowledge or to build groups of log streams.
- ``mock`` to write random log messages to text files for testing

One or more known services

At the service hosts ``service-1`` and ``service-2`` run:

.. code:: sh

logsweet watch -b *:9000 "/var/log/myservice/*.log"

To watch the two services:

.. code:: sh

logsweet listen -c service-1:9000 -c service-2:9000

Use a proxy for dynamic services

First start the proxy on host ``log-proxy`` to wait for backend
connections from services on port 9001 and frontend connections from
listeners on port 9002:

.. code:: sh

logsweet proxy -bb *:9001 -fb *:9002

Run one or more listeners to print the messages:

.. code:: sh

logsweet listen -c log-proxy:9002

Start one or more watchers to send log messages to the proxy:

.. code:: sh

logsweet watch -c log-proxy:9001 "/var/log/myservice/*.log"

Use two proxies for high availability

First start two proxies on host ``log-proxy-1`` and ``log-proxy-2``:

.. code:: sh

logsweet proxy -bb *:9001 -fb *:9002

Then start one or more listeners to print the messages:

.. code:: sh

logsweet listen -c log-proxy-1:9002 -c log-proxy-2:9002

Start one or more watchers to send log messages to the proxies:

.. code:: sh

logsweet watch -c log-proxy-1:9002 -c log-proxy-2:9002 "/var/log/myservice/*.log"

If multiple proxies are alive, the load is balanced evenly over all
proxies. If a proxy becomes unavailable, its load is automatically taken
by the remaining proxies.

Write random log messages for testing

You can write random log messages to one or more log files in order to
test *logsweet* and its usage scenarios:

.. code:: sh

logsweet mock -i 0.2 /var/log/test/1.log /var/log/test/2.log

Use the ``-i`` or ``--interval`` option to control the interval for new
log messages in seconds.


The commands ``watch`` and ``listen`` can process a YAML configuration
file. The configuration controls *filtering*, *coloring* output and
triggering *actions*.

For the command ``watch`` to print the text lines, the switch ``--echo``
must be used.

A configuration file is specified with the ``-cfg <yaml file>`` option.
Here is an example:

.. code:: yaml

version: '0.1'

# The include statement is an array of regular expressions or a single one.
# A log messages is dropped if it does not match any of the expressions.
- 'error|Error|ERROR'
- 'warning|Warning|WARNING'

# The exclude statement is an array of regular expressions or a single one.
# A log message is dropped if it matches any of the expressions.
exclude: 'ignore\s+this'

# The colors statement is an array of colorization rules.
# The rules are tried in the given order until one matches.
# A colorization rule must have a pattern, which is a regular expression,
# optionally with named groups.
- pattern: '\[ERROR\]'
# There are multiple color statements with the syntax of.
# <foreground color> [on <background color>]
# Every color is a color name from the X11 rgb.txt.
# Spcifically the names supported by the Python package colorful
# are supported.
# The color names are Camel Case with a lower first letter.
line: red # line specifies a color for the whole line
match: black on lightGrey # match specifies a color for the whole match

- pattern: '^.*?(?P<level>\[[A-Z]+\]).*(?P<user>admin|user)'
line: lightGrey
match: white
level: yellow
user: cyan

# The actions statement is an array of action rules.
# The rules are all tried, regardless of matches.
# To execute actions from a watch or listen command,
# the switch -x or --exec-action must be used.
# An action rule must have a pattern, which is a regular expression,
# optionally with named groups.
- pattern: '\[ERROR\] (?P<err>.*)$'
# If the rule has an url property, it is considered an HTTP GET action.
# The URL is treaded as a Python string template,
# that means it can contain variables from the named regex groups.
url: '${err}'
# A timeout in seconds can be specified,
# to limit the amount of time used to execute the HTTP request.
timeout: 2.5


All notable changes to this project will be documented in this file.
Until this project reaches the maturity of version 1.0.0, the versioning
follows the following rules:

- The MAJOR number is 0.
- The MINOR number increases, if backward compatibility was broken
or an important set of features has been added.
- The PATCH number increases, if backward compatible features
have been added or bugs got fixed.

After this project reaches the version number 1.0.0 it will follow
`Semantic Versioning <>`_.

This document follows the guidelines in
Use the following change groups:

Added, Changed, Deprecated, Removed, Fixed, Security

Add a link to the GitHub diff like

.. _`<this version>`:<last-version>...v<this-version>
`<this version>`_ - <date>



- proxy
- binding and connecting on all sides
- Docker release
- YAML configuration for filtering, coloring and HTTP actions
- improved CLI

0.0.0 - 2019-02-21


- basic project structure
- text file watcher with ZeroMQ PUB
- ZeroMQ SUB with printing
- mockup log files
- basic CLI

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

logsweet-0.1.0.tar.gz (17.2 kB view hashes)

Uploaded Source

Built Distribution

logsweet-0.1.0-py3-none-any.whl (22.4 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page