Skip to main content

logs watching + webui

Project description


distributed log tail viewer


- tail log files and forward them to web in runtime
- dynamically parse and (pre)process logs
- aggregate and collect logs
- alerting

Quick start

Running a simple one-host configuration.

.. code-block:: bash

$ pip install logdog

Help output:

.. code-block:: bash

$  logdog --help 
logdog command line interface

logdog [<pipe-namespace>...] [options]
logdog (-h | --help)
logdog --version

<pipe-namespace> One or more pipe namespaces to be run

-h --help Show this screen
--version Show version
-v --verbose Run in verbose mode
-l --log-level=<level> Set internal logging level [default: INFO]
-f --log-format=<format> Set internal logging format [default: quiet]
-c --config=<config> Configuration file (yaml config)
-s --sources=<file:...> Force specify files to be watched
-H --handler=<handler> Force set handler for all sources
(e.g. --handler=viewers.console)
--reset-indices Remove current indices (will reset watching state)

Prepare config file:

.. code-block:: yaml

# config.yml
- /var/log/*.log
- /var/log/*/*.log
- /var/log/syslog

Please, see ``_

.. logdog/

Start watching:

.. code-block:: bash

$ logdog --config=config.yml

You can run watching and viewing parts separately:

.. code-block:: bash

$ logdog watch --config=config.yml
# another console
$ logdog view --config=config.yml


YAML is used as a main format for writing configuration.

Default config:

.. code-block:: yaml

# <path-to-file>
- /var/log/*.log
- /var/log/*/*.log
- /var/log/syslog

``sources`` is a list of target files/logs. Alternatively, this section can be re-written the following way:

.. code-block:: yaml

- /var/log/*.log:
- /var/log/*/*.log:
# ^ note. 4 spaces
# in case of 2 spaces it will be a key in the list object
# {'/var/log/*/*.log': None,
# 'handler': ''}
# but must be {'/var/log/*/*.log': {'handler': ''}}
- /var/log/syslog: {handler:}

Pipe is a sequence of steps to process / parse / forward / collect log messages.
```` is a predefined pipe (see ``_).

Full ``sources`` format:

.. code-block:: none

- (path | search pattern)
# or (`handler`, `watcher`, `meta` are optional)
- (path | search pattern):
handler: handler-name # default
watcher: watcher-name # default pollers.file-watcher
meta: a-dictionary-containing-any-meta-info # e.g. {tags: [tag1, tag2]}
# or
- (path | search pattern): handler-name
# or
- (path | search pattern): {handler:}
# or
- (path | search pattern): {watcher: poller.custom-file-poller}
# or
- (path | search pattern): {meta: {tags: [log]}}

Example 1:

.. code-block:: yaml

- /var/log/syslog: {handler:, meta: {tags: [syslog]}
# or
- /var/log/syslog2:
- syslog


Predefined configs:


- `` - defines a simple flow (strip -> zmq localhost:7789 -> zmq *:7789 -> webui)


- `viewers.null` - does nothing with incoming data
- `viewers.console` - print incoming log messages into stdout
- `viewers.webui` - forward all incoming messages to all connected clients using websockets


- `connectors.zmq-tunnel` - allows to create any zmq sockets to push/pull data

For more details see ``_.


.. image::


- cover with tests
- detecting new files
- colorize logs
- add documentation
- zmq connectors
- mongodb collector
- webui storages
- webui filtering / searching
- implement `--validate-config`

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

logdog-0.2.11.tar.gz (213.0 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page