This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description
python-logdog
-------------

distributed log tail viewer

Why?

- tail log files and forward them to web in runtime
- dynamically parse and (pre)process logs
- aggregate and collect logs
- alerting


Quick start
===========

Running a simple one-host configuration.

.. code-block:: bash

$ pip install logdog

Help output:

.. code-block:: bash

$  logdog --help 
logdog command line interface

Usage:
logdog [<pipe-namespace>...] [options]
logdog (-h | --help)
logdog --version

Arguments:
<pipe-namespace> One or more pipe namespaces to be run

Options:
-h --help Show this screen
--version Show version
-v --verbose Run in verbose mode
-l --log-level=<level> Set internal logging level [default: INFO]
-f --log-format=<format> Set internal logging format [default: quiet]
-c --config=<config> Configuration file (yaml config)
-s --sources=<file:...> Force specify files to be watched
-H --handler=<handler> Force set handler for all sources
(e.g. --handler=viewers.console)
--reset-indices Remove current indices (will reset watching state)


Prepare config file:

.. code-block:: yaml

# config.yml
---
sources:
- /var/log/*.log
- /var/log/*/*.log
- /var/log/syslog

Please, see `default_config.py`_

.. _default_config.py: logdog/default_config.py

Start watching:

.. code-block:: bash

$ logdog --config=config.yml

You can run watching and viewing parts separately:

.. code-block:: bash

$ logdog watch --config=config.yml
# another console
$ logdog view --config=config.yml


Config
======

YAML is used as a main format for writing configuration.

Default config:

.. code-block:: yaml

---
sources:
# <path-to-file>
- /var/log/*.log
- /var/log/*/*.log
- /var/log/syslog

``sources`` is a list of target files/logs. Alternatively, this section can be re-written the following way:

.. code-block:: yaml

---
sources:
- /var/log/*.log: pipes.to-web
- /var/log/*/*.log:
handler: pipes.to-web
# ^ note. 4 spaces
# in case of 2 spaces it will be a key in the list object
# {'/var/log/*/*.log': None,
# 'handler': 'pipes.to-web'}
# but must be {'/var/log/*/*.log': {'handler': 'pipes.to-web'}}
- /var/log/syslog: {handler: pipes.to-web}


Pipe is a sequence of steps to process / parse / forward / collect log messages.
``pipes.to-web`` is a predefined pipe (see `default_config.py`_).


Full ``sources`` format:

.. code-block:: none

---
sources:
- (path | search pattern)
# or (`handler`, `watcher`, `meta` are optional)
- (path | search pattern):
handler: handler-name # default pipes.to-web
watcher: watcher-name # default pollers.file-watcher
meta: a-dictionary-containing-any-meta-info # e.g. {tags: [tag1, tag2]}
# or
- (path | search pattern): handler-name
# or
- (path | search pattern): {handler: pipes.to-web}
# or
- (path | search pattern): {watcher: poller.custom-file-poller}
# or
- (path | search pattern): {meta: {tags: [log]}}


Example 1:

.. code-block:: yaml

---
sources:
- /var/log/syslog: {handler: pipes.to-web, meta: {tags: [syslog]}
# or
- /var/log/syslog2:
handler: pipes.to-web
meta:
tags:
- syslog


Builtins
========

Predefined configs:

``pipes``:

- `pipes.to-web` - defines a simple flow (strip -> zmq localhost:7789 -> zmq *:7789 -> webui)

``viewers``:

- `viewers.null` - does nothing with incoming data
- `viewers.console` - print incoming log messages into stdout
- `viewers.webui` - forward all incoming messages to all connected clients using websockets

``connectors``:

- `connectors.zmq-tunnel` - allows to create any zmq sockets to push/pull data

For more details see `default_config.py`_.

Screenshots
===========

.. image:: http://i.imgur.com/B4JQ57T.png


TODO
====

- cover with tests
- detecting new files
- colorize logs
- add documentation
- zmq connectors
- mongodb collector
- webui storages
- webui filtering / searching
- implement `--validate-config`
Release History

Release History

0.2.11

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.9

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.8

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.7

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.5

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.2.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
logdog-0.2.11.tar.gz (213.0 kB) Copy SHA256 Checksum SHA256 Source Jul 4, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting