Theia is a lightweight log aggregator.
Simple and lightweight log aggregator
Theia is a log aggregator that collects logs from multiple log files usually on multiple machines and provides the log entries on a single endpoint. Each recorded entry comes from a single source (log file) and can be tagged with multiple tags. This way you can query an filter by the source (for example /var/log/apache or /var/log/reds) and tags (for example "production1", "cache-server", "testing" etc), or you can just simply match a pattern against the content of the entry.
Theia has three parts:
- watcher - this is the agent process running on the machine where you have your logs. You will have multiple watchers, but usually a single watcher per node
- collector - this is the master process, usually running on a sperate node where you're going to collect the logs. Provides the interface for pushing the log events and an interface for querying the aggregated logs.
- client - client for querying and watching live events. This connects to the collector via websocket and queries the collected events or registers a filter for live events.
Theia comes as a single python package (only python3 is supported).
pip install theia
Running the collector
This process collects and stores the events. The built-in database uses text files to store the events.
Create a directory where you want to keep the events, and run the collector server:
mkdir data python -m theia.cli collect -d ./data
This will run the server listening on port
Running a watcher
Make sure you have the collector running.
Then, run the watcher:
python -m theia.cli watch -c localhost -f /var/log/httpd/access.log -t web-access
This would watch for changes in
access.log and send events to the collector
Run a query
To query for all log events with tag
python -m theia.cli query -c localhost -t web-access
Run a live filter
To watch the events with tag
web-access as they arrive in real-time:
python -m theia.cli query -l -c localhost -t web-access
This is still work in progress and is far from production ready
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size theia-0.9.2.tar.gz (30.6 kB)||File type Source||Python version None||Upload date||Hashes View|