AMULOG (A Manager for Unstructured LOGs)

Amulog is a tool to support system log management. The main function is to classify log messages with automatically generated log templates (formats and variable locations), and to store the data in a database. This system works on python3.

Source: https://github.com/amulog/amulog
Bug Reports: https://github.com/amulog/amulog/issues
Author: Satoru Kobayashi
License: BSD-3-Clause

Main features

Support multiple databases: sqlite and mysql
Smart log segmentation with log2seq
Multiple template generation algorithms such as: Drain, SHISO, LenMa, FT-tree, Dlog, etc.
Support Online (incremental) and Offline (hindsight) use
Suspend and resume the template generation process
Import and Export log templates if you need
Edit log templates manually if you need
Search API with datetime, hostname and log template IDs
Ready-to-run examples for loghub open datasets (see example/loghub_*)

Tutorial

Install

$ pip install amulog

Generate config

For the first step, save following config as test.conf on an empty directory.

[general]
src_path = logfile.txt
src_recur = false
logging = auto.log

[database]
database = sqlite3
sqlite3_filename = log.db

[log_template]
lt_methods = drain
indata_filename = ltgen.dump

Then modify general.src_path option to a logfile you want to load. (If you want to use multiple files, change general.src_recur into true and specify directory name to general.src_path.)

Generate database

Try following command to generate database:

$ python -m amulog db-make -c test.conf

Check database

$ python -m amulog show-db-info -c test.conf

shows status of the generated database.

$ python -m amulog show-lt -c test.conf

shows all generated log templates in the given logfile.

$ python -m amulog show-log -c test.conf ltid=2

shows all log messages corresponding to log template ID 2.

Resume generating database

Try following command to resume generating database:

$ python -m amulog db-add -c test.conf logfile2.txt

Export and Import templates

Following command exports all log templates in the database:

$ python3 -m amulog show-lt-import -c test.conf > exported_tpl.txt

You can modify the exported templates manually. Note that some special letters (\\, @, *) are escaped in the exported templates.

To import the templates, save following config as test2.conf.

[general]
src_path = logfile.txt
src_recur = false
logging = new_auto.log

[database]
database = sqlite3
sqlite3_filename = new_log.db

[log_template]
lt_methods = import
indata_filename = new_ltgen.dump

[log_template_import]
def_path = exported_tpl.txt

Then, try generating database again:

python -m amulog db-make -c test2.conf

Using your data

Parsing your data

Amulog uses log2seq to parse input log messages in DB generation. If your data is not a default syslog output format, you need to specify an appropriate log2seq parser script. The log2seq parser script is specified in manager.parser_script in amulog config file.

[manager]
parser_script = test_parser.py
fail_output = fail.log

If the parser fails to parse some of the input log messages, they are stored in manager.fail_output file. You can check this file to test whether the parser is working appropriately or not.

There are example parser scripts in log2seq repository.

Further usage

see help with following command:

python -m amulog -h

Reference

This tool is demonstrated at International Journal of Network Management and CNSM2020.

If you use this code, please consider citing:

@article{Kobayashi_IJNM2022,
  author = {Kobayashi, Satoru and Yamashiro, Yuya and Otomo, Kazuki and Fukuda, Kensuke},
  title = {amulog: A general log analysis framework for comparison and combination of diverse template generation methods*},
  journal = {International Journal of Network Management},
  volume = {32},
  number = {4},
  pages = {e2195},
  doi = {https://doi.org/10.1002/nem.2195},
  year = {2022}
}

@inproceedings{Kobayashi_CNSM2020,
  author = {Kobayashi, Satoru and Yamashiro, Yuya and Otomo, Kazuki and Fukuda, Kensuke},
  booktitle = {Proceedings of the 16th International Conference on Network and Service Management (CNSM'20)},
  title = {amulog: A General Log Analysis Framework for Diverse Template Generation Methods},
  pages={1-5},
  year = {2020}
}

Metadata

Release files for amulog 0.5.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for amulog 0.5.2
File	Size	Uploaded
amulog-0.5.2.tar.gz	111.9 kB	Sep 29, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for amulog 0.5.2
File	Interpreter	ABI	Platform	Reset
amulog-0.5.2-py3-none-any.whl 131.0 kB Sep 29, 2026	Python 3	none	any	Details

Total release size: 242.9 kB

Release files / amulog-0.5.2.tar.gz

Download URL	amulog-0.5.2.tar.gz
Size	111.9 kB
Tags	Source
SHA-256 checksum How to use checksums	`5d4c392dffa6898797d89e93eaf17f47b15512e4d40497c7035acb476060649b`
BLAKE2b-256 checksum How to use checksums	`b0e8de4f6fb17f39559a8e6564ba5b80c7702f6802493484a096ccb349d4a0ab`
Upload date	Sep 29, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/amulog/amulog
Publishing commit: github.com/amulog/amulog/tree/10f341c9e8454a61dcb77bfbf669d640a8311fa6

Workflow

Publishing configuration: github.com/amulog/amulog/blob/10f341c9e8454a61dcb77bfbf669d640a8311fa6/.github/workflows/publish.yml
Publishing logs: github.com/amulog/amulog/actions/runs/36546249792/attempts/1

Artifact

Subject: amulog-0.5.2.tar.gz
SHA-256 checksum: 5d4c392dffa6898797d89e93eaf17f47b15512e4d40497c7035acb476060649b Verifying attestations

Release files / amulog-0.5.2-py3-none-any.whl

Download URL	amulog-0.5.2-py3-none-any.whl
Size	131.0 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`931d6c8a6c6c6e8d94176c90b7adafa6bdc0621d44c69cf466ab09f78d1ac38e`
BLAKE2b-256 checksum How to use checksums	`cfa59aa44f0dcf1c4c7161e66be1884458c9c5b33312500cd64c4e6aa27ecf99`
Upload date	Sep 29, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	Yes
Uploaded via	`twine/7.0.0 CPython/3.13.14`

Provenance

Identity

Publishing platform: GitHub Actions
Publishing repository: github.com/amulog/amulog
Publishing commit: github.com/amulog/amulog/tree/10f341c9e8454a61dcb77bfbf669d640a8311fa6

Workflow

Publishing configuration: github.com/amulog/amulog/blob/10f341c9e8454a61dcb77bfbf669d640a8311fa6/.github/workflows/publish.yml
Publishing logs: github.com/amulog/amulog/actions/runs/36546249792/attempts/1

Artifact

Subject: amulog-0.5.2-py3-none-any.whl
SHA-256 checksum: 931d6c8a6c6c6e8d94176c90b7adafa6bdc0621d44c69cf466ab09f78d1ac38e Verifying attestations