A log anomaly detection framework
Project description
Alogamous
Table of Contents
Description
A Python program that can process log files and generate a daily report on anomalies in a set of logs.
- It should be able to read multiple files and output an html-formatted file.
- It should have multiple metrics/checks for anomalous log lines.
- It should be able to take in a log line format so that it knows how to extract useful information from a log line.
- It should be able to traverse a well-defined file system structure to access all logs.
- It should be able to generate metrics that span multiple files.
- It should be well-tested.
Installation
pip install alogamous
Development
Setup
-
Open a terminal
-
Install hatch
curl -Lo hatch-universal.pkg https://github.com/pypa/hatch/releases/latest/download/hatch-universal.pkg sudo installer -pkg ./hatch-universal.pkg -target /
-
Restart your terminal
Hatch modifies your system PATH variable, and this won't take effect unless you restart the terminal.
-
Make sure hatch works
hatch --version
-
Configure hatch
hatch config set dirs.env.virtual .venv
-
Clone the repo
git clone https://github.com/aquanauts/alogamous.git cd alogamous
-
Install pre-commit hooks
This will make sure certain checks are run when committing code.
hatch run pre-commit:install
-
Run the tests
hatch test
-
Setup your IDE
-
Sign up for a free JetBrains Educational License or buy one
-
Download Pycharm
- Depending on your platform you may need to download and install Python
-
Open Pycharm
-
Click Open on the welcome screen and then select the file alogamous
-
Possible Errors
Permission Error
-
Message: PermissionError: [Errno 13] Permission denied: '/usr/local/hatch/bin/hatch'
-
Fix by running:
sudo chmod a+r /usr/local/hatch/bin/hatch
Adding to Gitignore
Gitignore is a way to explicitly tell Git that certain files should not be committed. For example, we did this with the .idea folder, which is automatically generated and contains settings that do not need to be committed.
- Create the gitignore file
touch .gitignore
- Add the current file to the gitignore
git add .
- Commit these changes
git commit
If you open the .gitignore file, you can add lines directly there as well.
- If you would like to exclude any file or directory whose name begins with a certain phrase (e.g. "hello."), you can use the pattern of name followed by asterisk (e.g. hello.*)
- If you would like to match a directory and the paths underneath it, but not a regular file with the same name, you can use the pattern of name followed by forward slash (e.g. foo/)
- If you would like to include a certain file which has been excluded by a previous pattern, you can use the prefix "!". (Note that it is not possible to re-include a file if a parent directory of that file is excluded)
- For additional information on gitignore's pattern format and examples, see the git docs
License
alogamous
is distributed under the terms of the MIT license.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for alogamous-1.0.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 88928aa491a3e947c95666dcd762f36f763dbda9b955dfe617917b88eb279b28 |
|
MD5 | c184ca2e93490abcb1e7910ce321fc41 |
|
BLAKE2b-256 | 4f3f0b39337556aabead128b62406214804debf07f9dc890393c39956706f9b5 |