logs parsing subsystem library for airflow
Project description
LogsParser
Common log parser and analyzer
Project structure
How to dev
poetry shell
poetry install
├── debug.py
├── docs
│ ├── context diagram.drawio
│ └── uml.drawio
├── logs_parser
│ ├── analyzer - analyzer for logs
│ │ ├── analyzer.py
│ │ ├── data - data for IP2Location Lib
│ │ │ ├── IP2LOCATION-LITE-DB11.BIN
│ │ │ ├── LICENSE_LITE.TXT
│ │ │ └── README_LITE.TXT
│ │ └── plots.py - plots (deprecated)
│ ├── parser (multithread parser)
│ │ ├── filter.py - filter decorators
│ │ ├── logs_reader.py - logs reader
│ │ ├── log_structure.py - data classes
│ │ ├── logs_writer.py - logs writers
│ │ └── parser.py - parser
│ └── python_callables.py - callables for Airflow
├── notebooks - Jupyter notebooks,some experiment
│ ├── experiment.ipynb
│ └── gorod_09102021_231021.csv
├── poetry.lock
├── pyproject.toml
├── README.md
└── scripts - utils
├── split_logs.py
└── unite_logs.py
Arhitecture contex diagram
UML diagram
How to public to Pypi
First you need create api on pypi
poetry config pypi-token.pypi $(cat .token)
poetry publish --build
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
logsparsersubsystem-1.0.2.tar.gz
(21.6 MB
view hashes)
Built Distribution
Close
Hashes for logsparsersubsystem-1.0.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 22d2ab2ea32fa04b23b23efc2adb850aae750ab7898f992e9b1b23f3cb25c05e |
|
MD5 | f1b754e8e6fdb73f7f8374200e2018b2 |
|
BLAKE2b-256 | ba51628f05f122a20d15f645c5296ecf7b1873a106be1f0c1a3dd8044b2e2df2 |
Close
Hashes for logsparsersubsystem-1.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 046dca09d32af0bcff6846e92c9014432465f99f943ed10867a55121c19312ff |
|
MD5 | aa65680a4c45d31e91693c7e3d586c2d |
|
BLAKE2b-256 | 5b9f3fa8943d742a1b2158c50267f6ca10ddd6cecf6b814892de1dbfba79dfa8 |