logs parsing subsystem library for airflow
Project description
LogsParser
Common log parser and analyzer
Project structure
How to dev
poetry shell
poetry install
├── debug.py
├── docs
│ ├── context diagram.drawio
│ └── uml.drawio
├── logs_parser
│ ├── analyzer - analyzer for logs
│ │ ├── analyzer.py
│ │ ├── data - data for IP2Location Lib
│ │ │ ├── IP2LOCATION-LITE-DB11.BIN
│ │ │ ├── LICENSE_LITE.TXT
│ │ │ └── README_LITE.TXT
│ │ └── plots.py - plots (deprecated)
│ ├── parser (multithread parser)
│ │ ├── filter.py - filter decorators
│ │ ├── logs_reader.py - logs reader
│ │ ├── log_structure.py - data classes
│ │ ├── logs_writer.py - logs writers
│ │ └── parser.py - parser
│ └── python_callables.py - callables for Airflow
├── notebooks - Jupyter notebooks,some experiment
│ ├── experiment.ipynb
│ └── gorod_09102021_231021.csv
├── poetry.lock
├── pyproject.toml
├── README.md
└── scripts - utils
├── split_logs.py
└── unite_logs.py
Arhitecture contex diagram
UML diagram
How to public to Pypi
First you need create api on pypi
poetry config pypi-token.pypi $(cat .token)
poetry publish --build
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
logsparsersubsystem-1.0.1.tar.gz
(21.6 MB
view hashes)
Built Distribution
Close
Hashes for logsparsersubsystem-1.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2a087f89235b5963b97c898b611a66c769ea67f99225eb650d66aedbf3443c79 |
|
MD5 | e2710927fa1d486619ea1409ec876a7a |
|
BLAKE2b-256 | 325689372528ec4d1818ebb912242eb09e9c1f7610f6059bdb112173ea7a1e92 |
Close
Hashes for logsparsersubsystem-1.0.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d7f8ea9713426d9e599c04ea4a7c12ef073caffb1e37a398051d6babaae2bcb |
|
MD5 | 6a20d3da618f7981c8bff9e20eaface9 |
|
BLAKE2b-256 | 3957f49418f72be5e3b355b32c4e291c151c118d748812c4d815bdc74abb0b9d |