logs parsing subsystem library for airflow
Project description
LogsParser
Common log parser and analyzer
Project structure
How to dev
poetry shell
poetry install
├── debug.py
├── docs
│ ├── context diagram.drawio
│ └── uml.drawio
├── logs_parser
│ ├── analyzer - analyzer for logs
│ │ ├── analyzer.py
│ │ ├── data - data for IP2Location Lib
│ │ │ ├── IP2LOCATION-LITE-DB11.BIN
│ │ │ ├── LICENSE_LITE.TXT
│ │ │ └── README_LITE.TXT
│ │ └── plots.py - plots (deprecated)
│ ├── parser (multithread parser)
│ │ ├── filter.py - filter decorators
│ │ ├── logs_reader.py - logs reader
│ │ ├── log_structure.py - data classes
│ │ ├── logs_writer.py - logs writers
│ │ └── parser.py - parser
│ └── python_callables.py - callables for Airflow
├── notebooks - Jupyter notebooks,some experiment
│ ├── experiment.ipynb
│ └── gorod_09102021_231021.csv
├── poetry.lock
├── pyproject.toml
├── README.md
└── scripts - utils
├── split_logs.py
└── unite_logs.py
Arhitecture contex diagram
UML diagram
How to public to Pypi
First you need create api on pypi
poetry config pypi-token.pypi $(cat .token)
poetry publish --build
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
logsparsersubsystem-1.0.3.tar.gz
(21.6 MB
view hashes)
Built Distribution
Close
Hashes for logsparsersubsystem-1.0.3.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8dd7390fa5d51a63c1120980529e2b30039979c76aef5ae9891b0932c76661e6 |
|
MD5 | 07d988659d0156d68609617b54bff6d7 |
|
BLAKE2b-256 | c1c1f04fd74942303fcb09dcf5cdca9bdd3cfcde5c0f3399d0ef9cbe7f9f72a7 |
Close
Hashes for logsparsersubsystem-1.0.3-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | a3e71b53891954dd89b123a781c89eeaaf8656dd472629a28cb7617095ffb1b2 |
|
MD5 | 39f447ba0f0451330032be16aa6bf0f7 |
|
BLAKE2b-256 | aeb45b7e127b73c66f8c6a485bb5bd1505df6cba75455872fe25e82e9eae7353 |