ETL Entities lib for onETL
Project description
What is ETL Entities?
Collection of classes used for handling High Water Mark (HWM) and gathering Lineage graph.
- Currently implemented:
IntHWM
DateHWM
DateTimeHWM
FileListHWM
Column
Table
RemoteFolder
Process
How to install
pip install etl-entities
Documentation
Contribution guide
See CONTRIBUTING.rst
Security
See SECURITY.rst
Develop
Clone repo
Clone repo:
git clone git@github.com:MobileTeleSystems/etl-entities.git -b develop
cd etl-entities
Setup environment
Create virtualenv and install dependencies:
# create virtual environment
python -m venv venv
source venv/bin/activate
pip install -U wheel
pip install -U pip setuptools
# install requirements
pip install -U -r requirements.txt
Install dependencies for development:
# install linters, formatters, etc
pip install -U -r requirements-dev.txt
Enable pre-commit hooks
Install pre-commit hooks:
pre-commit install --install-hooks
Test pre-commit hooks run:
pre-commit run
Run tests
# install requirements for testing
pip install -U -r requirements-test.txt
# run tests
pytest
Build documentation
# install requirements for documentation
pip install -U -r requirements-docs.txt
cd docs
# generate html documentation
make html
Then open docs/_build/html/index.html file in browser.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
etl-entities-1.3.2.tar.gz
(22.7 kB
view hashes)
Built Distribution
Close
Hashes for etl_entities-1.3.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 5dd1fb2b55224c54e24c4d22f5f342d3cc6d10b4b1dba52dbe43e92678b9e8d5 |
|
MD5 | b78bb7ecf76aa996f8e18f8ee9fb5225 |
|
BLAKE2b-256 | 7f20cdb88ca1245cd9739ce305c573ab4dca0f5f1229062a6df23b7580f8700f |