No project description provided
Project description
Data Extract-Load-Transform for Autosink Project
Parse the values in the data lake and load them into memory, then make it executable for the data labeler. It organizes the labeled data into a format that is easy for the feature store to store. The autosink-data-elt contains utility classes and data type definitions for reading and writing json files stored in the data lake. When the data extracted from the Raspberry Pi is stored in the data lake, it is loaded, labeled, and transformed into a form that can be stored in the feature store, so it is named ELT.
Features
- Parse the values in the data lake and load them into memory, then organize them into a format that is easy for the data labeler to read.
- Call the data labeler.
- Store the labeled data in the feature store.
Environment
The environment is based on MacOS and Linux.
Makefile
The Makefile
has the following functions.
make lint
- To use the
.vscode
settings, install thepylint
extension. - Override the options specified in the
pyproject.toml
file to lint the code with the default settings of the linter.
make format
- The formatter uses google's
yapf
. - Override the options specified in the
pyproject.toml
file to format the code with the default settings of theyapf
formatter. - To use the
.vscode
settings, install theyapf
extension.
make test
- The test uses
unittest
. - Supports both
test_*.py
and*_test.py
patterns. - The test file must be connected to
__init__.py
up to the location where the test file exists.
make publish
- Write the
~/.pypirc
file as follows.[pypi] username = __token__ password = pypi-어쩌고저쩌고 # Write your personal API token.
- This command uses
flit
to push the package to the PyPI public registry. - The package uploaded with the name specified earlier as `
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file autosink_data_elt-0.2.0.tar.gz
.
File metadata
- Download URL: autosink_data_elt-0.2.0.tar.gz
- Upload date:
- Size: 12.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: python-requests/2.31.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 254d2a7ceb4c191017cdacbc19114395eb5c8b8716061381712c9e622579d33e |
|
MD5 | 3e5a7ba69b9c7b8fc64ce3789e401fd0 |
|
BLAKE2b-256 | f563cb66de72965312a3b55b2c7303db786187cf95a88af2a1311beef6716954 |
File details
Details for the file autosink_data_elt-0.2.0-py3-none-any.whl
.
File metadata
- Download URL: autosink_data_elt-0.2.0-py3-none-any.whl
- Upload date:
- Size: 10.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: python-requests/2.31.0
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1de9dd147d62597b212afe9a4024364a63d8758ae1eb15e74d865da4c45a0ec0 |
|
MD5 | bcbd576e345b293fdcb6590e58c21725 |
|
BLAKE2b-256 | d4b5a8353fd3b087809ebb11d40a9e75c358878f88e53d280a323142f9b5227f |