An internal Quantile development kit for making working with data easier
Project description
Quantile Data Kit 🔍
Publish to pypi
How to deploy a new version of the QDK?
- Update the package version in
setup.py
. - Remove the old build folder
rm -r build dist
. - Build the new version
python setup.py sdist bdist_wheel
. - Publish to pypi
python -m twine upload dist/*
.
Components
There are four types of base components in the QDK.
LoadComponent
. Takes nothing as input and outputs a DataFrame.TransformComponent
. Takes a DataFrame as input and outputs a DataFrame.TrainingComponent
. Takes data and a model as input and outputs a trained model.InferenceComponent
. Takes data and a model as input and ouputs prediction data.
Adding a new component?
Adding a new component to the QDK requires the following steps:
- Type of component: Decide which type of the four components above you are adding.
- Add component: Once you decide which type of component you are adding, add in the corresponding folder (e.g.
qdk/loader
) a new Python file that inherits from the parent component. In this file you can optionally overwriteinput_defs
,output_defs
andconfig_schema
. When adding a new component, you are required to add a classmethod with the same name as thecompute_function
attribute on the parent class. The keys in theconfig_schema
are injected into the parameters of the compute function. Lastly, you need to import the new component toqdk/__init__.py
. This allows you to import it from top-level. - Write tests: To continuously check the robustness of the components, we highly encourage you to add tests using
pytest
. The tests can be added atqdk/tests
. Reminder to prefix the folder, files and functions withtest_
. One is able to test the components using either VScode testing or the terminal (e.g. withpytest -s qdk/tests/test_loaders
).
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
quantile-data-kit-0.0.9.tar.gz
(13.6 kB
view hashes)
Built Distribution
Close
Hashes for quantile_data_kit-0.0.9-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 7c8df4400d4ca70abf3374fd1f9033baeb5419603c0f846d87d459699f284538 |
|
MD5 | ac2502e5df8f93680e7fdf507b8052de |
|
BLAKE2b-256 | 84d6746757ebd4e244b44a5c28cfaad76659c1037958b02febe1e9079fc77ad9 |