An internal Quantile development kit for making working with data easier
Project description
Quantile Data Kit 🔍
Publish to pypi
How to deploy a new version of the QDK?
- Update the package version in
setup.py
. - Run the Makefile
make publish
Components
There are four types of base components in the QDK.
LoadComponent
. Takes nothing as input and outputs a DataFrame.TransformComponent
. Takes a DataFrame as input and outputs a DataFrame.TrainingComponent
. Takes data and a model as input and outputs a trained model.InferenceComponent
. Takes data and a model as input and ouputs prediction data.
Adding a new component?
Adding a new component to the QDK requires the following steps:
- Type of component: Decide which type of the four components above you are adding.
- Add component: Once you decide which type of component you are adding, add in the corresponding folder (e.g.
qdk/loader
) a new Python file that inherits from the parent component. In this file you can optionally overwriteinput_defs
,output_defs
andconfig_schema
. When adding a new component, you are required to add a classmethod with the same name as thecompute_function
attribute on the parent class. The keys in theconfig_schema
are injected into the parameters of the compute function. Lastly, you need to import the new component toqdk/__init__.py
. This allows you to import it from top-level. - Write tests: To continuously check the robustness of the components, we highly encourage you to add tests using
pytest
. The tests can be added atqdk/tests
. Reminder to prefix the folder, files and functions withtest_
. One is able to test the components using either VScode testing or the terminal (e.g. withpytest -s qdk/tests/test_loaders
).
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
quantile-data-kit-0.0.38.tar.gz
(23.7 kB
view hashes)
Built Distribution
Close
Hashes for quantile_data_kit-0.0.38-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 35bffc14ff8c43b68005f1cd24ed6097d6cacc5a32de4b25354453cb1fc1167f |
|
MD5 | 3ee8651ee104c8769fb6d56fbe39596f |
|
BLAKE2b-256 | 8414b955dc321773622549f0b256ff5cc203056e0759f4392b1683cd8e63e421 |