An internal Quantile development kit for making working with data easier
Project description
Quantile Data Kit 🔍
Publish to pypi
How to deploy a new version of the QDK?
- Update the package version in
setup.py
. - Run the Makefile
make publish
Components
There are four types of base components in the QDK.
LoadComponent
. Takes nothing as input and outputs a DataFrame.TransformComponent
. Takes a DataFrame as input and outputs a DataFrame.TrainingComponent
. Takes data and a model as input and outputs a trained model.InferenceComponent
. Takes data and a model as input and ouputs prediction data.
Adding a new component?
Adding a new component to the QDK requires the following steps:
- Type of component: Decide which type of the four components above you are adding.
- Add component: Once you decide which type of component you are adding, add in the corresponding folder (e.g.
qdk/loader
) a new Python file that inherits from the parent component. In this file you can optionally overwriteinput_defs
,output_defs
andconfig_schema
. When adding a new component, you are required to add a classmethod with the same name as thecompute_function
attribute on the parent class. The keys in theconfig_schema
are injected into the parameters of the compute function. Lastly, you need to import the new component toqdk/__init__.py
. This allows you to import it from top-level. - Write tests: To continuously check the robustness of the components, we highly encourage you to add tests using
pytest
. The tests can be added atqdk/tests
. Reminder to prefix the folder, files and functions withtest_
. One is able to test the components using either VScode testing or the terminal (e.g. withpytest -s qdk/tests/test_loaders
).
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
quantile-data-kit-0.0.43.tar.gz
(24.0 kB
view hashes)
Built Distribution
Close
Hashes for quantile_data_kit-0.0.43-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 91175e4e46f6195af0e6f2bdd9e8171a798aafc1e41c9cb6b78aadfc020054fa |
|
MD5 | 1e13743c46b323398ef002ea9caff419 |
|
BLAKE2b-256 | 61b0188edec714e78dbcbfbf39db97df51be5ab3819a77d72125c5f7046b3bf9 |