An internal Quantile development kit for making working with data easier
Project description
Quantile Data Kit 🔍
Publish to pypi
How to deploy a new version of the QDK?
- Update the package version in
setup.py
. - Run the Makefile
make publish
Components
There are four types of base components in the QDK.
LoadComponent
. Takes nothing as input and outputs a DataFrame.TransformComponent
. Takes a DataFrame as input and outputs a DataFrame.TrainingComponent
. Takes data and a model as input and outputs a trained model.InferenceComponent
. Takes data and a model as input and ouputs prediction data.
Adding a new component?
Adding a new component to the QDK requires the following steps:
- Type of component: Decide which type of the four components above you are adding.
- Add component: Once you decide which type of component you are adding, add in the corresponding folder (e.g.
qdk/loader
) a new Python file that inherits from the parent component. In this file you can optionally overwriteinput_defs
,output_defs
andconfig_schema
. When adding a new component, you are required to add a classmethod with the same name as thecompute_function
attribute on the parent class. The keys in theconfig_schema
are injected into the parameters of the compute function. Lastly, you need to import the new component toqdk/__init__.py
. This allows you to import it from top-level. - Write tests: To continuously check the robustness of the components, we highly encourage you to add tests using
pytest
. The tests can be added atqdk/tests
. Reminder to prefix the folder, files and functions withtest_
. One is able to test the components using either VScode testing or the terminal (e.g. withpytest -s qdk/tests/test_loaders
).
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
quantile-data-kit-0.0.25.tar.gz
(16.8 kB
view hashes)
Built Distribution
Close
Hashes for quantile_data_kit-0.0.25-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 09650e6c682050277b669350ab769caeea428f312a2bab75b9fa52d259ec5c8a |
|
MD5 | d2846d6889c9ef649b34066772a9b4eb |
|
BLAKE2b-256 | 8039424ea8bbbbe2ce6f88a07d943189811b79df152cd4e98083053ee5468e6f |