An internal Quantile development kit for making working with data easier
Project description
Quantile Data Kit 🔍
Publish to pypi
How to deploy a new version of the QDK?
- Update the package version in
setup.py
. - Run the Makefile
make publish
Components
There are four types of base components in the QDK.
LoadComponent
. Takes nothing as input and outputs a DataFrame.TransformComponent
. Takes a DataFrame as input and outputs a DataFrame.TrainingComponent
. Takes data and a model as input and outputs a trained model.InferenceComponent
. Takes data and a model as input and ouputs prediction data.
Adding a new component?
Adding a new component to the QDK requires the following steps:
- Type of component: Decide which type of the four components above you are adding.
- Add component: Once you decide which type of component you are adding, add in the corresponding folder (e.g.
qdk/loader
) a new Python file that inherits from the parent component. In this file you can optionally overwriteinput_defs
,output_defs
andconfig_schema
. When adding a new component, you are required to add a classmethod with the same name as thecompute_function
attribute on the parent class. The keys in theconfig_schema
are injected into the parameters of the compute function. Lastly, you need to import the new component toqdk/__init__.py
. This allows you to import it from top-level. - Write tests: To continuously check the robustness of the components, we highly encourage you to add tests using
pytest
. The tests can be added atqdk/tests
. Reminder to prefix the folder, files and functions withtest_
. One is able to test the components using either VScode testing or the terminal (e.g. withpytest -s qdk/tests/test_loaders
).
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
quantile-data-kit-0.0.30.tar.gz
(23.6 kB
view hashes)
Built Distribution
Close
Hashes for quantile_data_kit-0.0.30-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | b6f1419442accd6a9163299e4fc61964e5128671d79e244d0544f0be19eccd53 |
|
MD5 | bf0f843c962994caecea82752b62e022 |
|
BLAKE2b-256 | f371b62df95500e8178b57a44e42b9568a2cafe4b10da1f815cc6073299df205 |