Training Pipeline

Install for Development

Create virtual environment:

cd training-pipeline
poetry shell
poetry install

Check the Set Up Additional Tools and Usage sections to see how to set up the additional tools and credentials you need to run this project.

Check out this Medium article for more details about this module.

Usage for Development

Run the scripts in the following order:

1. Start the hyperparameter tuning script:

python -m training_pipeline.hyperparameter_tuning

2. Upload the best config based on the previous hyperparameter tuning step:

python -m training_pipeline.best_config

3. Start the training script using the best configuration uploaded one step before:

python -m training_pipeline.train

NOTE: Be careful to set the ML_PIPELINE_ROOT_DIR variable as explain in this section.