Batch Prediction Pipeline

Install for Development

The batch prediction pipeline uses the training pipeline module as a dependency. Thus, as a first step, we must ensure that the training pipeline module is published to our private PyPi server.

NOTE: Make sure that your private PyPi server is running. Check the Usage section if it isn't.

Build & publish the training-pipeline to your private PyPi server:

cd training-pipeline
poetry build
poetry publish -r my-pypi
cd ..

Install the virtual environment for batch-prediction-pipeline:

cd batch-prediction-pipeline
poetry shell
poetry install

Check the Set Up Additional Tools and Usage sections to see how to set up the additional tools and credentials you need to run this project.

Usage for Development

To start batch prediction script, run:

python -m batch_prediction_pipeline.batch

To compute the monitoring metrics based, run the following:

python -m batch_prediction_pipeline.monitoring

Check out this Medium article for more details about this module.

NOTE: Be careful to set the ML_PIPELINE_ROOT_DIR variable as explained in this section.