Machine Learning infra
Project description
Agoge
An opinionated ML development environment
Train pytorch models, hyperparameter tune them with single loc change.
Libraries
Pytorch
Pytorch is a Python first machine learning library
Ray
Ray Provides easy experiment scaling + hyper parameter optimisation
MLFlow
Provides tracking of model metrics and hyperparameters. Also allows for intelligent storing of training artifacts
Static Components
These components should not need to be customised for model specific use cases
Train Worker
Setups all the required components to train a model
Inference Worker
Setups all the required components for inference. Also attempts to download model weights if they are not found locally.
Tracker
Slim interface into MLFlow, to set the MLFlow server address set the MLFLOW_TRACKING_URI
environment variable either from the CLI or before importing agoge
.
Examples
CLI
MLFLOW_TRACKING_URI="http://localhost:5001"
Python
from os import environ
environ['MLFLOW_TRACKING_URI'] = 'http://localhost:5001'
import agoge
Data Handler
Loads the dataset and handles the dataset split
User Provided Components
These components need to be inherited by project specific classes
Model
Provides some convenience functions around loading models. This class will hold all model specific code and is used by the train worker and inference workers
Solver
Override the solve
method with the code required to train your model
Dataset
Any dataset that is compatiable with the Pytorch map style dataset model
Disclaimer
This code is subject to change. I will try not to break anything but can't promise. File an issue if an update breaks your code
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.