Skip to main content

Machine Learning infra

Project description

Agoge

An opinionated ML development environment

Train pytorch models, hyperparameter tune them with single loc change.

Libraries

Pytorch

Pytorch is a Python first machine learning library

Ray

Ray Provides easy experiment scaling + hyper parameter optimisation

MLFlow

Provides tracking of model metrics and hyperparameters. Also allows for intelligent storing of training artifacts

Static Components

These components should not need to be customised for model specific use cases

Train Worker

Setups all the required components to train a model

Inference Worker

Setups all the required components for inference. Also attempts to download model weights if they are not found locally.

Tracker

Slim interface into MLFlow, to set the MLFlow server address set the MLFLOW_TRACKING_URI environment variable either from the CLI or before importing agoge.

Examples

CLI

MLFLOW_TRACKING_URI="http://localhost:5001"

Python

from os import environ
environ['MLFLOW_TRACKING_URI'] = 'http://localhost:5001'
import agoge

Data Handler

Loads the dataset and handles the dataset split

User Provided Components

These components need to be inherited by project specific classes

Model

Provides some convenience functions around loading models. This class will hold all model specific code and is used by the train worker and inference workers

Solver

Override the solve method with the code required to train your model

Dataset

Any dataset that is compatiable with the Pytorch map style dataset model

Disclaimer

This code is subject to change. I will try not to break anything but can't promise. File an issue if an update breaks your code

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

agoge-0.0.3.tar.gz (6.9 kB view details)

Uploaded Source

File details

Details for the file agoge-0.0.3.tar.gz.

File metadata

  • Download URL: agoge-0.0.3.tar.gz
  • Upload date:
  • Size: 6.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.2.0.post20200511 requests-toolbelt/0.9.1 tqdm/4.46.0 CPython/3.6.10

File hashes

Hashes for agoge-0.0.3.tar.gz
Algorithm Hash digest
SHA256 f88245d72b54d2d1a71829bb7211412aed6083b4d6e5f6106babe3bd2f5faafe
MD5 2e5289ec9dc4e3d8b87af4e331d4ad8e
BLAKE2b-256 3c50185ab5fa8d5d53bfaeb0675d9ab46453834064143617ffe78a1be4ab4e19

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page