chariots 0.2.4

machine learning pipelines

chariots

chariots aims to be a complete framework to build and deploy versioned machine learning pipelines.

• Documentation: https://chariots.readthedocs.io.

Getting Started: 30 seconds to Chariots:

You can check the chariots docutemtation for a complete tutorial on getting started with chariots, but here are the essentials:

you can create operations to execute steps in your pipeline:

>>> from chariots.sklearn import SKUnsupervisedOp, SKSupervisedOp
>>> from chariots.versioning import VersionType, VersionedFieldDict, VersionedField
>>> from sklearn.decomposition import PCA
>>> from sklearn.linear_model import LogisticRegression
...
...
>>> class PCAOp(SKUnsupervisedOp):
...     training_update_version = VersionType.MAJOR
...     model_parameters = VersionedFieldDict(VersionType.MAJOR, {"n_components": 2})
...     model_class = VersionedField(PCA, VersionType.MAJOR)
...
>>> class LogisticOp(SKSupervisedOp):
...     training_update_version = VersionType.PATCH
...     model_class = LogisticRegression

Once your ops are created, you can create your various training and prediction pipelines:

>>> from chariots import Pipeline, MLMode
>>> from chariots.nodes import Node
...
...
>>> train = Pipeline([
...     Node(IrisFullDataSet(), output_nodes=["x", "y"]),
...     Node(PCAOp(MLMode.FIT_PREDICT), input_nodes=["x"], output_nodes="x_transformed"),
...     Node(LogisticOp(MLMode.FIT), input_nodes=["x_transformed", "y"])
... ], 'train')
...
>>> pred = Pipeline([
...     Node(PCAOp(MLMode.PREDICT), input_nodes=["__pipeline_input__"], output_nodes="x_transformed"),
...     Node(LogisticOp(MLMode.PREDICT), input_nodes=["x_transformed"], output_nodes=['__pipeline_output__'])
... ], 'pred')

Once all your pipelines have been created, deploying them is as easy as creating a creating a Chariots object:

>>> from chariots import Chariots
...
...
>>> app = Chariots([train, pred], app_path, import_name='iris_app')

The Chariots class inherits from the Flask class so you can deploy this the same way you would any flask application

Once this the server is started, you can use the chariots client to query your machine learning micro-service from python:

>>> from chariots import Client
...
...
>>> client = Client()

with this client we will be

• training the models

• saving them and reloading the prediction pipeline (so that it uses the latest/trained version of our models)

• query some prediction

  >>> client.call_pipeline(train)
  >>> client.save_pipeline(train)
  >>> client.load_pipeline(pred)
  >>> client.call_pipeline(pred, [[1, 2, 3, 4]])
  [1]
  

Features

• versionable individual op

• easy pipeline building

• easy pipelines deployment

• ML utils (implementation of ops for most popular ML libraries with adequate Versionedfield) for sklearn and keras at first

• A CookieCutter template to properly structure your Chariots project

Comming Soon

Some key features of Chariot are still in development and should be coming soon:

• Cloud integration (integration with cloud services to fetch and load models from)

• Graphql API to store and load information on different ops and pipelines (performance monitoring, …)

• ABTesting

Credits

This package was created with Cookiecutter and the audreyr/cookiecutter-pypackage project template. audreyr/cookiecutter-pypackage’s project is also the basis of the Chariiots project template

History

0.1.0 (2019-06-15)

• First release on PyPI.

0.2.0 (2019-06-15)

• sci-kit learn and keras integration

• multiple outputs per nodes

• project template

• tutorials