Monitor ML models in production
Project description
======= Mltrics
.. image:: https://pyup.io/repos/github/deshraj/mltrics/shield.svg :target: https://pyup.io/repos/github/deshraj/mltrics/ :alt: Updates
Monitor ML models in production
Installation
- Install ruby on mac:
brew install ruby
- Check installation using the command:
ruby -v
- Install travis for deployment:
gem install travis --no-document
Features
- TODO
Credits
This package was created with Cookiecutter_ and the audreyr/cookiecutter-pypackage
_ project template.
.. _Cookiecutter: https://github.com/audreyr/cookiecutter
.. _audreyr/cookiecutter-pypackage
: https://github.com/audreyr/cookiecutter-pypackage
Mltrics
Official command line utility to use Mltrics API programmatically.
We help businesses evaluate, compare, and monitor machine learning models in production. Therefore, identify failure cases and take action immidiately.
Installation
Mltrics and its required dependencies can be installed using pip:
pip install mltrics
Usage
Once mltrics package is installed, check out the following usage documentation:
Create model
from mltrics.models import (
create_model,
get_models,
upload_model_predictions,
get_model_predictions,
compare_model_predictions,
)
org = "<your-organization-name>"
model = create_model(model_id="svm", model_name="Support Vector Machine", org=org)
print(model)
Get all models
models = get_models(org=org)
print(models[0])
Upload model predictions
pred_data = [
{
'pred_class': 'dog',
'label_class': 'cat',
'model_id': 'svm',
'image_id': 'img1',
'image_url': 'https://mltrics.s3.us-west-2.amazonaws.com/datasets/cats_vs_dogs/Cat/10896.jpg',
'pred_file': None,
'predictions': {} # store metadata in this dict
},
]
predictions = upload_model_predictions(model_id="svm", org=org, predictions=pred_data)
print(predictions[0])
Get model predictions
predictions = get_model_predictions(model_id="svm", org=org)
print(predictions[0])
Get models prediction comparison programatically
result = compare_model_predictions(model_1="lr1", model_2="rf1", org="mltrics")
print(result[0])
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.