Transportation of ML models
Project description
Overview
PyMilo is an open source Python package that provides a simple, efficient, and safe way for users to export pre-trained machine learning models in a transparent way. By this, the exported model can be used in other environments, transferred across different platforms, and shared with others. PyMilo allows the users to export the models that are trained using popular Python libraries like scikit-learn, and then use them in deployment environments, or share them without exposing the underlying code or dependencies. The transparency of the exported models ensures reliability and safety for the end users, as it eliminates the risks of binary or pickle formats.
PyPI Counter |
|
Github Stars |
|
Branch | main | dev |
CI |
|
|
Code Quality |
Installation
PyPI
- Check Python Packaging User Guide
- Run
pip install pymilo==0.8
Source code
- Download Version 0.8 or Latest Source
- Run
pip install .
Usage
Model preparation
>>> from sklearn import datasets
>>> from pymilo import Export, Import
>>> from sklearn.linear_model import LinearRegression
>>> import os
>>> X, Y = datasets.load_diabetes(return_X_y=True)
>>> threshold = 20
>>> X_train, X_test = X[:-threshold], X[-threshold:]
>>> Y_train, Y_test = Y[:-threshold], Y[-threshold:]
>>> model = LinearRegression()
>>> #### Train the model using the training sets
>>> model.fit(X_train, Y_train)
Save model
>>> #### Export the fitted model to a transparent json file
>>> exported_model = Export(model)
>>> PATH_TO_JSON_FILE = os.path.join(os.getcwd(),"test.json")
>>> exported_model.save(PATH_TO_JSON_FILE)
Load model
>>> #### Import the pymilo-exported model and get a real scikit model
>>> imported_model = Import(PATH_TO_JSON_FILE)
Get the associated model
>>> imported_sklearn_model = imported_model.to_model()
- Note:
imported_sklearn_model
has the exact same functionality as themodel
object earlier.
Supported ML models
scikit-learn | PyTorch |
---|---|
Linear Models ✅ | - |
Neural networks ✅ | - |
Trees ✅ | - |
Clustering ✅ | - |
Naïve Bayes ✅ | - |
Support vector machines (SVMs) ✅ | - |
Nearest Neighbors ✅ | - |
Ensemble Models ✅ | - |
Pipeline Model ✅ | - |
Preprocessing Models ⏳ | - |
Details are available in Supported Models.
Issues & bug reports
Just fill an issue and describe it. We'll check it ASAP! or send an email to pymilo@openscilab.com.
- Please complete the issue template
You can also join our discord server
Show your support
Star this repo
Give a ⭐️ if this project helped you!
Donate to our project
If you do like our project and we hope that you do, can you please support us? Our project is not and is never going to be working for profit. We need the money just so we can continue doing what we do ;-) .
Changelog
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
Unreleased
Added
Changed
0.8 - 2024-05-06
Added
StandardScaler
Transformer inpymilo_param.py
PreprocessingTransporter
Transporter- ndarray shape config in
GeneralDataStructure
Transporter util.py
in chainsBinMapperTransporter
TransporterBunchTransporter
TransporterGeneratorTransporter
TransporterTreePredictorTransporter
TransporterAdaboostClassifier
modelAdaboostRegressor
modelBaggingClassifier
modelBaggingRegressor
modelExtraTreesClassifier
modelExtraTreesRegressor
modelGradientBoosterClassifier
modelGradientBoosterRegressor
modelHistGradientBoosterClassifier
modelHistGradientBoosterRegressor
modelRandomForestClassifier
modelRandomForestRegressor
modelIsolationForest
modelRandomTreesEmbedding
modelStackingClassifier
modelStackingRegressor
modelVotingClassifier
modelVotingRegressor
modelPipeline
model- Ensemble models test runner
- Ensemble chain
SECURITY.md
Changed
Pipeline
test updatedLabelBinarizer
,LabelEncoder
andOneHotEncoder
got embedded inPreprocessingTransporter
- Preprocessing support added to Ensemble chain
- Preprocessing params initialized in
pymilo_param
util.py
in utils updatedtest_pymilo.py
updatedpymilo_func.py
updatedlinear_model_chain.py
updatedneural_network_chain.py
updateddecision_tree_chain.py
updatedclustering_chain.py
updatednaive_bayes_chain.py
updatedneighbours_chain.py
updatedsvm_chain.py
updatedGeneralDataStructure
Transporter updatedLossFunction
Transporter updatedAbstractTransporter
updated- Tests config modified
- Unequal sklearn version error added in
pymilo_param.py
- Ensemble params initialized in
pymilo_param
- Ensemble support added to
pymilo_func.py
SUPPORTED_MODELS.md
updatedREADME.md
updated
0.7 - 2024-04-03
Added
pymilo_nearest_neighbor_test
function added totest_pymilo.py
NeighborsTreeTransporter
TransporterLocalOutlierFactor
modelRadiusNeighborsClassifier
modelRadiusNeighborsRegressor
modelNearestCentroid
modelNearestNeighbors
modelKNeighborsClassifier
modelKNeighborsRegressor
model- Neighbors models test runner
- Neighbors chain
Changed
- Tests config modified
- Neighbors params initialized in
pymilo_param
- Neighbors support added to
pymilo_func.py
SUPPORTED_MODELS.md
updatedREADME.md
updated
0.6 - 2024-03-27
Added
deserialize_primitive_type
function inGeneralDataStructureTransporter
is_deserialized_ndarray
function inGeneralDataStructureTransporter
deep_deserialize_ndarray
function inGeneralDataStructureTransporter
deep_serialize_ndarray
function inGeneralDataStructureTransporter
SVR
modelSVC
modelOne Class SVM
modelNuSVR
modelNuSVC
modelLinear SVR
modelLinear SVC
model- SVM models test runner
- SVM chain
Changed
pymilo_param.py
updatedpymilo_obj.py
updated to use predefined stringsTreeTransporter
updatedget_homogeneous_type
function inutil.py
updatedGeneralDataStructureTransporter
updated to use deep ndarray serializer & deserializercheck_str_in_iterable
updatedLabel Binarizer
Transporter updatedFunction
Transporter updatedCFNode
Transporter updatedBisecting Tree
Transporter updated- Tests config modified
- SVM params initialized in
pymilo_param
- SVM support added to
pymilo_func.py
SUPPORTED_MODELS.md
updatedREADME.md
updated
0.5 - 2024-01-31
Added
reset
function in theTransport
interfacereset
function implementation inAbstractTransporter
Gaussian Naive Bayes
declared asGaussianNB
modelMultinomial Naive Bayes
model declared asMultinomialNB
modelComplement Naive Bayes
model declared asComplementNB
modelBernoulli Naive Bayes
model declared asBernoulliNB
modelCategorical Naive Bayes
model declared asCategoricalNB
model- Naive Bayes models test runner
- Naive Bayes chain
Changed
Transport
function ofAbstractTransporter
updated- fix the order of
CFNode
fields serialization inCFNodeTransporter
GeneralDataStructureTransporter
support list of ndarray with different shapes- Tests config modified
- Naive Bayes params initialized in
pymilo_param
- Naive Bayes support added to
pymilo_func.py
SUPPORTED_MODELS.md
updatedREADME.md
updated
0.4 - 2024-01-22
Added
has_named_parameter
method inutil.py
CFSubcluster
Transporter(insideCFNode
Transporter)CFNode
TransporterBirch
modelSpectralBiclustering
modelSpectralCoclustering
modelMiniBatchKMeans
modelfeature_request.yml
templateconfig.yml
for issue templateBayesianGaussianMixture
modelserialize_tuple
method inGeneralDataStructureTransporter
import_function
method inutil.py
Function
TransporterFeatureAgglomeration
modelHDBSCAN
modelGaussianMixture
modelOPTICS
modelDBSCAN
modelAgglomerativeClustering
modelSpectralClustering
modelMeanShift
modelAffinityPropagation
modelKmeans
model- Clustering models test runner
- Clustering chain
Changed
LossFunctionTransporter
enhanced to handle scikit 1.4.0_loss_function_
field- Codacy Static Code Analyzer's suggestions applied
- Spectral Clustering test folder refactored
- Bug report template modified
GeneralDataStructureTransporter
updated- Tests config modified
- Clustering data set preparation added to
data_exporter.py
- Clustering params initialized in
pymilo_param
- Clustering support added to
pymilo_func.py
Python 3.12
added totest.yml
dev-requirements.txt
updated- Code quality badges added to
README.md
SUPPORTED_MODELS.md
updatedREADME.md
updated
0.3 - 2023-09-27
Added
- scikit-learn decision tree models
ExtraTreeClassifier
modelExtraTreeRegressor
modelDecisionTreeClassifier
modelDecisionTreeRegressor
modelTree
Transporter- Decision Tree chain
Changed
- Tests config modified
- DecisionTree params initialized in
pymilo_param
- Decision Tree support added to
pymilo_func.py
0.2 - 2023-08-02
Added
- scikit-learn neural network models
MLP Regressor
modelMLP Classifier
modelBernoulliRBN
modelSGDOptimizer
transporterRandomState(MT19937)
transporterAdamoptimizer
transporter- Neural Network chain
- Neural Network exceptions
ndarray_to_list
method inGeneralDataStructureTransporter
list_to_ndarray
method inGeneralDataStructureTransporter
neural_network_chain.py
chain
Changed
GeneralDataStructure
Transporter updatedLabelBinerizer
Transporter updatedlinear model
chain updated- GeneralDataStructure transporter enhanced
- LabelBinerizer transporter updated
- transporters' chain router added to
pymilo func
- NeuralNetwork params initialized in
pymilo_param
pymilo_test
updated to support multiple modelslinear_model_chain
refactored
0.1 - 2023-06-29
Added
- scikit-learn linear models support
Export
classImport
class
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.