Transportation of ML models
Project description
Overview
PyMilo is an open source Python package that provides a simple, efficient, and safe way for users to export pre-trained machine learning models in a transparent way. By this, the exported model can be used in other environments, transferred across different platforms, and shared with others. PyMilo allows the users to export the models that are trained using popular Python libraries like scikit-learn, and then use them in deployment environments, or share them without exposing the underlying code or dependencies. The transparency of the exported models ensures reliability and safety for the end users, as it eliminates the risks of binary or pickle formats.
PyPI Counter |
|
Github Stars |
|
Branch | main | dev |
CI |
|
|
Code Quality |
Installation
PyPI
- Check Python Packaging User Guide
- Run
pip install pymilo==0.9
Source code
- Download Version 0.9 or Latest Source
- Run
pip install .
Conda
- Check Conda Managing Package
- Update Conda using
conda update conda
- Run
conda install -c openscilab pymilo
Usage
Imagine you want to train a LinearRegression
model representing this equation: $y = x_0 + 2x_1 + 3$. You will create data points (X
, y
) and train your model as follows.
>>> import numpy as np
>>> from sklearn.linear_model import LinearRegression
>>> X = np.array([[1, 1], [1, 2], [2, 2], [2, 3]])
>>> y = np.dot(X, np.array([1, 2])) + 3
# y = 1 * x_0 + 2 * x_1 + 3
>>> model = LinearRegression().fit(X, y)
>>> pred = model.predict(np.array([[3, 5]]))
# pred = [16.] (=1 * 3 + 2 * 5 + 3)
Using PyMilo Export
class you can easily serialize and export your trained model into a JSON file.
>>> from pymilo import Export
>>> Export(model).save("model.json")
You can check out your model as a JSON file now.
{
"data": {
"fit_intercept": true,
"copy_X": true,
"n_jobs": null,
"positive": false,
"n_features_in_": 2,
"coef_": {
"pymiloed-ndarray-list": [
1.0000000000000002,
1.9999999999999991
],
"pymiloed-ndarray-dtype": "float64",
"pymiloed-ndarray-shape": [
2
],
"pymiloed-data-structure": "numpy.ndarray"
},
"rank_": 2,
"singular_": {
"pymiloed-ndarray-list": [
1.618033988749895,
0.6180339887498948
],
"pymiloed-ndarray-dtype": "float64",
"pymiloed-ndarray-shape": [
2
],
"pymiloed-data-structure": "numpy.ndarray"
},
"intercept_": {
"value": 3.0000000000000018,
"np-type": "numpy.float64"
}
},
"sklearn_version": "1.4.2",
"pymilo_version": "0.8",
"model_type": "LinearRegression"
}
You can see all the learned parameters of the model in this file and change them if you want. This JSON representation is a transparent version of your model.
Now let's load it back. You can do it easily by using PyMilo Import
class.
>>> from pymilo import Import
>>> model = Import("model.json").to_model()
>>> pred = model.predict(np.array([[3, 5]]))
# pred = [16.] (=1 * 3 + 2 * 5 + 3)
This loaded model is exactly the same as the original trained model.
Supported ML models
scikit-learn | PyTorch |
---|---|
Linear Models ✅ | - |
Neural networks ✅ | - |
Trees ✅ | - |
Clustering ✅ | - |
Naïve Bayes ✅ | - |
Support vector machines (SVMs) ✅ | - |
Nearest Neighbors ✅ | - |
Ensemble Models ✅ | - |
Pipeline Model ✅ | - |
Preprocessing Models ✅ | - |
Details are available in Supported Models.
Issues & bug reports
Just fill an issue and describe it. We'll check it ASAP! or send an email to pymilo@openscilab.com.
- Please complete the issue template
You can also join our discord server
Show your support
Star this repo
Give a ⭐️ if this project helped you!
Donate to our project
If you do like our project and we hope that you do, can you please support us? Our project is not and is never going to be working for profit. We need the money just so we can continue doing what we do ;-) .
Changelog
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
Unreleased
Added
Changed
0.9 - 2024-07-01
Added
- Anaconda workflow
prefix_list
function inutils.util.py
KBinsDiscretizer
preprocessing modelPowerTransformer
preprocessing modelSplineTransformer
preprocessing modelTargetEncoder
preprocessing modelQuantileTransformer
preprocessing modelRobustScaler
preprocessing modelPolynomialFeatures
preprocessing modelOrdinalEncoder
preprocessing modelNormalizer
preprocessing modelMaxAbsScaler
preprocessing modelMultiLabelBinarizer
preprocessing modelKernelCenterer
preprocessing modelFunctionTransformer
preprocessing modelBinarizer
preprocessing model- Preprocessing models test runner
Changed
Command
enum class intransporter.py
SerializationErrorTypes
enum class inserialize_exception.py
DeserializationErrorTypes
enum class indeserialize_exception.py
meta.yaml
modifiedNaN
type inpymilo_param
NaN
type transportation inGeneralDataStructureTransporter
TransporterBSpline
Transportation inPreprocessingTransporter
Transporter- one layer deeper transportation in
PreprocessingTransporter
Transporter - dictating outer ndarray dtype in
GeneralDataStructureTransporter
Transporter - preprocessing params fulfilled in
pymilo_param
SUPPORTED_MODELS.md
updatedREADME.md
updatedserialize_possible_ml_model
in the Ensemble chain
0.8 - 2024-05-06
Added
StandardScaler
Transformer inpymilo_param.py
PreprocessingTransporter
Transporter- ndarray shape config in
GeneralDataStructure
Transporter util.py
in chainsBinMapperTransporter
TransporterBunchTransporter
TransporterGeneratorTransporter
TransporterTreePredictorTransporter
TransporterAdaboostClassifier
modelAdaboostRegressor
modelBaggingClassifier
modelBaggingRegressor
modelExtraTreesClassifier
modelExtraTreesRegressor
modelGradientBoosterClassifier
modelGradientBoosterRegressor
modelHistGradientBoosterClassifier
modelHistGradientBoosterRegressor
modelRandomForestClassifier
modelRandomForestRegressor
modelIsolationForest
modelRandomTreesEmbedding
modelStackingClassifier
modelStackingRegressor
modelVotingClassifier
modelVotingRegressor
modelPipeline
model- Ensemble models test runner
- Ensemble chain
SECURITY.md
Changed
Pipeline
test updatedLabelBinarizer
,LabelEncoder
andOneHotEncoder
got embedded inPreprocessingTransporter
- Preprocessing support added to Ensemble chain
- Preprocessing params initialized in
pymilo_param
util.py
in utils updatedtest_pymilo.py
updatedpymilo_func.py
updatedlinear_model_chain.py
updatedneural_network_chain.py
updateddecision_tree_chain.py
updatedclustering_chain.py
updatednaive_bayes_chain.py
updatedneighbours_chain.py
updatedsvm_chain.py
updatedGeneralDataStructure
Transporter updatedLossFunction
Transporter updatedAbstractTransporter
updated- Tests config modified
- Unequal sklearn version error added in
pymilo_param.py
- Ensemble params initialized in
pymilo_param
- Ensemble support added to
pymilo_func.py
SUPPORTED_MODELS.md
updatedREADME.md
updated
0.7 - 2024-04-03
Added
pymilo_nearest_neighbor_test
function added totest_pymilo.py
NeighborsTreeTransporter
TransporterLocalOutlierFactor
modelRadiusNeighborsClassifier
modelRadiusNeighborsRegressor
modelNearestCentroid
modelNearestNeighbors
modelKNeighborsClassifier
modelKNeighborsRegressor
model- Neighbors models test runner
- Neighbors chain
Changed
- Tests config modified
- Neighbors params initialized in
pymilo_param
- Neighbors support added to
pymilo_func.py
SUPPORTED_MODELS.md
updatedREADME.md
updated
0.6 - 2024-03-27
Added
deserialize_primitive_type
function inGeneralDataStructureTransporter
is_deserialized_ndarray
function inGeneralDataStructureTransporter
deep_deserialize_ndarray
function inGeneralDataStructureTransporter
deep_serialize_ndarray
function inGeneralDataStructureTransporter
SVR
modelSVC
modelOne Class SVM
modelNuSVR
modelNuSVC
modelLinear SVR
modelLinear SVC
model- SVM models test runner
- SVM chain
Changed
pymilo_param.py
updatedpymilo_obj.py
updated to use predefined stringsTreeTransporter
updatedget_homogeneous_type
function inutil.py
updatedGeneralDataStructureTransporter
updated to use deep ndarray serializer & deserializercheck_str_in_iterable
updatedLabel Binarizer
Transporter updatedFunction
Transporter updatedCFNode
Transporter updatedBisecting Tree
Transporter updated- Tests config modified
- SVM params initialized in
pymilo_param
- SVM support added to
pymilo_func.py
SUPPORTED_MODELS.md
updatedREADME.md
updated
0.5 - 2024-01-31
Added
reset
function in theTransport
interfacereset
function implementation inAbstractTransporter
Gaussian Naive Bayes
declared asGaussianNB
modelMultinomial Naive Bayes
model declared asMultinomialNB
modelComplement Naive Bayes
model declared asComplementNB
modelBernoulli Naive Bayes
model declared asBernoulliNB
modelCategorical Naive Bayes
model declared asCategoricalNB
model- Naive Bayes models test runner
- Naive Bayes chain
Changed
Transport
function ofAbstractTransporter
updated- fix the order of
CFNode
fields serialization inCFNodeTransporter
GeneralDataStructureTransporter
support list of ndarray with different shapes- Tests config modified
- Naive Bayes params initialized in
pymilo_param
- Naive Bayes support added to
pymilo_func.py
SUPPORTED_MODELS.md
updatedREADME.md
updated
0.4 - 2024-01-22
Added
has_named_parameter
method inutil.py
CFSubcluster
Transporter(insideCFNode
Transporter)CFNode
TransporterBirch
modelSpectralBiclustering
modelSpectralCoclustering
modelMiniBatchKMeans
modelfeature_request.yml
templateconfig.yml
for issue templateBayesianGaussianMixture
modelserialize_tuple
method inGeneralDataStructureTransporter
import_function
method inutil.py
Function
TransporterFeatureAgglomeration
modelHDBSCAN
modelGaussianMixture
modelOPTICS
modelDBSCAN
modelAgglomerativeClustering
modelSpectralClustering
modelMeanShift
modelAffinityPropagation
modelKmeans
model- Clustering models test runner
- Clustering chain
Changed
LossFunctionTransporter
enhanced to handle scikit 1.4.0_loss_function_
field- Codacy Static Code Analyzer's suggestions applied
- Spectral Clustering test folder refactored
- Bug report template modified
GeneralDataStructureTransporter
updated- Tests config modified
- Clustering data set preparation added to
data_exporter.py
- Clustering params initialized in
pymilo_param
- Clustering support added to
pymilo_func.py
Python 3.12
added totest.yml
dev-requirements.txt
updated- Code quality badges added to
README.md
SUPPORTED_MODELS.md
updatedREADME.md
updated
0.3 - 2023-09-27
Added
- scikit-learn decision tree models
ExtraTreeClassifier
modelExtraTreeRegressor
modelDecisionTreeClassifier
modelDecisionTreeRegressor
modelTree
Transporter- Decision Tree chain
Changed
- Tests config modified
- DecisionTree params initialized in
pymilo_param
- Decision Tree support added to
pymilo_func.py
0.2 - 2023-08-02
Added
- scikit-learn neural network models
MLP Regressor
modelMLP Classifier
modelBernoulliRBN
modelSGDOptimizer
transporterRandomState(MT19937)
transporterAdamoptimizer
transporter- Neural Network chain
- Neural Network exceptions
ndarray_to_list
method inGeneralDataStructureTransporter
list_to_ndarray
method inGeneralDataStructureTransporter
neural_network_chain.py
chain
Changed
GeneralDataStructure
Transporter updatedLabelBinerizer
Transporter updatedlinear model
chain updated- GeneralDataStructure transporter enhanced
- LabelBinerizer transporter updated
- transporters' chain router added to
pymilo func
- NeuralNetwork params initialized in
pymilo_param
pymilo_test
updated to support multiple modelslinear_model_chain
refactored
0.1 - 2023-06-29
Added
- scikit-learn linear models support
Export
classImport
class
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.