waylay-ml-adapter-torch
Provides the ml_adapter.sklearn module as Waylay ML Adapter for pytorch.
Installation
pip install waylay-ml-adapter-torch
The torch module is needed, but NOT declared as a package dependency, to leave the user full control over the way the (rather heavy) pytorch installation is done.
Use pip install waylay-ml-adapter-torch[torch] to include it.
You might want to install additional libraries such as torchaudio or torchvision.
Usage
This ML Adapter uses the standard torch mechanisms to save and load models within a waylay plugin or webscript.
The model_path argument defines the file name of the serialized model in the function archive:
- A
model_pathending inweights.ptorweights.pthsave/loads the model weights using its state_dict. It is a recommended, more robust method, but requires you to also specifiy amodel_class. - Any other
model_pathwith.ptor.pthsuffix save/loads the entire model. It implicitly saves (references to) the used model class. You'll have to make sure that all dependencies used are also included or declared in the archive. - You can also pass an instantiated the
modeldirectly to the adapter.
Creating a model for a webscript
from ml_adapter.torch import V1TorchAdapter
# assuming we save a AutoEncoder torch.nn.Module class in a `autoencoder.py` file
from autoencoder import AutoEncoder
model = AutoEncoder()
# ... train the model ...
# a local directory to prepare the webscript archive
ARCHIVE_LOC='~/webscripts/autoencoder-pytorch'
# use a `weights` model path to use _weights_ serialization
MODEL_PATH='autoencoder.weights.pt'
adapter = V1TorchAdapter(
model=model,
model_path='model-weights.pt',
location=ARCHIVE_LOC,
)
# add our model script to the webscript archive
await adapter.add_script('autoencoder.py')
# write the archive
await adapter.save()
# inspect the archive:
list(adapter.assets)
#> [requirements.txt <ml_adapter.base.assets.python.PythonRequirementsAsset>,
#> main.py <ml_adapter.base.assets.python.PythonScriptAsset>,
#> model-weights.pt <ml_adapter.torch.adapter.TorchModelWeightsAsset>,
#> autoencoder.py <ml_adapter.base.assets.python.PythonScriptAsset>]
Upload the adapter archive as webscript using the ml_tool SDK plugin
from waylay.sdk import WaylayClient
client = WaylayClient.from_profile('staging')
ref = await client.ml_tool.create_webscript(adapter, name='MyAutoEncoder', version='0.0.1')
ref = await client.ml_tool.wait_until_ready(ref)
await client.ml_tool.test_webscript(ref, [2,3,4])
The generated code in main.py uses the following to load your model:
MODEL_PATH = os.environ.get('MODEL_PATH', 'model-weights.pt')
MODEL_CLASS = os.environ.get('MODEL_CLASS', 'autoencoder.AutoEncoder')
adapter = V1TorchAdapter(model_path=MODEL_PATH, model_class=MODEL_CLASS)
You can modify that loading mechanism, e.g. by creating the model your self, and providing it as
adapter = V1TorchAdapter(model=model)
When you want additional processing before or after the model invocation that prevents
your model to be loadable by the default torch.load mechanisms, you can alternatively
use the ml_adapter.torch.V1TorchNoLoadAdapter.
This wrapper model class is then responsible for the loading of the model.
Classes exported by ml_adapter.torch
The module ml_adapter.torch exports the following classes
V1TorchAdapter
ml_adapter.torch.adapter.V1TorchAdapter extending ml_adapter.base.adapter.ModelAdapterBase
Adapts a torch model with torch arrays as input and output.
When initialized with a trained model (using a model parameter):
- will store the model weights as
model_weights.pt(alt: set themodel_pathparameter) - requires that the model class in a library or asset file
(e.g. a class extending
torch.nn.Modulein anmymodel.pyscript asset) The generated server script will use this name as asmodel_class
To load from a serialized model, use the model_path (default model_weights.pt)
and model_class (no default).
Alternatively, when the model_path does not have a weights.pt or weights.pth
extension, the adapter will try to load it as a dill-serialized model.
This is not recommended because of the brittleness of this serialization method
with respect to versions.
V1TorchNoLoadAdapter
ml_adapter.torch.adapter.V1TorchNoLoadAdapter extending ml_adapter.torch.adapter.V1TorchAdapter
Adapts a callable with torch arrays as input and output.
This adapter does not manage the model as a standard asset.
It relies on the model or model_class constructor arguments
to define and load the model.
When model is not provided, any model_path is passed as a constructor
argument to model_class if the signature allows it.
Note that if you internally rely on torch models, the model constructor is responsible for
- setting that wrapped model to evaluation mode
- setting the model to the
correct device and/or dtype
(normally to
"cuda" if torch.cuda.is_available() else "cpu")
The model adapter will still enforce a
torch.no_grad
context around model invocations.
def load_my_model(weights_file='my_weights.pt'):
wrapped_model = AWrappedTorchModel()
wrapped_model.load_state_dict(torch.load(weights_file))
wrapped_model.eval()
wrapped_model.to('cpu')
return wrapped_model
class MyTorchWrappingModel():
def __init__(self, model_file):
self.torch_model = load_my_model(model_file)
# custom pre/postprocessing
def __call__(self, x, y, z):
# preprocess
x = x + y + z
result = this.torch_model(x)
# postprocess
return result[0]
adapter = V1TorchNoLoadAdapter(model_class=MyTorchWrappingModel)
If all you need is add pre- and post-processing of torch tensors,
you can still use V1TorchAdapter to load the wrapped model,
but might want to wrap the __call__ method or set another model_method
class MyTorchModel(torch.nn.Module):
def __init__(self):
super().__init__()
# ... initialize layers
def forward(self, x):
# ... inference
# custom pre/postprocessing
def __call__(self, x, y, z):
# preprocess
x = x + y + z
result = super().__call__(x)
# postprocess
return result[0]
V1TorchMarshaller
ml_adapter.torch.marshall.V1TorchMarshaller extending ml_adapter.base.marshall.v1.base.V1ValueOrDictRequestMarshallerBase, ml_adapter.base.marshall.v1.base.V1ValueOrDictResponseMarshallerBase, ml_adapter.torch.marshall.V1TorchNativeEncoding
Convert v1 payload from and to torch tensors.
See also
- waylay-ml-adapter-sklearn ML adapter for scikit-learn models.
- waylay-ml-adapter-sdk provides the
ml_toolextension to the waylay-sdk - waylay-ml-adapter-base provides the basic ML adapter infrastructure.
- waylay-ml-adapter-api defines the remote data interfaces.
Metadata
Release files for waylay-ml-adapter-torch 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| waylay_ml_adapter_torch-0.1.1.tar.gz | 10.4 kB | Without provenance | Details for waylay_ml_adapter_torch-0.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| waylay_ml_adapter_torch-0.1.1-py3-none-any.whl | Python 3 | none | any | Details for waylay_ml_adapter_torch-0.1.1-py3-none-any.whl |
Total release size: 20.2 kB
Release files / waylay_ml_adapter_torch-0.1.1.tar.gz
| Download URL | waylay_ml_adapter_torch-0.1.1.tar.gz |
|---|---|
| Size | 10.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bfc4453b62482d002acc1a2560d5a03e8335157134d86d19010d6292d1a22e15
|
|
BLAKE2b-256 checksum How to use checksums |
fb251f9e6254dbe41b2466a475b95a60cdb07d26a06dcfd676e98fc90356596d
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.12
|
Release files / waylay_ml_adapter_torch-0.1.1-py3-none-any.whl
| Download URL | waylay_ml_adapter_torch-0.1.1-py3-none-any.whl |
|---|---|
| Size | 9.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
358dac018cead682e45da287ea216768ec8fa5a1cf4c8e7191ec5542179c705a
|
|
BLAKE2b-256 checksum How to use checksums |
c9573c291c6a650c7a802417913d155664795b9c5ba96db1b0400d98ba2b042b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.11.12
|
Security for waylay-ml-adapter-torch 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.