Skip to main content

Host your deep learning models easily.

Project description

Ventu

Help you host your deep learning models easily.

Features

  • Only need to implement Model(inference, preprocess, postprocess)
  • request & response data check using pydantic
  • API document using SpecTree
  • health check

Example

source code can be found in demo.py

from ventu import VentuModel, VentuService
from typing import Tuple
from pydantic import BaseModel
import numpy
import onnxruntime


# define the input schema
class Input(BaseModel):
    x: Tuple[(str,) * 3]


# define the output schema
class Output(BaseModel):
    label: Tuple[(bool,) * 3]


class CustomModel(VentuModel):
    def __init__(self, *args, **kwargs):
        super().__init__()
        # load model
        self.sess = onnxruntime.InferenceSession('./sigmoid.onnx')
        self.input_name = self.sess.get_inputs()[0].name
        self.output_name = self.sess.get_outputs()[0].name

    def preprocess(self, data: Input):
        # data format is defined in ``Input``
        words = [sent.split(' ')[:4] for sent in data.x]
        # padding
        words = [word + [''] * (4 - len(word)) for word in words]
        # build embedding
        emb = [[
            numpy.random.random(5) if w else [0] * 5
            for w in word]
            for word in words]
        return numpy.array(emb, dtype=numpy.float32)

    def inference(self, data):
        # model inference
        return self.sess.run([self.output_name], {self.input_name: data})[0]

    def postprocess(self, data) -> Output:
        # generate the same format as defined in ``Output``
        return {'label': [bool(numpy.mean(d) > 0.5) for d in data]}


if __name__ == "__main__":
    service = VentuService(CustomModel, Input, Output)
    service.run(host='localhost', port=8000)

Run with Gunicorn

gunicorn -w 2 service.app

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ventu, version 0.1.0
Filename, size File type Python version Upload date Hashes
Filename, size ventu-0.1.0-py3-none-any.whl (4.6 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size ventu-0.1.0.tar.gz (3.8 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page