Skip to main content

A framework for developing a realtime model-inference service.

Project description

Ganesha

A framework for developing a realtime model-inference service.

Allows you to set up an inference-endpoint for you ML Model easily.

Table of Contents:

  1. Installation
  2. Basic Setup
  3. Your First Endpoint
  4. Models
  5. Logging
  6. CLI Commands

Installation

  • Run: pip install ganesha

Basic Setup

from ganesha.app import Ganesha, GaneshaConfig
from ganesha.app.api import generate_api_v1

# Initialize configuration
conf = GaneshaConfig(service_name='sample_ganesha',
                           listen_port=1234)

# Setup an application
ganesha_app = Ganesha(conf).setup()

# Get pre-made ping/health methods for free!
api_v1 = generate_api_v1()
ganesha_app.register_router(url='/api/v1',
                           router=api_v1)

# Run development server
ganesha_app.run()

Your First Endpoint

When using ganesha, it is easy to set up new endpoints.

The most basic layer is a Router, once you add your routes to your router, you can register it to your application:

import json
from ganesha.app.api import Router

router = Router('v1')

@router.route(url='hello_world', method='GET')
def hello_world():
    return json.dumps({"hello": "world"})

ganesha_app.register_router(url='/api/v1',
                           router=router)

When using the decorator @router.route(...) you can add a new route to your router By using register_router, you register the router under the specified url.

Request Parameters

ganesha gives you an easy way to extract and validate your request-parameters:

import json
from ganesha.app.api import Router, validate_params

router = Router('v1')

# Define a schema for the request body
schema = {'vector': {'type': 'list', 'schema': {'type': 'float'}}}

@router.route(url='predict', method='POST')
@validate_params(schema)
def predict(vector):
    prediction = my_model.predict_proba(vector)
    return json.dumps({"hello": "world"})

ganesha_app.register_router(url='/api/v1',
                           router=router)

ganesha uses Cerberus for schema validation, see https://docs.python-cerberus.org/en/stable/validation-rules.html for schema syntax

Pre-Made API

Most of the time, your aplication may want a ping+health route, ganesha gives that for free:

from ganesha.app.api import generate_api_v1, validate_params

api_v1 = generate_api_v1() # Returns a pre-made Router with /ping and /health

What's left? just register your own route in:

@api_v1.route(url='predict', method='POST')
@validate_params(schema)
def predict(**params):
    # code

ganesha_app.register_router(url='/api/v1',
                           router=api_v1)

Models

Implementing your inference business-logic is easy

For convinience reasons, you can inherit from GaneshaModel class and override specifc methods:

from ganesha.models import GenericModel

class SampleModel(GenericModel):
    def init(self, directory_path):
        load_path = os.path.join(directory_path, 'sample_model.pkl')
        with open(load_path, 'r') as f:
            # self.model should be defined here
            self.model = pickle.load(f)

    def infer(self, ...):
        # Implement
        pass

    # You can add as many methods as you wish, like pre-procssing methods & infer method...

ganesha will automatically init your model instance upon calling ganesha_app.setup()

# Build a Ganesha Model instance
my_model = MyModel()

conf = GaneshaConfig(service_name='sample_ganesha',
                           listen_port=1234,
                           # Make ganesha aware of your instance
                           models_instances=[my_model])

# Setup an application
# Ganesha will call `init` on your instance -> your model will be loaded here
ganesha_app = Ganesha(conf).setup()

# Get pre-made ping/health methods for free!
api_v1 = generate_api_v1()

# Add your prediction route
schema = {...}

@api_v1.route(url='predict', method='POST')
@validate_params(schema)
def predict(**params):
    probability = my_model.predict(**params)

    return json.dumps({'probability': probability})

ganesha_app.register_router(url='/api/v1',
                           router=api_v1)

Logging

You can access the logger in 2 way, One way - you can get the logger with: logging.getLogger(service_name) Another way - the logger is a member of your app: ganesha_app.logger After you create your app, automatically a new logger is defined (named after your service):

# Initialize configuration
conf = GaneshaConfig(service_name='sample_ganesha',
                           listen_port=1234)

# Setup an application
ganesha_app = Ganesha(conf).setup()

# 1st way:
import logging
logger = logging.getLogger('sample_ganesha')
logger.info('Message 1')

# 2nd way:
ganesha_app.logger.info('Message 2')

CLI Commands

When installing ganesha, you get for free the CLI command ganesha This command has utilities for both development and production.

Available commands:

  • ganesha test
  • ganesha shell
  • ganesha run dev/gunicorn

The ganesha commands looks for your application in app.py by default, you can modify it by setting the environement variable: GANESHA_APP

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for ganesha, version 0.1.0
Filename, size File type Python version Upload date Hashes
Filename, size ganesha-0.1.0-py3-none-any.whl (24.1 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size ganesha-0.1.0.tar.gz (16.8 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page