iaparc-inference 0.0.2

Inference service package for IAPARC

iaparc_inference

The IAParc inference plugin allows developers to easily integrate their inference pipeline into the IAParc production module.

Installation

pip install iaparc-inference

Usage

• If your inference pipeline support batching:

  from iaparc_inference import IAPListener
  
  # Define a callback to query your inference pipeline
  # To load your model only once it is recommended to use a class:
  class MyModel:
      def __init__(self, model_path: str):
          ## Load your model in pytorch, tensorflow or any other backend
      
      def batch_query(batch: list) -> list:
          ## execute your pipeline on a batch input
  
  if __name__ == '__main__':
      # Initiate your model class
      my_model = MyModel("path/to/my/model")
  
      # Initiate IAParc listener
      listener = IAPListener(my_model.batch_query)
      # Start the listener
      listener.run()
  

• If your inference pipeline do not support batching:

  from iaparc_inference import IAPListener
  
  # Define a callback to query your inference pipeline
  # To load your model only once it is recommended to use a class:
  class MyModel:
      def __init__(self, model_path: str):
          ## Load your model in pytorch, tensorflow or any other backend
      
      def single_query(one_input):
          ## execute your pipeline on a single input
  
  if __name__ == '__main__':
      # Initiate your model class
      my_model = MyModel("path/to/my/model")
  
      # Initiate IAParc listener
      listener = IAPListener(my_model.single_query, batch=1)
      # Start the listener
      listener.run()
  

Features

• Dynamic batching
• Autoscalling
• Support both synchronous and asynchronous queries
• Data agnostic

License

This project is licensed under the Apache License Version 2.0 - see the Apache LICENSE file for details.