Inference service package for IAPARC
Project description
iaparc_inference
The IA Parc inference plugin allows developers to easily integrate their inference pipeline into IA Parc's production module.
Installation
pip install iaparc-inference
Usage
-
If your inference pipeline support batching:
from iaparc_inference import IAPListener # Define a callback to query your inference pipeline # To load your model only once it is recommended to use a class: class MyModel: def __init__(self, model_path: str): ## Load your model in pytorch, tensorflow or any other backend def batch_query(batch: list) -> list: ## execute your pipeline on a batch input if __name__ == '__main__': # Initiate your model class my_model = MyModel("path/to/my/model") # Initiate IAParc listener listener = IAPListener(my_model.batch_query) # Start the listener listener.run()
-
If your inference pipeline do not support batching:
from iaparc_inference import IAPListener # Define a callback to query your inference pipeline # To load your model only once it is recommended to use a class: class MyModel: def __init__(self, model_path: str): ## Load your model in pytorch, tensorflow or any other backend def single_query(one_input): ## execute your pipeline on a single input if __name__ == '__main__': # Initiate your model class my_model = MyModel("path/to/my/model") # Initiate IAParc listener listener = IAPListener(my_model.single_query, batch=1) # Note that batch size is forced to 1 here # Start the listener listener.run()
Features
- Dynamic batching
- Autoscalling
- Support both synchronous and asynchronous queries
- Data agnostic
License
This project is licensed under the Apache License Version 2.0 - see the Apache LICENSE file for details.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
iaparc_inference-0.0.10.tar.gz
(11.1 kB
view hashes)
Built Distribution
Close
Hashes for iaparc_inference-0.0.10-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 29b81882af6475dccdc1a9468d43e57e5e5a75e474811fc167f2a2700a8ad77b |
|
MD5 | a1f6dafdda038204e2e6a4269ec15f73 |
|
BLAKE2b-256 | d371d22201618e1b35736b7bcadb9c4ad89930e18af9a1e0c084fc345827f957 |