inferless 0.1.5

No project description provided

Python Client to call Inferless API

Installation

$ pip install inferless

Usage

This client can be used to call Inferless API from your python code. It supports both synchronous and asynchronous calls.

Constants

Fetch the URL and API_KEY from the Inferless console https://console-dev.inferless.com/

URL = "<url>"
API_KEY = "<api_key>"
data = { "inputs" : 
          [
              {
                "name": "prompt",
                "shape": [
                  1
                ],
                "datatype": "BYTES",
                "data": [
                  "Once upon a time"
                ]
              }
          ]
        }

Synchrounous call

An example to call Inferless API synchronously

import inferless
import datetime
def main():
    t1 = datetime.datetime.now()
    data = inferless.call(URL, API_KEY, data)
    t2 = datetime.datetime.now()
    print(f"time taken: {t2-t1}")

main()

Output

time taken: 0:00:05.218835

For a particular url, the synchronous call took approximately 5 seconds to complete.

Parallel call with callback

This library also supports parallel calls to Inferless API. The following example shows how to call Inferless API in parallel with a callback function.

import inferless
import datetime

def callback_func(e, response):
    # e is the error object
    # response is the response object
    with open("response.json", "w") as f:
        f.write(json.dumps(response))
    

t1 = datetime.now()
inferless.parallel_call(URL, KEY, DATA, callback_func)
t2 = datetime.now()
print(t2 - t1)

Output

time taken: 0:00:00.000141

It can be seen that the program exits without waiting for the response. The response is written to a file by the callback function after it is received.