No project description provided
Project description
Python Client to call Inferless API
Installation
$ pip install inferless
Usage
This client can be used to call Inferless API from your python code. It supports both synchronous and asynchronous calls.
Constants
Fetch the URL and API_KEY from the Inferless console https://console-dev.inferless.com/
URL = "<url>"
API_KEY = "<api_key>"
Input Data
Input data can be provided in two formats, as shown below as inputs and data.
INPUTS = { "inputs" :
[
{
"name": "prompt",
"shape": [
1
],
"datatype": "BYTES",
"data": [
"Once upon a time"
]
}
]
}
This other format is simpler and can be used if other parameters are not required. datatype
and shape
are automatically inferred.
DATA = {
"prompt": "Once upon a time"
}
Synchrounous call
An example to call Inferless API synchronously
import inferless
import datetime
def main():
t1 = datetime.datetime.now()
data = inferless.call(url=URL, workspace_api_key=API_KEY, inputs=INPUTS)
t2 = datetime.datetime.now()
print(f"time taken: {t2-t1}")
main()
Output
time taken: 0:00:05.218835
For a particular url, the synchronous call took approximately 5 seconds to complete.
Async call with callback
This library also supports async calls to Inferless API. The following example shows how to call Inferless API in background with a callback function.
import inferless
import datetime
def callback_func(e, response):
# e is the error object
# response is the response object
with open("response.json", "w") as f:
f.write(json.dumps(response))
t1 = datetime.now()
inferless.call_async(url=URL, workspace_api_key=KEY, data=data, callback=callback_func)
t2 = datetime.now()
print(t2 - t1)
Output
time taken: 0:00:00.000141
It can be seen that the program continues without waiting for the response. The response is written to a file by the callback function after it is received.
Batch call
If the model is configured for batch processing, is_batch
should be set to True
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for inferless-0.2.0rc1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 43c99767abf057fe9e2ffffd11fa4c66e48e8fea75cf64a7de6beb79158bb253 |
|
MD5 | 4e16373749d843a88e4c8eed5618c9da |
|
BLAKE2b-256 | 7041fff1b446359a2e0218c263e896d3a9d6bb5e7c763c6935b02d57c80269a7 |