Python Client to call Inferless API
Installation
$ pip install inferless
Usage
This client can be used to call Inferless API from your python code. It supports both synchronous and asynchronous calls.
Constants
Fetch the URL and API_KEY from the Inferless console https://console-dev.inferless.com/
URL = "<url>"
API_KEY = "<api_key>"
Input Data
Input data can be provided in two formats, as shown below as inputs and data.
INPUTS = { "inputs" :
[
{
"name": "prompt",
"shape": [
1
],
"datatype": "BYTES",
"data": [
"Once upon a time"
]
}
]
}
This other format is simpler and can be used if other parameters are not required. datatype and shape are automatically inferred.
DATA = {
"prompt": "Once upon a time"
}
Synchrounous call
An example to call Inferless API synchronously
import inferless
import datetime
def main():
t1 = datetime.datetime.now()
data = inferless.call(url=URL, workspace_api_key=API_KEY, inputs=INPUTS)
t2 = datetime.datetime.now()
print(f"time taken: {t2-t1}")
main()
Output
time taken: 0:00:05.218835
For a particular url, the synchronous call took approximately 5 seconds to complete.
Async call with callback
This library also supports async calls to Inferless API. The following example shows how to call Inferless API in background with a callback function.
import inferless
import datetime
def callback_func(e, response):
# e is the error object
# response is the response object
with open("response.json", "w") as f:
f.write(json.dumps(response))
t1 = datetime.now()
inferless.call_async(url=URL, workspace_api_key=KEY, data=data, callback=callback_func)
t2 = datetime.now()
print(t2 - t1)
Output
time taken: 0:00:00.000141
It can be seen that the program continues without waiting for the response. The response is written to a file by the callback function after it is received.
Batch call
If the model is configured for batch processing, is_batch should be set to True
4 maintainers
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file inferless-0.2.15.tar.gz.
File metadata
- Download URL: inferless-0.2.15.tar.gz
- Upload date:
- Size: 9.8 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.6.1 CPython/3.11.9 Darwin/24.5.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
31efbf4b18e332a515a756dcd12034427ecc40793fc1e348cfe5550cf39be844
|
|
| MD5 |
10b4c872bbdb04ee13d36e8699de4edb
|
|
| BLAKE2b-256 |
fc27bff97aa006e09842cd6aed61a24e74d53dceb1ed73d604f003f094cf1bb0
|
File details
Details for the file inferless-0.2.15-py3-none-any.whl.
File metadata
- Download URL: inferless-0.2.15-py3-none-any.whl
- Upload date:
- Size: 11.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
poetry/1.6.1 CPython/3.11.9 Darwin/24.5.0
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
8a5f4a38b7067b70cff460faf63033a62740cba80933af00fbc6decbb7f9304c
|
|
| MD5 |
24d60aef5ef51ab80fb1d9969254a520
|
|
| BLAKE2b-256 |
3c2e612322fa830edcacefc8dbce02fdc758a58c544e4a8477b4ab987d758cc0
|
