project descriptions here
Project description
tritonv2
A client library for promote triton official client
Installation
pip install tritonv2==1.3.5.1
Usage
First, you need to create a client object.
from tritonv2.client_factory import TritonClientFactory
server_url = "localhost:8000"
http_client = TritonClientFactory.create_grpc_client(server_url)
or
async with TritonClientFactory.create_http_aio_client(server_url) as http_aio_client
or
grpc_client = TritonClientFactory.create_grpc_client(server_url)
or
async with TritonClientFactory.create_grpc_aio_client(server_url) as grpc_aio_client
In addition, you can set retry for grpc client:
client = TritonClientFactory.create_grpc_client(server_url, num_retries=3,max_interval_secs=20,base_interval_secs=0.3)
for http client we have default setting:
NUMBER_RETRIES = 3
MAX_INTERVAL_SECS = 20
BASE_INTERVAL_SECS = 0.3
Now you can easy use the client to send requests to the server.
for server:
client.server_live()
client.server_ready()
client.server_metadata()
for model:
client.model_metadata(model_name)
client.model_config(model_name)
client.model_ready(model_name)
client.model_statistics(model_name)
for infer:
client.model_infer(model_name, inputs, model_version, outputs)
client.stream_infer(inputs_iterator)
for repository:
client.repository_index()
client.repository_model_load(model_name)
client.repository_model_unload(model_name)
for system shared memory:
client.system_shared_memory_status()
client.system_shared_memory_register()
client.system_shared_memory_unregister()
for cuda shared memory:
client.cuda_shared_memory_status()
client.cuda_shared_memory_register()
client.cuda_shared_memory_unregister()
for trace setting:
client.trace_setting()
client.get_trace_settings()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tritonv2-1.3.5.1.tar.gz
(59.1 kB
view hashes)
Built Distribution
tritonv2-1.3.5.1-py3-none-any.whl
(82.3 kB
view hashes)
Close
Hashes for tritonv2-1.3.5.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 88f1bf7115f76a373afbb43fa398341e32a799e1c7853cc92e269ad80e5ec579 |
|
MD5 | e05c6d57ee64fad9de6f2e87fab9490b |
|
BLAKE2b-256 | 6a1e9297a421b9a74effe719ee07eeaebe51a823f15cb1e4d5eef941b3f703cc |