project descriptions here
Project description
tritonv2
A client library for promote triton official client
Installation
pip install tritonv2==1.3.7.dev9
Usage
First, you need to create a client object.
from tritonv2.client_factory import TritonClientFactory
server_url = "localhost:8000"
http_client = TritonClientFactory.create_grpc_client(server_url)
or
async with TritonClientFactory.create_http_aio_client(server_url) as http_aio_client
or
grpc_client = TritonClientFactory.create_grpc_client(server_url)
or
async with TritonClientFactory.create_grpc_aio_client(server_url) as grpc_aio_client
In addition, you can set retry for grpc client:
client = TritonClientFactory.create_grpc_client(server_url, num_retries=3,max_interval_secs=20,base_interval_secs=0.3)
for http client we have default setting:
NUMBER_RETRIES = 3
MAX_INTERVAL_SECS = 20
BASE_INTERVAL_SECS = 0.3
Now you can easy use the client to send requests to the server.
for server:
client.server_live()
client.server_ready()
client.server_metadata()
for model:
client.model_metadata(model_name)
client.model_config(model_name)
client.model_ready(model_name)
client.model_statistics(model_name)
for infer:
client.model_infer(model_name, inputs, model_version, outputs)
client.stream_infer(inputs_iterator)
for repository:
client.repository_index()
client.repository_model_load(model_name)
client.repository_model_unload(model_name)
for system shared memory:
client.system_shared_memory_status()
client.system_shared_memory_register()
client.system_shared_memory_unregister()
for cuda shared memory:
client.cuda_shared_memory_status()
client.cuda_shared_memory_register()
client.cuda_shared_memory_unregister()
for trace setting:
client.trace_setting()
client.get_trace_settings()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
Close
Hashes for tritonv2-1.3.7.dev9-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | d6c7d1f95867aa286156029992a3ccedd4ae41d2c2c7d7408e7ee82c2ec174b0 |
|
MD5 | 37791786af1f3d30a865fe208aa8ae2c |
|
BLAKE2b-256 | 45b8c0c364d9eeb73a02087fe6900291edf255f042849532d988186739d303b1 |