project descriptions here
Project description
tritonv2
A client library for promote triton official client
Installation
pip install tritonv2==1.3.7-dev11
Usage
First, you need to create a client object.
from tritonv2.client_factory import TritonClientFactory
server_url = "localhost:8000"
http_client = TritonClientFactory.create_grpc_client(server_url)
or
async with TritonClientFactory.create_http_aio_client(server_url) as http_aio_client
or
grpc_client = TritonClientFactory.create_grpc_client(server_url)
or
async with TritonClientFactory.create_grpc_aio_client(server_url) as grpc_aio_client
In addition, you can set retry for grpc client:
client = TritonClientFactory.create_grpc_client(server_url, num_retries=3,max_interval_secs=20,base_interval_secs=0.3)
for http client we have default setting:
NUMBER_RETRIES = 3
MAX_INTERVAL_SECS = 20
BASE_INTERVAL_SECS = 0.3
Now you can easy use the client to send requests to the server.
for server:
client.server_live()
client.server_ready()
client.server_metadata()
for model:
client.model_metadata(model_name)
client.model_config(model_name)
client.model_ready(model_name)
client.model_statistics(model_name)
for infer:
client.model_infer(model_name, inputs, model_version, outputs)
client.stream_infer(inputs_iterator)
for repository:
client.repository_index()
client.repository_model_load(model_name)
client.repository_model_unload(model_name)
for system shared memory:
client.system_shared_memory_status()
client.system_shared_memory_register()
client.system_shared_memory_unregister()
for cuda shared memory:
client.cuda_shared_memory_status()
client.cuda_shared_memory_register()
client.cuda_shared_memory_unregister()
for trace setting:
client.trace_setting()
client.get_trace_settings()
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
tritonv2-1.3.7.dev11.tar.gz
(59.0 kB
view hashes)
Built Distribution
Close
Hashes for tritonv2-1.3.7.dev11-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 11ba59454f5ba33af92d9341dbd4e168893550094a70ccf2b66901fa3ca32f20 |
|
MD5 | 6ebcd06a108188312f30e81652f9fe82 |
|
BLAKE2b-256 | b2e52d8dc8963c7cb3b83bc4e454c7bd2793a63fa8fc3bc1bc35d30cd94be91e |