tritony 0.0.8

Tiny configuration for Triton Inference Server

tritony - Tiny configuration for Triton Inference Server

Key Features

• Simple configuration. Only $host:$port and $model_name are required.
• Generating asynchronous requests with asyncio.Queue

Requirements

$ pip install tritonclient[all]

Install

$ pip install tritony

Test

With Triton

docker run --rm \
    -v ${PWD}:/models \
    nvcr.io/nvidia/tritonserver:22.01-pyt-python-py3 \
    tritonserver --model-repo=/models

pytest -m -s tests/test_tritony.py

Example with image_client.py

• Follow steps in the official triton server documentation

# Download Images from https://github.com/triton-inference-server/server.git
python ./example/image_client.py --image_folder "./server/qa/images"