inference-gateway 0.2.3

A Python SDK for Inference Gateway

Inference Gateway Python SDK

An SDK written in Python for the Inference Gateway.

• Inference Gateway Python SDK
  • Installation
  • Usage
    • Creating a Client
    • Listing Models
    • Generating Content
    • Health Check
  • License

Installation

pip install inference-gateway

Usage

Creating a Client

from inference_gateway.client import InferenceGatewayClient, Provider

client = InferenceGatewayClient("http://localhost:8080")

# With authentication token(optional)
client = InferenceGatewayClient("http://localhost:8080", token="your-token")

Listing Models

To list available models, use the list_models method:

models = client.list_models()
print("Available models:", models)

Generating Content

To generate content using a model, use the generate_content method:

from inference_gateway.client import Provider, Role, Message

messages = [
    Message(Role.SYSTEM, "You are a helpful assistant"),
    Message(Role.USER, "Hello!"),
]

response = client.generate_content(
    provider=Provider.OPENAI,
    model="gpt-4",
    messages=messages
)
print("Assistant:", response["choices"][0]["message"]["content"])

Health Check

To check the health of the API, use the health_check method:

is_healthy = client.health_check()
print("API Status:", "Healthy" if is_healthy else "Unhealthy")

License

This SDK is distributed under the MIT License, see LICENSE for more information.