📌 Overview

ibm-watsonx-ai is the official Python SDK for IBM watsonx.ai, an enterprise-grade AI platform for building, training, tuning, deploying, and operating AI models at scale.

The SDK provides a unified and production-ready Python interface to the full watsonx.ai ecosystem, including Foundation Models (within LLMs), AutoAI experiments, Retrieval-Augmented Generation (RAG), model tuning, deployment, and data integration.

With ibm-watsonx-ai, developers and data scientists can seamlessly integrate advanced AI capabilities into Python applications running on IBM watsonx.ai for IBM Cloud or IBM watsonx.ai software, while meeting enterprise requirements such as security, governance, and scalability.

🎯 What This SDK Is Used For

The ibm-watsonx-ai SDK is designed to support the entire AI lifecycle:

• 🔐 Secure authentication and environment configuration
• 🤖 Inference with Foundation Models (LLMs, embeddings, time-series, audio)
• 📚 Building Retrieval-Augmented Generation (RAG) systems
• 🧪 Running and optimizing AutoAI experiments
• ⚙️ Fine-tuning and prompt tuning of models
• 🚀 Deploying models to scalable inference endpoints
• 🔗 Integrating enterprise data sources into AI workflows

It is suitable for research, prototyping, and production deployments.

📦 Installation

Install from PyPI:

pip install ibm-watsonx-ai

Install with optional extras:

pip install "ibm-watsonx-ai[rag]"

🚀 Quick Start

Authentication

Set up your Credentials and create an APIClient instance:

from ibm_watsonx_ai import Credentials, APIClient

credentials = Credentials(
    url="https://us-south.ml.cloud.ibm.com",
    api_key="<your-ibm-api-key>"
)

# Initialize APIClient using a space ID (you can also use a project ID)
api_client = APIClient(credentials, space_id="<your-space-id>")

Working with ModelInference

List available chat models

list(api_client.foundation_models.ChatModels)

# Output example:
# [<ChatModels.GRANITE_3_3_8B_INSTRUCT: 'ibm/granite-3-3-8b-instruct'>,
#  <ChatModels.GRANITE_4_H_SMALL: 'ibm/granite-4-h-small'>,
#  <ChatModels.LLAMA_3_3_70B_INSTRUCT: 'meta-llama/llama-3-3-70b-instruct'>,
#  <ChatModels.GPT_OSS_120B: 'openai/gpt-oss-120b'>]

Initialize ModelInference

from ibm_watsonx_ai.foundation_models import ModelInference

# Create a `ModelInference` instance for the selected model
model = ModelInference(
    api_client=api_client,
    model_id="ibm/granite-4-h-small"
)

Chat with the model

# Prepare messages
messages = [
    {"role": "system", "content": "You are a helpful assistant that translates English to French."},
    {"role": "user", "content": "I love you for listening to Rock."}
]

# Get model response
response = model.chat(messages=messages)
print(response)