mlflow-crusoe

An MLflow deployment plugin for Crusoe AI's managed inference API.

This plugin lets you manage named model endpoint configurations and run inference on Crusoe's renewable-powered GPU infrastructure directly from MLflow's deployment interface.

Installation

pip install mlflow-crusoe

Setup

Set your Crusoe API key:

export CRUSOE_API_KEY="your-api-key"

You can generate one from the Crusoe Console under Security > Inference API Key.

Usage

import mlflow.deployments

client = mlflow.deployments.get_deploy_client("crusoe")

# Create a deployment
client.create_deployment(
    name="my-llm",
    model_uri="meta-llama/Llama-3.3-70B-Instruct",
    config={"temperature": 0.7, "max_tokens": 2048},
)

# Run inference
result = client.predict("my-llm", inputs={"prompt": "Hello!"})
print(result["choices"][0]["message"]["content"])

# List deployments
client.list_deployments()

# Update
client.update_deployment("my-llm", model_uri="zai/GLM-5.3")

# Delete
client.delete_deployment("my-llm")

Input formats

The predict method accepts three input formats:

# Chat messages (recommended)
client.predict("my-llm", inputs={
    "messages": [
        {"role": "system", "content": "You are helpful."},
        {"role": "user", "content": "Hello!"},
    ]
})

# Simple prompt
client.predict("my-llm", inputs={"prompt": "Hello!"})

# Plain string
client.predict("my-llm", inputs="Hello!")

CLI

# Help
mlflow deployments help -t crusoe

# Create
mlflow deployments create -t crusoe --name my-llm -m meta-llama/Llama-3.3-70B-Instruct

# List
mlflow deployments list -t crusoe

# Predict
mlflow deployments predict -t crusoe --name my-llm --input '{"prompt": "Hi"}'

# Delete
mlflow deployments delete -t crusoe --name my-llm

Available Models

Any other model ID available on Crusoe Managed Inference can be passed as a plain string. See the Crusoe model list for current availability.

Configuration

Compatibility

Requires MLflow 2.0 or newer, including MLflow 3.x. The plugin registers through the mlflow.deployments entry point and implements the full plugin interface (CrusoeDeploymentClient, target_help, run_local).

References

• Crusoe Serverless Inference Docs
• Crusoe Model List
• MLflow Deployment Plugins
• Crusoe Console