llm-gcp-vertex

llm plugin for Google Cloud Vertex AI, providing access to Gemini and Claude models through their official SDKs. If you are looking for support for Google AI Studio, see the llm-gemini plugin.

Features

• Gemini models via the official google-genai SDK
• Claude models via the official anthropic[vertex] SDK
• Full streaming support for both model families
• Async support for high-throughput applications

Installation

llm install llm-gcp-vertex

Prerequisites

1. Google Cloud Authentication

This plugin uses Application Default Credentials (ADC):

gcloud auth application-default login

2. Configure Project and Location

Using environment variables:

# Falls back to GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION if not set
export LLM_VERTEX_CLOUD_PROJECT="your-project-id"
export LLM_VERTEX_CLOUD_LOCATION="us-central1"  # optional, defaults to us-central1

Or using LLM keys:

llm keys set vertex-project
llm keys set vertex-location  # optional

Available Models

Gemini

Claude (via Vertex AI)

Usage

Command Line

# Gemini
llm -m gemini-2.5-flash "Explain quantum computing"
llm -m gemini-2.5-pro "Write a poem" -o temperature 0.9

# Claude
llm -m claude-sonnet-4.5 "Review this code"
llm -m claude-haiku-4.5 "Summarize this" -o max_tokens 500

# With system prompt
llm -m gemini-2.0-flash "Explain recursion" -s "You are a patient teacher"

Options

Gemini Options

Claude Options

Development

git clone https://github.com/ASRagab/llm-gcp-vertex.git
cd llm-gcp-vertex

# Install with dev dependencies
uv sync --group dev

# Run tests
uv run pytest

# Type check (strict)
uv run basedpyright llm_gcp_vertex.py

Troubleshooting

"Vertex AI project ID required"

export LLM_VERTEX_CLOUD_PROJECT="your-project-id"
# or
llm keys set vertex-project

"Could not automatically determine credentials"

gcloud auth application-default login

Claude models not working

Claude on Vertex AI requires:

1. Enable the Claude API in your GCP project
2. Accept the usage agreement in Model Garden
3. Use a supported region (e.g., us-east5, europe-west1)

export LLM_VERTEX_CLOUD_LOCATION="us-east5"

License

Apache 2.0