LlamaIndex LLM Integration: Anthropic

Anthropic is an AI research company focused on developing advanced language models, notably the Claude series. Their flagship model, Claude, is designed to generate human-like text while prioritizing safety and alignment with human intentions. Anthropic aims to create AI systems that are not only powerful but also responsible, addressing potential risks associated with artificial intelligence.

Installation

%pip install llama-index-llms-anthropic
!pip install llama-index

# Set Tokenizer
# First we want to set the tokenizer, which is slightly different than TikToken.
# NOTE: The Claude 3 tokenizer has not been updated yet; using the existing Anthropic tokenizer leads
# to context overflow errors for 200k tokens. We've temporarily set the max tokens for Claude 3 to 180k.

Basic Usage

import os

from llama_index.llms.anthropic import Anthropic
from llama_index.core import Settings


os.environ["ANTHROPIC_API_KEY"] = "YOUR ANTHROPIC API KEY"
from llama_index.llms.anthropic import Anthropic

# To customize your API key, do this
# otherwise it will lookup ANTHROPIC_API_KEY from your env variable
# llm = Anthropic(api_key="<api_key>")
llm = Anthropic(model="claude-3-opus-20240229")

Settings.tokenizer = llm.tokenizer

resp = llm.complete("Paul Graham is ")
print(resp)

# Sample response
# Paul Graham is a well-known entrepreneur, programmer, venture capitalist, and essayist.
# He is best known for co-founding Viaweb, one of the first web application companies, which was later
# sold to Yahoo! in 1998 and became Yahoo! Store. Graham is also the co-founder of Y Combinator, a highly
# successful startup accelerator that has helped launch numerous successful companies, such as Dropbox,
# Airbnb, and Reddit.

Using Anthropic model through Vertex AI

import os

os.environ["ANTHROPIC_PROJECT_ID"] = "YOUR PROJECT ID HERE"
os.environ["ANTHROPIC_REGION"] = "YOUR PROJECT REGION HERE"
# Set region and project_id to make Anthropic use the Vertex AI client

llm = Anthropic(
    model="claude-3-5-sonnet@20240620",
    region=os.getenv("ANTHROPIC_REGION"),
    project_id=os.getenv("ANTHROPIC_PROJECT_ID"),
)

resp = llm.complete("Paul Graham is ")
print(resp)

Chat example with a list of messages

from llama_index.core.llms import ChatMessage
from llama_index.llms.anthropic import Anthropic

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]
resp = Anthropic(model="claude-3-opus-20240229").chat(messages)
print(resp)

Streaming example

from llama_index.llms.anthropic import Anthropic

llm = Anthropic(model="claude-3-opus-20240229", max_tokens=100)
resp = llm.stream_complete("Paul Graham is ")
for r in resp:
    print(r.delta, end="")

Chat streaming with pirate story

llm = Anthropic(model="claude-3-opus-20240229")
messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality"
    ),
    ChatMessage(role="user", content="Tell me a story"),
]
resp = llm.stream_chat(messages)
for r in resp:
    print(r.delta, end="")

Configure Model

from llama_index.llms.anthropic import Anthropic

llm = Anthropic(model="claude-3-sonnet-20240229")
resp = llm.stream_complete("Paul Graham is ")
for r in resp:
    print(r.delta, end="")

Async completion

from llama_index.llms.anthropic import Anthropic

llm = Anthropic("claude-3-sonnet-20240229")
resp = await llm.acomplete("Paul Graham is ")
print(resp)

Using Anthropic Tools (Web Search)

from llama_index.llms.anthropic import Anthropic

# Initialize with web search tool
llm = Anthropic(
    model="claude-3-7-sonnet-latest",  # Must be a tool-supported model
    max_tokens=1024,
    tools=[
        {
            "type": "web_search_20250305",
            "name": "web_search",
            "max_uses": 3,  # Limit to 3 searches
        }
    ],
)

# Get response with citations
response = llm.complete("What are the latest AI research trends?")

# Access the main response content
print(response.text)

# Access citations if available
for citation in response.citations:
    print(f"Source: {citation.get('url')} - {citation.get('cited_text')}")

Structured Prediction Example

from llama_index.llms.anthropic import Anthropic
from llama_index.core.prompts import PromptTemplate
from llama_index.core.bridge.pydantic import BaseModel
from typing import List


class MenuItem(BaseModel):
    """A menu item in a restaurant."""

    course_name: str
    is_vegetarian: bool


class Restaurant(BaseModel):
    """A restaurant with name, city, and cuisine."""

    name: str
    city: str
    cuisine: str
    menu_items: List[MenuItem]


llm = Anthropic("claude-3-5-sonnet-20240620")
prompt_tmpl = PromptTemplate(
    "Generate a restaurant in a given city {city_name}"
)

# Option 1: Use `as_structured_llm`
restaurant_obj = (
    llm.as_structured_llm(Restaurant)
    .complete(prompt_tmpl.format(city_name="Miami"))
    .raw
)
print(restaurant_obj)

# Option 2: Use `structured_predict`
# restaurant_obj = llm.structured_predict(Restaurant, prompt_tmpl, city_name="Miami")

# Streaming Structured Prediction
from llama_index.core.llms import ChatMessage
from IPython.display import clear_output
from pprint import pprint

input_msg = ChatMessage.from_str("Generate a restaurant in San Francisco")

sllm = llm.as_structured_llm(Restaurant)
stream_output = sllm.stream_chat([input_msg])
for partial_output in stream_output:
    clear_output(wait=True)
    pprint(partial_output.raw.dict())

LLM Implementation example

https://docs.llamaindex.ai/en/stable/examples/llm/anthropic/

Release files for llama-index-llms-anthropic 0.12.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for llama-index-llms-anthropic 0.12.1
File	Size	Uploaded
llama_index_llms_anthropic-0.12.1.tar.gz	16.2 kB	Sep 25, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for llama-index-llms-anthropic 0.12.1
File	Interpreter	ABI	Platform	Reset
llama_index_llms_anthropic-0.12.1-py3-none-any.whl 16.6 kB Sep 25, 2026	Python 3	none	any	Details

Total release size: 32.8 kB

Release files / llama_index_llms_anthropic-0.12.1.tar.gz

Download URL	llama_index_llms_anthropic-0.12.1.tar.gz
Size	16.2 kB
Tags	Source
SHA-256 checksum How to use checksums	`b75c9dd9b4cf53905869418b495603903f6613fa428098c0c607836f8dd6e423`
BLAKE2b-256 checksum How to use checksums	`b8f8d76cf55c59b807b1dc2be812d3eb9228c662970db7ea13640fd1bf883e21`
Upload date	Sep 25, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.19 {"installer":{"name":"uv","version":"0.12.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Release files / llama_index_llms_anthropic-0.12.1-py3-none-any.whl

Download URL	llama_index_llms_anthropic-0.12.1-py3-none-any.whl
Size	16.6 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`2d23113c7cec907056e92fc9646c6189cc9c4d384df612fa5732cb5209de50dd`
BLAKE2b-256 checksum How to use checksums	`57d88b37300f3ce0195d97654cc1de9602ddbb7d03918604fd9230cdfa190d4b`
Upload date	Sep 25, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.19 {"installer":{"name":"uv","version":"0.12.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`