LlamaIndex Llms Integration: Ollama

Installation

To install the required package, run:

pip install llama-index-llms-ollama

Setup

Follow the Ollama README to set up and run a local Ollama instance.
When the Ollama app is running on your local machine, it will serve all of your local models on localhost:11434.
Select your model when creating the Ollama instance by specifying model=":".
You can increase the default timeout (30 seconds) by setting Ollama(..., request_timeout=300.0).
If you set llm = Ollama(..., model="<model family>") without a version, it will automatically look for the latest version.

Usage

Initialize Ollama

from llama_index.llms.ollama import Ollama

llm = Ollama(model="llama3.1:latest", request_timeout=120.0)

Generate Completions

To generate a text completion for a prompt, use the complete method:

resp = llm.complete("Who is Paul Graham?")
print(resp)

Chat Responses

To send a chat message and receive a response, create a list of ChatMessage instances and use the chat method:

from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality."
    ),
    ChatMessage(role="user", content="What is your name?"),
]
resp = llm.chat(messages)
print(resp)

Streaming Responses

Stream Complete

To stream responses for a prompt, use the stream_complete method:

response = llm.stream_complete("Who is Paul Graham?")
for r in response:
    print(r.delta, end="")

Stream Chat

To stream chat responses, use the stream_chat method:

messages = [
    ChatMessage(
        role="system", content="You are a pirate with a colorful personality."
    ),
    ChatMessage(role="user", content="What is your name?"),
]
resp = llm.stream_chat(messages)
for r in resp:
    print(r.delta, end="")

JSON Mode

Ollama supports a JSON mode to ensure all responses are valid JSON, which is useful for tools that need to parse structured outputs:

llm = Ollama(model="llama3.1:latest", request_timeout=120.0, json_mode=True)
response = llm.complete(
    "Who is Paul Graham? Output as a structured JSON object."
)
print(str(response))

Structured Outputs

You can attach a Pydantic class to the LLM to ensure structured outputs:

from llama_index.core.bridge.pydantic import BaseModel
from llama_index.core.tools import FunctionTool


class Song(BaseModel):
    """A song with name and artist."""

    name: str
    artist: str


llm = Ollama(model="llama3.1:latest", request_timeout=120.0)
sllm = llm.as_structured_llm(Song)

response = sllm.chat([ChatMessage(role="user", content="Name a random song!")])
print(
    response.message.content
)  # e.g., {"name": "Yesterday", "artist": "The Beatles"}

Asynchronous Chat

You can also use asynchronous chat:

response = await sllm.achat(
    [ChatMessage(role="user", content="Name a random song!")]
)
print(response.message.content)

LLM Implementation example

https://docs.llamaindex.ai/en/stable/examples/llm/ollama/

Metadata

Release files for llama-index-llms-ollama 0.11.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for llama-index-llms-ollama 0.11.0
File	Size	Uploaded
llama_index_llms_ollama-0.11.0.tar.gz	9.1 kB	Aug 31, 2026	Details

Built distribution (wheel)

Table of built distributions (wheels) for llama-index-llms-ollama 0.11.0
File	Interpreter	ABI	Platform	Reset
llama_index_llms_ollama-0.11.0-py3-none-any.whl 8.8 kB Aug 31, 2026	Python 3	none	any	Details

Total release size: 17.9 kB

Release files / llama_index_llms_ollama-0.11.0.tar.gz

Download URL	llama_index_llms_ollama-0.11.0.tar.gz
Size	9.1 kB
Tags	Source
SHA-256 checksum How to use checksums	`e6b8cac200a9ff67caea531fece0a6272a89e3506b2b9648ec709acffe4a7980`
BLAKE2b-256 checksum How to use checksums	`1199b82667831e75aca2f49488b2904520180c19cba3c24fc55ee4b177d77888`
Upload date	Aug 31, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.7 {"installer":{"name":"uv","version":"0.12.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`

Release files / llama_index_llms_ollama-0.11.0-py3-none-any.whl

Download URL	llama_index_llms_ollama-0.11.0-py3-none-any.whl
Size	8.8 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`835d552625082a79a4dc17e4a2c5e9bf70de26c7f679db7b3bf427dd593e5247`
BLAKE2b-256 checksum How to use checksums	`649cbf68e265e52ce76df8511b8052f97975d320ec4703ae0f4e2c19c0e99a25`
Upload date	Aug 31, 2026
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`uv/0.12.7 {"installer":{"name":"uv","version":"0.12.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"24.04","id":"noble","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}`