LlamaIndex Llms Integration: Ollama
Installation
To install the required package, run:
pip install llama-index-llms-ollama
Setup
- Follow the Ollama README to set up and run a local Ollama instance.
- When the Ollama app is running on your local machine, it will serve all of your local models on
localhost:11434. - Select your model when creating the
Ollamainstance by specifyingmodel=":". - You can increase the default timeout (30 seconds) by setting
Ollama(..., request_timeout=300.0). - If you set
llm = Ollama(..., model="<model family>")without a version, it will automatically look for the latest version.
Usage
Initialize Ollama
from llama_index.llms.ollama import Ollama
llm = Ollama(model="llama3.1:latest", request_timeout=120.0)
Generate Completions
To generate a text completion for a prompt, use the complete method:
resp = llm.complete("Who is Paul Graham?")
print(resp)
Chat Responses
To send a chat message and receive a response, create a list of ChatMessage instances and use the chat method:
from llama_index.core.llms import ChatMessage
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality."
),
ChatMessage(role="user", content="What is your name?"),
]
resp = llm.chat(messages)
print(resp)
Streaming Responses
Stream Complete
To stream responses for a prompt, use the stream_complete method:
response = llm.stream_complete("Who is Paul Graham?")
for r in response:
print(r.delta, end="")
Stream Chat
To stream chat responses, use the stream_chat method:
messages = [
ChatMessage(
role="system", content="You are a pirate with a colorful personality."
),
ChatMessage(role="user", content="What is your name?"),
]
resp = llm.stream_chat(messages)
for r in resp:
print(r.delta, end="")
JSON Mode
Ollama supports a JSON mode to ensure all responses are valid JSON, which is useful for tools that need to parse structured outputs:
llm = Ollama(model="llama3.1:latest", request_timeout=120.0, json_mode=True)
response = llm.complete(
"Who is Paul Graham? Output as a structured JSON object."
)
print(str(response))
Structured Outputs
You can attach a Pydantic class to the LLM to ensure structured outputs:
from llama_index.core.bridge.pydantic import BaseModel
from llama_index.core.tools import FunctionTool
class Song(BaseModel):
"""A song with name and artist."""
name: str
artist: str
llm = Ollama(model="llama3.1:latest", request_timeout=120.0)
sllm = llm.as_structured_llm(Song)
response = sllm.chat([ChatMessage(role="user", content="Name a random song!")])
print(
response.message.content
) # e.g., {"name": "Yesterday", "artist": "The Beatles"}
Asynchronous Chat
You can also use asynchronous chat:
response = await sllm.achat(
[ChatMessage(role="user", content="Name a random song!")]
)
print(response.message.content)
LLM Implementation example
Metadata
Release files for llama-index-llms-ollama 0.12.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| llama_index_llms_ollama-0.12.0.tar.gz | 9.1 kB | Without provenance | Details for llama_index_llms_ollama-0.12.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| llama_index_llms_ollama-0.12.0-py3-none-any.whl | Python 3 | none | any | Details for llama_index_llms_ollama-0.12.0-py3-none-any.whl |
Total release size: 17.9 kB
Release files / llama_index_llms_ollama-0.12.0.tar.gz
| Download URL | llama_index_llms_ollama-0.12.0.tar.gz |
|---|---|
| Size | 9.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
813262d539dc8e02facf98e3cc3317c2cb8ae858cfb20ab30a4ad09091e4d0ec
|
|
BLAKE2b-256 checksum How to use checksums |
0e8e611ef85e09aeb24dea347b7330fd12d1c24e40ad2a556f9c4ea450946989
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.8 {"installer":{"name":"uv","version":"0.13.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Release files / llama_index_llms_ollama-0.12.0-py3-none-any.whl
| Download URL | llama_index_llms_ollama-0.12.0-py3-none-any.whl |
|---|---|
| Size | 8.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
5540c82a395e38db74caeed3d8fb5f4903b8ec12862d3881e6299735dd78b66d
|
|
BLAKE2b-256 checksum How to use checksums |
b26583678cd458bf5c6cf998a9a485f8931fd7644621b72d2f6c865400098056
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.13.8 {"installer":{"name":"uv","version":"0.13.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}
|
Security for llama-index-llms-ollama 0.12.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.