Skip to main content

LlamaIndex Llms Integration: Cohere

Installation

%pip install llama-index-llms-openai
%pip install llama-index-llms-cohere
!pip install llama-index

Basic usage

# Import Cohere and ChatMessage
from llama_index.core.llms import ChatMessage
from llama_index.llms.cohere import Cohere

# Set your API key
api_key = "Your api key"

llm = Cohere(api_key=api_key)
messages = [ChatMessage(role="user", content="Paul Graham is ")]
resp = llm.chat(messages)
print(resp.message.content)

# Output
# an English computer scientist, entrepreneur, investor, and essayist.

# Call chat with a list of messages
from llama_index.core.llms import ChatMessage

messages = [
    ChatMessage(role="user", content="hello there"),
    ChatMessage(
        role="assistant", content="Arrrr, matey! How can I help ye today?"
    ),
    ChatMessage(role="user", content="What is your name"),
]

resp = Cohere(api_key=api_key).chat(
    messages, preamble_override="You are a pirate with a colorful personality"
)
print(resp)

# Output
# assistant: Traditionally, ye refers to gender-nonconforming people of any gender,
# and those who are genderless, whereas matey refers to a friend, commonly used to
# address a fellow pirate. According to pop culture in works like "Pirates of the
# Caribbean", the romantic interest of Jack Sparrow refers to themselves using the
# gender-neutral pronoun "ye".

# Are you interested in learning more about the pirate culture?

Streaming: Using stream_complete endpoint

from llama_index.llms.cohere import Cohere

llm = Cohere(api_key=api_key)
resp = llm.stream_complete("Paul Graham is ")
for r in resp:
    print(r.delta, end="")

# Output
# an English computer scientist, essayist, and venture capitalist.
# He is best known for his work as a co-founder of the Y Combinator startup incubator,
# and his essays, which are widely read and influential in the startup community.

# Using stream_chat endpoint
messages = [
    ChatMessage(role="user", content="hello there"),
    ChatMessage(
        role="assistant", content="Arrrr, matey! How can I help ye today?"
    ),
    ChatMessage(role="user", content="What is your name"),
]

resp = llm.stream_chat(
    messages, preamble_override="You are a pirate with a colorful personality"
)
for r in resp:
    print(r.delta, end="")

# Output
# Arrrr, matey! According to etiquette, we are suppose to exchange names first!
# Mine remains a mystery for now.

Configure Model

llm = Cohere(model="command", api_key=api_key)
resp = llm.complete("Paul Graham is ")
# Note: Your text contains a trailing whitespace, which has been trimmed to ensure high quality generations.
print(resp)

# Output
# an English computer scientist, entrepreneur and investor.
# He is best known for his work as a co-founder of the seed accelerator Y Combinator.
# He is also the co-founder of the online dating platform Match.com.

# Async calls
llm = Cohere(model="command", api_key=api_key)
resp = await llm.acomplete("Paul Graham is ")
# Note: Your text contains a trailing whitespace, which has been trimmed to ensure high quality generations.
print(resp)

# Output
# an English computer scientist, entrepreneur and investor.
# He is best known for his work as a co-founder of the startup incubator and seed fund
# Y Combinator, and the programming language Lisp. He has also written numerous essays,
# many of which have become highly influential in the software engineering field.

# Streaming async
resp = await llm.astream_complete("Paul Graham is ")
async for delta in resp:
    print(delta.delta, end="")

# Output
# an English computer scientist, essayist, and businessman.
# He is best known for his work as a co-founder of the startup accelerator Y Combinator,
# and his essay "Beating the Averages."

Set API Key at a per-instance level

# If desired, you can have separate LLM instances use separate API keys.
from llama_index.llms.cohere import Cohere

llm_good = Cohere(api_key=api_key)
llm_bad = Cohere(model="command", api_key="BAD_KEY")

resp = llm_good.complete("Paul Graham is ")
print(resp)

resp = llm_bad.complete("Paul Graham is ")
print(resp)

LLM Implementation example

https://docs.llamaindex.ai/en/stable/examples/llm/cohere/

Using a Custom Base URL

You can now specify a custom base URL when initializing the Cohere LLM. This is useful for enterprise scenarios or when using a proxy.

from llama_index.llms.cohere import Cohere

# Initialize with a custom base URL
llm = Cohere(
    api_key="your-api-key", base_url="https://your-custom-endpoint.com/v1"
)

resp = llm.complete("What is LlamaIndex?")
print(resp)

Metadata

Release files for llama-index-llms-cohere 0.10.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for llama-index-llms-cohere 0.10.0
File Size Uploaded Provenance Actions
llama_index_llms_cohere-0.10.0.tar.gz 12.6 kB Without provenance Details for llama_index_llms_cohere-0.10.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for llama-index-llms-cohere 0.10.0
File Interpreter ABI Platform Actions
llama_index_llms_cohere-0.10.0-py3-none-any.whl Python 3 none any Details for llama_index_llms_cohere-0.10.0-py3-none-any.whl

Total release size: 25.6 kB

Release files / llama_index_llms_cohere-0.10.0.tar.gz

Download URL llama_index_llms_cohere-0.10.0.tar.gz
Size 12.6 kB
Tags Source
SHA-256 checksum
How to use checksums
d1abf447b2adc577ee761408abe2cf25472f3eb92e922dbe66b55512e2ed5bd0
BLAKE2b-256 checksum
How to use checksums
35c85a0f244c41de36a23ad02c89927026c5257612962f890870fd516ffa79bd
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.8 {"installer":{"name":"uv","version":"0.13.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Release files / llama_index_llms_cohere-0.10.0-py3-none-any.whl

Download URL llama_index_llms_cohere-0.10.0-py3-none-any.whl
Size 12.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
891ab368d5efb4dfa64e897153b387b3a2f4d420aed921a0ea65198eeb07336a
BLAKE2b-256 checksum
How to use checksums
2465ed1f3cb790336073dc1d6fc43d7a9fde72c16da868a9f912fc3563b43951
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.13.8 {"installer":{"name":"uv","version":"0.13.8","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":true}

Security for llama-index-llms-cohere 0.10.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page