Skip to main content

llama-index multi_modal_llms reka integration

Project description

LlamaIndex Multi_Modal_Llms Integration: Reka

LlamaIndex Multi-Modal LLMs Integration: Reka This package provides integration between the Reka multi-modal language model and LlamaIndex, allowing you to use Reka's powerful language models with image input capabilities in your LlamaIndex applications. Installation To use this integration, you need to install the llama-index-multi-modal-llms-reka package:

pip install llama-index-multi-modal-llms-reka

To obtain an API key, please visit https://platform.reka.ai/ Our baseline models always available for public access are:

  • reka-edge
  • reka-flash
  • reka-core

Other models may be available. The Get Models API allows you to list what models you have available to you. Using the Python SDK, it can be accessed as follows:

from reka.client import Reka

client = Reka()
print(client.models.get())

Usage

Here are some examples of how to use the Reka Multi-Modal LLM integration with LlamaIndex: Initialize the Reka Multi-Modal LLM client

import os
from llama_index.llms.reka import RekaMultiModalLLM

api_key = os.getenv("REKA_API_KEY")
reka_mm_llm = RekaMultiModalLLM(model="reka-flash", api_key=api_key)

Chat completion with image

from llama_index.core.base.llms.types import ChatMessage, MessageRole
from llama_index.core.schema import ImageDocument

# Create an ImageDocument with the image URL or local file path
image_doc = ImageDocument(image_url="https://example.com/image.jpg")
# Or for a local file:
image_doc = ImageDocument(image_path="/path/to/local/image.jpg")

messages = [
    ChatMessage(
        role=MessageRole.SYSTEM, content="You are a helpful assistant."
    ),
    ChatMessage(
        role=MessageRole.USER, content="What do you see in this image?"
    ),
]

response = reka_mm_llm.chat(messages, image_documents=[image_doc])
print(response.message.content)

Text completion with image

from llama_index.core.schema import ImageDocument

image_doc = ImageDocument(image_url="https://example.com/image.jpg")
prompt = "Describe the contents of this image:"

response = reka_mm_llm.complete(prompt, image_documents=[image_doc])
print(response.text)

Streaming Responses

Streaming chat completion with image

messages = [
    ChatMessage(
        role=MessageRole.SYSTEM, content="You are a helpful assistant."
    ),
    ChatMessage(
        role=MessageRole.USER, content="Describe the colors in this image."
    ),
]

for chunk in reka_mm_llm.stream_chat(messages, image_documents=[image_doc]):
    print(chunk.delta, end="", flush=True)

Streaming text completion with image

prompt = "List the objects you can see in this image:"

for chunk in reka_mm_llm.stream_complete(prompt, image_documents=[image_doc]):
    print(chunk.delta, end="", flush=True)

Asynchronous Usage

import asyncio


async def main():
    # Async chat completion with image
    messages = [
        ChatMessage(
            role=MessageRole.SYSTEM, content="You are a helpful assistant."
        ),
        ChatMessage(
            role=MessageRole.USER,
            content="What's the main subject of this image?",
        ),
    ]
    response = await reka_mm_llm.achat(messages, image_documents=[image_doc])
    print(response.message.content)

    # Async text completion with image
    prompt = "Describe the background of this image:"
    response = await reka_mm_llm.acomplete(prompt, image_documents=[image_doc])
    print(response.text)

    # Async streaming chat completion with image
    messages = [
        ChatMessage(
            role=MessageRole.SYSTEM, content="You are a helpful assistant."
        ),
        ChatMessage(
            role=MessageRole.USER,
            content="What objects are visible in this image?",
        ),
    ]
    async for chunk in await reka_mm_llm.astream_chat(
        messages, image_documents=[image_doc]
    ):
        print(chunk.delta, end="", flush=True)

    # Async streaming text completion with image
    prompt = "List the colors present in this image:"
    async for chunk in await reka_mm_llm.astream_complete(
        prompt, image_documents=[image_doc]
    ):
        print(chunk.delta, end="", flush=True)


asyncio.run(main())

Running Tests

To run the tests for this integration, you'll need to have pytest and pytest-asyncio installed. You can install them using pip:

pip install pytest pytest-asyncio

Then, set your Reka API key as an environment variable:

export REKA_API_KEY=your_api_key_here

Now you can run the tests using pytest:

pytest tests/test_multi_modal_llms_reka.py -v

To run only mock integration tests without remote connections:

pytest tests/test_multi_modal_llms_reka.py -v -k "mock"

Note: The test file should be named test_multi_modal_llms_reka.py and placed in the appropriate directory.

The Reka Multi-Modal LLM supports various image input formats, including URLs, local file paths, and base64-encoded image strings. When using local file paths, make sure the files are accessible to your application. The model can process multiple images in a single request by passing a list of ImageDocument objects.

Contributing

Contributions to improve this integration are welcome. Please ensure that you add or update tests as necessary when making changes. When adding new features or modifying existing ones, please update this README to reflect those changes.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

Built Distribution

File details

Details for the file llama_index_multi_modal_llms_reka-0.2.0.tar.gz.

File metadata

File hashes

Hashes for llama_index_multi_modal_llms_reka-0.2.0.tar.gz
Algorithm Hash digest
SHA256 d02933afeac5aa9a80a0d4c6e29bb49df975e0983a0ab786f8f6d32bbe86cf47
MD5 a3c44a6f25698f17164c9e253fced706
BLAKE2b-256 0e24e497322aa5b5ea89903fbd2f24dbbbb8977112579b90434af4c7f1c1d516

See more details on using hashes here.

File details

Details for the file llama_index_multi_modal_llms_reka-0.2.0-py3-none-any.whl.

File metadata

File hashes

Hashes for llama_index_multi_modal_llms_reka-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 eac43398a548d242276925700d998ab2b5b7a4a1ddaee9b15ba71ced17224702
MD5 e889e362bf4ab20579c9f30778f95227
BLAKE2b-256 298ecf08f4173b725bb281e9cc5428fab25841ab39a558767c3c4be96b875202

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page