llmer 1.0.0

llmer is a lightweight Python library designed to streamline the development of applications leveraging large language models (LLMs). It provides high-level APIs and utilities for parallel processing, runtime management, file handling, and prompt generation, reducing the overhead of repetitive tasks.

LLMER: Simplify LLM Application Development

llmer is a lightweight Python library designed to streamline the development of applications leveraging large language models (LLMs). It provides high-level APIs and utilities for parallel processing, runtime management, file handling, and prompt generation, reducing the overhead of repetitive tasks.

---

Features

• OpenAI Calling:
• Parallel Processing: Execute functions in parallel with thread pools or async tasks.
• Runtime Management: Handle execution timeouts and thread-safe locks with decorators.
• File Utilities: Work seamlessly with YAML, JSONL, and Base64 encoded images.
• Prompt Management: Generate ChatML-compliant prompts for conversational models.
• More Features: Comming soon...

---

Installation

To install llmer, use:

pip install llmer

Quick Start Example

from llmer.parallel.thread_pool import ThreadPool


@ThreadPool(parallel_count=4)
def square(num):
  return num ** 2


tasks = [{"num": i} for i in range(10)]
results = square(tasks)
print(results)  # Output: [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Module Overview

The project is organized into the following modules:

1. OpenAI Calling

• Azure: Calling language models hosted on Microsoft Azure.
• OpenAI: Calling language models from OpenAI and the models that deployed within openai style.

2. Parallel Processing

• OpenAI Calling: Calling azure models and the models that deployed within openai style.
• ThreadPool: Execute functions in parallel using a thread pool.
• MultiThread: Execute functions with multi threads.
• AsyncParallel: Asynchronous parallel execution for async function.
• AsyncExecutor: Flexible async execution manager for non-async function.

3. Runtime Management

• Timeout decorators for functions (include generator-based functions).
• Parallel-safe locks with customizable timeout settings.

4. File Utilities

• Read and write JSONL and YAML files.
• Encode images to Base64 strings with optional MIME type prefixes.

5. Prompt Management

• Generate ChatML-compatible prompts for LLM-based conversational agents.

API Documentation

1. Model Integration

The Model module within llmer provides an abstraction layer for integrating with various large language models (LLMs) from different providers. The goal is to simplify the process of querying these models, handling the communication, and retrieving the results in a consistent way. Currently, the module supports integration with Azure models and the models that deployed within openai style as well.

1.1 Azure

The Azure model integration allows users to interact with language models hosted on Microsoft Azure.

Azure provides scalable language models for tasks such as text generation, question answering, function call, and more.

Usage Example:

from llmer.model import Azure

openai_api_key = "xxxx"
openai_model_name = "gpt-4o"
openai_api_version = 'xxxx'
openai_api_base = "xxxx"

headers = {}

gpt4o = Azure(
  headers=headers,
  api_key=openai_api_key,
  api_version=openai_api_version,
  endpoint=openai_api_base,
  timeout=10,
  retry=2
)

gpt_response = gpt4o.chat(
  model=openai_model_name,
  temperature=0.1,
  stream=True,
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Can you tell me the weather today?"},
  ],
  response_format=None,
)

for chunk in gpt_response:
  print(chunk)

1.2 OpenAI Style

The llmer library also supports integration with OpenAI's models, providing an easy-to-use interface for interacting with OpenAI's language models, such as GPT-3 and GPT-4 and other models that deployed within openai style.

Usage Example:

from llmer.model import Azure, OpenAI

claude_api_key = "xxxx"
claud_model_name = "anthropic.claude-3-5-sonnet-20240620-v1:0"
claude_api_base = "xxxx"

headers = {}

claude = OpenAI(
  headers=headers,
  api_key=claude_api_key,
  endpoint=claude_api_base,
  timeout=10,
  retry=2
)

response = claude.chat(
  model=claud_model_name,
  temperature=0.1,
  stream=True,
  messages=[
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Can you tell me the weather today?"},
  ],
  response_format=None,
)
for chunk in response:
  print(chunk)

2. Parallel Execution

2.1 ThreadPool

Executes a function in parallel using a thread pool.

Usage Example:

from llmer.parallel.thread_pool import ThreadPool


@ThreadPool(parallel_count=3)
def add_one(num):
  return num + 1


tasks = [{"num": i} for i in range(5)]
results = add_one(tasks)
print(results)  # Output: [1, 2, 3, 4, 5]

2.2 MultiThread

Provides explicit control over multi threads execution.

Usage Example:

from llmer.parallel.multi_thread import MultiThread


@MultiThread(parallel_count=2)
def multiply(num):
  return num * 2


tasks = [{"num": i} for i in range(4)]
results = multiply(tasks)
print(results)  # Output: [0, 2, 4, 6]

2.3 AsyncParallel

Executes asynchronous tasks in parallel with limited concurrency.

Usage Example:

import asyncio
from llmer.parallel.async_parallel import AsyncParallel


@AsyncParallel(parallel_count=2)
async def async_task(num):
  await asyncio.sleep(1)
  return num * 10


tasks = [{"num": i} for i in range(4)]
results = asyncio.run(async_task(tasks))
print(results)  # Output: [0, 10, 20, 30]

2.4 AsyncExecutor

Executes tasks in parallel using asynchronous execution for non-async functions.

Usage Example:

import asyncio
import time
from llmer.parallel.async_executor import AsyncExecutor


@AsyncExecutor(parallel_count=2)
def async_task(num):
  time.sleep(1)
  return num * 5


tasks = [{"num": i} for i in range(4)]
results = asyncio.run(async_task(tasks))
print(results)  # Output: [0, 5, 10, 15]

3. Runtime Management

3.1 Timeout Decorator

The timeout decorator allows you to set a time limit for a function to execute. If the function exceeds the specified time, an ExecutionTimeoutError is raised.

Usage Example:

import time
from llmer.runtime.context import timeout
from llmer.runtime.exceptions import ExecutionTimeoutError


@timeout(3)  # Set timeout to 3 seconds
def long_running_function():
  time.sleep(5)  # Simulate a long task


try:
  long_running_function()
except ExecutionTimeoutError as e:
  print(e)  # Output: Function 'long_running_function' timed out after 3 seconds

3.2 Parallel Safe Lock Decorator

The parallel_safe_lock decorator ensures that a function can only be executed if it successfully acquires a lock. If the function cannot acquire the lock within the specified timeout, it raises a AcquireLockTimeoutError. This is useful when functions are working with shared resources and you want to prevent concurrent execution of these functions.

Usage Example:

from llmer.runtime import parallel_safe_lock, AcquireLockTimeoutError
import threading
from time import sleep

lock = threading.Lock()


@parallel_safe_lock(lock, seconds=0.5)
def write_data(data: str):
  print(f"Writing data: {data}")
  sleep(0.6)


def task():
  try:
    write_data("some data")
  except AcquireLockTimeoutError as e:
    print(e)  # Output: critical_section acquires lock, timeout exceeded 0.5


threads = []
for _ in range(3):
  t = threading.Thread(target=task)
  t.start()
  threads.append(t)

for t in threads:
  t.join()

4. File Utilities

The file.py module provides utilities for file handling, such as reading from and writing to JSONL and YAML files, and converting images to Base64 encoded strings.

4.1 File to List

The file_to_list() function reads a JSONL file and returns its contents as a list of dictionaries. The path can be either absolute or relative to the script directory.

Parameters:

• path (Optional[str]): The path to the JSONL file. If not provided, it defaults to reading from the current script directory.

Usage Example:

from llmer.file import file_to_list

# Example usage: Reading a JSONL file from the current script directory
data = file_to_list("data.jsonl")
print(data)
# Output: List of dictionaries read from the JSONL file

This function is useful for reading structured data (such as JSONL format) and processing it in memory as a list.

4.2 File to List

The list_to_file() function saves a list of data (e.g., dictionaries, strings, etc.) into a JSONL file. You can specify the mode in which the file should be opened ('w' for write, 'a' for append).

Parameters:

• data (List[Any]): The data to be saved into the file. It should be a list, and each item in the list will be written to a new line in the JSONL file.
• path (Optional[str]): The path to the JSONL file. If not provided, it defaults to the current script directory.
• mode (str, default 'w'): The mode in which the file is opened. 'w' will overwrite the file, and 'a' will append data to the file.

Usage Example:

from llmer.file import list_to_file

# Example data to save
data = [{"name": "John", "age": 30}, {"name": "Jane", "age": 25}]

# Example usage: Saving a list of dictionaries to a JSONL file
list_to_file(data, "output.jsonl")

This function is useful for saving structured data in JSONL format, which is efficient for large datasets and stream processing.

4.3 YAML Reader

The yaml_reader() function reads a YAML configuration file and returns its contents as a Python dictionary. The function assumes the YAML file is structured correctly and returns the parsed data.

Parameters:

• path (Optional[str]): The path to the YAML file. If not an absolute path, it will be joined with the current script's directory.

Usage Example:

from llmer.file import yaml_reader

# Example YAML file path
yaml_file = "config.yaml"

# Example usage: Reading a YAML configuration file
config = yaml_reader(yaml_file)

print(config)

This function is useful for loading configuration files or structured data in YAML format, which is commonly used for application settings and configurations.

4.4 Image to Base64

The image_to_base64() function converts an image file into a Base64-encoded string. Optionally, it can include a data URI prefix for embedding images directly in web content or other use cases.

Parameters:

• path (Optional[str]): The path to the image file (can be relative or absolute).
• prefix (bool, optional): If True, adds a data URI prefix to the Base64 string. Default is False.

Usage Example:

from llmer.file import image_to_base64

# Example image file path
image_file = "example.png"

# Convert image to Base64 without prefix
encoded_image = image_to_base64(image_file)

print(encoded_image)

# Convert image to Base64 with prefix
encoded_image_with_prefix = image_to_base64(image_file, prefix=True)

print(encoded_image_with_prefix)

5. Prompt Utilities

5.1 ChatML Formatter

The chatml() function formats a list of messages into the ChatML format, which is commonly used for processing conversations in a structured way for language models. This function creates a string where each message is wrapped with appropriate tags for system, user, and assistant roles.

Parameters:

• messages (List[Dict[str, str]]): A list of messages where each message is a dictionary containing the keys:
  • role (str): The role of the speaker, e.g., "system", "user", or "assistant".
  • content (str): The content of the message.

Usage Example:

from llmer.prompt import chatml

# Example messages
messages = [
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "Can you tell me the weather today?"},
  {"role": "assistant", "content": "The weather is sunny with a chance of rain."}
]

# Format messages into ChatML
formatted_message = chatml(messages)

print(formatted_message)

<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Can you tell me the weather today?<|im_end|>
<|im_start|>assistant
The weather is sunny with a chance of rain.<|im_end|>
<|im_start|>assistant

This function is essential for formatting messages in a way that is compatible with APIs or models that use the ChatML format for structured input.

Contributing

Feel free to contribute to this project. You can open an issue or submit a pull request.

Contact info

You can contact at [e-mail]