func2llmtool 0.0.2

Turns a Python function into dictionary in the OpenAI standard you can attach as a tool to a LLM request.

func2llmtool - Python function to LLM tool

Usage

from func2llmtool.core import FuncInfo  # this library
import litellm  # or other LLM library
fi = FuncInfo.from_func(my_function)  # your def my_function()
litellm.completion(model='inference/mymodel', messages=[], tools=[fi()])

Installation

Install latest from the GitHub repository:

$ pip install git+https://github.com/NumesSanguis/func2llmtool.git

or from pypi

$ pip install func2llmtool

Documentation

• Documentation (in a nicer format) can be found hosted on this GitHub repository’s pages.
• Package manager specific guidelines on pypi.
• Find the code at: github.com/NumesSanguis/func2llmtool

How to use

Let’s turn our weather function into a tool a LLM can use. The default output format is “litellm”. Supported:

• LiteLLM
• OpenAI

# define our tool function with docstring and variable annotations
def get_weather(
    city: str  # City name
) -> str:  # Returns the weather in a city, e.g. "rainy"
    """Get the current weather for a city."""

    if city.lower() == "tokyo": weather = "sunny"
    elif city.lower() == "amsterdam": weather = "cloudy"
    else: weather = "rainy"

    return weather

Extract the required data:

fi = FuncInfo.from_func(get_weather)
fi  # same as print(fi)

{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current weather for a city. Return type: string (Returns the weather in a city, e.g. \"rainy\")",
        "parameters": {
            "type": "object",
            "properties": {
                "city": {
                    "type": "string",
                    "description": "City name"
                }
            },
            "required": [
                "city"
            ],
            "additionalProperties": false
        }
    }
}

Return the data without pretty printing:

fi()  # same as: fi.to_tool_dict()

{'type': 'function',
 'function': {'name': 'get_weather',
  'description': 'Get the current weather for a city. Return type: string (Returns the weather in a city, e.g. "rainy")',
  'parameters': {'type': 'object',
   'properties': {'city': {'type': 'string', 'description': 'City name'}},
   'required': ['city'],
   'additionalProperties': False}}}

Format output dict according to OpenAI standard:

fi.to_tool_dict(format="openai")

{'type': 'function',
 'name': 'get_weather',
 'description': 'Get the current weather for a city. Return type: string (Returns the weather in a city, e.g. "rainy")',
 'parameters': {'type': 'object',
  'properties': {'city': {'type': 'string', 'description': 'City name'}},
  'required': ['city'],
  'additionalProperties': False},
 'strict': True}

We can also provide the function name as long as it is in your Python globals:

FuncInfo.from_func('get_weather')()

{'type': 'function',
 'function': {'name': 'get_weather',
  'description': 'Get the current weather for a city. Return type: string (Returns the weather in a city, e.g. "rainy")',
  'parameters': {'type': 'object',
   'properties': {'city': {'type': 'string', 'description': 'City name'}},
   'required': ['city'],
   'additionalProperties': False}}}

Other docstring styles - Numpy

# define our tool function with numpy-style docstring
def get_weather_numpy_style(city: str) -> str:
    """
    Get the current weather for a city.

    Parameters
    ----------
    city : str
        City name

    Returns
    -------
    str
        The weather in a city, e.g. "rainy"
    """

    if city.lower() == "tokyo": weather = "sunny"
    elif city.lower() == "amsterdam": weather = "cloudy"
    else: weather = "rainy"

    return weather

FuncInfo.from_func(get_weather_numpy_style)()

{'type': 'function',
 'function': {'name': 'get_weather_numpy_style',
  'description': 'Get the current weather for a city. Return type: string (The weather in a city, e.g. "rainy")',
  'parameters': {'type': 'object',
   'properties': {'city': {'type': 'string', 'description': 'City name'}},
   'required': ['city'],
   'additionalProperties': False}}}

LiteLLM example - Attach tool info to LLM

Here we show how to attach the tool info to a LLM request, handle its tool request, return our tool output, and get our question about the weather answered.

import os
import litellm
from litellm.caching.caching import Cache

# set ENV variables for LLM usage; https://console.groq.com/keys
# os.environ["GROQ_API_KEY"] = "your-api-key"

Prepare our LLM call:

# groq is a fast inference provider (not to be confused with grok by X), which offers a free tier
MODEL = 'groq/llama-3.1-8b-instant'  # any model supported by LiteLLM and for which you set a key
MESSAGES = []
TOOLS = [fi()]
litellm.cache = Cache(type="disk")
litellm.suppress_debug_info = True  # prevent "Provider List: https://docs.litellm.ai/docs/providers" spam

# optional instructions
# MESSAGES.append({"role": "system", "content": "Use the results from tool calls to answer questions."})

MESSAGES.append({"role": "user", "content": "What is the weather in city Tokyo?"})

Call model with our Python function available as tool:

resp = litellm.completion(model=MODEL, messages=MESSAGES, tools=TOOLS, caching=True)
print(resp.choices[0].message.model_dump())

{'content': None, 'role': 'assistant', 'tool_calls': [{'function': {'arguments': '{"city":"Tokyo"}', 'name': 'get_weather'}, 'id': 'hjgzy4gz9', 'type': 'function'}], 'function_call': None, 'provider_specific_fields': None}

Note the 'tool_calls': [{'function': {'arguments': '{"city":"Tokyo"}', 'name': 'get_weather'}, ... part. This is the tool call that the LLM made. Let’s execute it and inform the LLM.

# example only shows how to handle 1 tool call (multiple can be requested in parallel)
tc = resp.choices[0].message.tool_calls[0]
tc

ChatCompletionMessageToolCall(function=Function(arguments='{"city":"Tokyo"}', name='get_weather'), id='hjgzy4gz9', type='function')

# The id associated to the tool call, so the LLM knows which output is associated to which tool call
tc.id

'hjgzy4gz9'

# The arguments are a JSON string, so we need to convert them before calling our function
import json

func_args = json.loads(tc.function.arguments)
func_args

{'city': 'Tokyo'}

Add tool call and usage to message history:

# keep track of the tool call by the LLM in our message history
MESSAGES.append({k:v for k,v in resp.choices[0].message.model_dump().items() if v is not None})

# Add tool call result to message history
MESSAGES.append({'role': 'tool', 'tool_call_id': tc.id, 'content': get_weather(**func_args)})
# We now have all info to query the LLM again
print(MESSAGES)

[{'role': 'user', 'content': 'What is the weather in city Tokyo?'}, {'role': 'assistant', 'tool_calls': [{'function': {'arguments': '{"city":"Tokyo"}', 'name': 'get_weather'}, 'id': 'hjgzy4gz9', 'type': 'function'}]}, {'role': 'tool', 'tool_call_id': 'hjgzy4gz9', 'content': 'sunny'}]

Give our LLM back the tool execution results and ask it to answer our initial question.

resp2 = litellm.completion(model=MODEL, messages=MESSAGES, tools=TOOLS, caching=True)
print(resp2.choices[0].message.content)

It's currently sunny in Tokyo.

Alternatives to this library

# LiteLLM has a build-in function, but it is not as flexible what is supported.
# Note the missing parameter description and nothing about what's returned
litellm.utils.function_to_dict(get_weather)

{'name': 'get_weather',
 'description': 'Get the current weather for a city.',
 'parameters': {'type': 'object',
  'properties': {'city': {'type': 'string'}},
  'required': ['city']}}

# gets the 'city' description, but nothing about what's returned
litellm.utils.function_to_dict(get_weather_numpy_style)

{'name': 'get_weather_numpy_style',
 'description': 'Get the current weather for a city.',
 'parameters': {'type': 'object',
  'properties': {'city': {'type': 'string', 'description': 'City name'}},
  'required': ['city']}}