LLM JSON Adapter

What is it ?

When using LLMs from the system, you often expect to get output results in JSON: OpenAPI's GPT API has a mechanism called Function Calling, which can return JSON, but Google's Gemini does not seem to have that functionality.

Therefore, I have created a wrapper library to switch LLMs and get results in JSON. What this library can do is as follows.

• Allows you to define the results you want to get in JSON Schema
• Switch between LLMs (currently supports OpenAI's GPT, Google's Gemini, Ollama and Bedrock for Llama and Anthropic Claude).
• Retry a specified number of times if the JSON retrieval fails

How to use

Use the following code to get the results in JSON.

OpenAI

Libraries

You need to install openai.

Attributes

Example

from llm_json_adapter import LLMJsonAdapter, Response

adapter = LLMJsonAdapter(provider_name="openai", max_retry_count=3, attributes={
    "api_key": "Your API Key",
    "model": "gpt-3.5-turbo",
})
result = adapter.generate(
    prompt="prompt",
    language="en",
    act_as="Professional Software Service Business Analyst",
    function=Response(
        name="response name",
        description="response description",
        parameters={
            "type": "object",
            "properties": {
                "data": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "title": {
                                "type": "string",
                            },
                            "description": {
                                "type": "string",
                            },
                        },
                        "required": ["title", "description"],
                    },
                },
            },
            "required": ["data"]
        },
    )
)

Gemini

Libraries

You need to install google-generativeai.

Attributes

Example

from llm_json_adapter import LLMJsonAdapter, Response

adapter = LLMJsonAdapter(provider_name="google", max_retry_count=3, attributes={
    "api_key": "Your API Key",
    "model": "gemini-1.5-pro-latest",
})
result = adapter.generate(
    prompt="prompt",
    language="en",
    act_as="Professional Software Service Business Analyst",
    function=Response(
        name="response name",
        description="response description",
        parameters={
            "type": "object",
            "properties": {
                "data": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "title": {
                                "type": "string",
                            },
                            "description": {
                                "type": "string",
                            },
                        },
                        "required": ["title", "description"],
                    },
                },
            },
            "required": ["data"]
        },
    )
)

Ollama

You need to prepare the Ollama server. ( https://ollama.com/ )

Libraries

You need to install ollama.

Attributes

Example

from llm_json_adapter import LLMJsonAdapter, Response

adapter = LLMJsonAdapter(provider_name="ollama", max_retry_count=3, attributes={
    "url": "http://localhost:11434",
    "model": "llama3",
})
result = adapter.generate(
    prompt="prompt",
    language="en",
    act_as="Professional Software Service Business Analyst",
    function=Response(
        name="response name",
        description="response description",
        parameters={
            "type": "object",
            "properties": {
                "data": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "title": {
                                "type": "string",
                            },
                            "description": {
                                "type": "string",
                            },
                        },
                        "required": ["title", "description"],
                    },
                },
            },
            "required": ["data"]
        },
    )
)

Bedrock

You need to setup the AWS Bedrock( https://aws.amazon.com/bedrock/ )

Libraries

You need to install boto3.

Attributes

Example

from llm_json_adapter import LLMJsonAdapter, Response

adapter = LLMJsonAdapter(provider_name="bedrock", max_retry_count=3, attributes={
    "access_key_id": "<YOUR AWS ACCESS KEY>",
    "secret_access_key": "<YOUR AWS SECRET ACCESS KEY>",
    "region": "us-east-1",
    "model": "anthropic.claude-3-haiku-20240307-v1:0",
    "max_tokens": 1024,
})
result = adapter.generate(
    prompt="prompt",
    language="en",
    act_as="Professional Software Service Business Analyst",
    function=Response(
        name="response name",
        description="response description",
        parameters={
            "type": "object",
            "properties": {
                "data": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "title": {
                                "type": "string",
                            },
                            "description": {
                                "type": "string",
                            },
                        },
                        "required": ["title", "description"],
                    },
                },
            },
            "required": ["data"]
        },
    )
)