prompttrail 0.1.0

A lightweight library for Prompt Programming: Agent as Code

PromptTrail

PromptTrail is a lightweight library to help you build something with LLM.

PromptTrail provide:

A unified interface to various LLMs

A simple and intuituve DSL for "Agent as Code"

And various "Developer Tools" to help you build LLM applications.

• PromptTrail
  • Qucikstart
  • Installation
  • What PromptTrail can do?
  • Examples
    • LLM API Call
    • Developer Tools
    • Agent as Code
    • Tooling
  • Next
  • License
  • Contributing
  • Q&A
    • Why bother yet another LLM library?
  • Showcase

Qucikstart

• See Quickstart for more details.

Installation

pip install prompttrail

or

git clone https://github.com/combinatrix-ai/PromptTrail.git
cd PromptTrail
pip install -e .

What PromptTrail can do?

• PromptTrail offers the following features:
  • Unified interface to various LLMs
    • OpenAI
    • Google Cloud (Palm)
      • [TODO] Gemini
    • Anthropic Claude
    • [TODO] Local LLMs
  • Developer Tools for prompt programming
    • Mocking LLMs for testing
    • [TODO] Logging
    • [TODO] Debugging
  • Everything you need to do "Agent as Code"
    • Template
    • Runner
    • Hooks
    • Calling other APIs other than LLMs (Tooling)
      • Function Calling
      • Built-in Tools
        • [TODO] Code Execution
        • [TODO] Vector Search

Examples

You can find more examples in examples directory.

LLM API Call

This is the simplest example of how to use PromptTrail as a thin wrapper around LLMs of various providers.

> import os
> from src.prompttrail.core import Session, Message
> from src.prompttrail.models.openai import OpenAIChatCompletionModel, OpenAIModelConfiguration, OpenAIModelParameters
> 
> api_key = os.environ["OPENAI_API_KEY"]
> config = OpenAIModelConfiguration(api_key=api_key)
> parameters = OpenAIModelParameters(model_name="gpt-3.5-turbo", max_tokens=100, temperature=0)
> model = OpenAIChatCompletionModel(configuration=config)
> session = Session(
>   messages=[
>     Message(content="Hey", sender="user"),
>   ]
> )
> message = model.send(parameters=parameters, session=session)

Message(content="Hello! How can I assist you today?", sender="assistant")

If you want streaming output, you can use the send_async method if the provider offers the feature.

> message_generator = model.send_async(parameters=parameters, session=session)
> for message in message_generator:
>     print(message.content, sep="", flush=True)

Hello! How can # text is incrementally typed

Developer Tools

We provide various tools for developers to build LLM applications. For example, you can mock LLMs for testing.

> # Change model class to mock model class
> model = OpenAIChatCompletionModelMock(configuration=config)
> # and just call the setup method to set up the mock provider
> model.setup(
>     mock_provider=OneTurnConversationMockProvider(
>         conversation_table={
>             "1+1": "1215973652716",
>         },
>         sender="assistant",
>     )
> )
> session = Session(
>     messages=[
>         Message(content="1+1", sender="user"),
>     ]
> )
> message = model.send(parameters=parameters, session=session)
> print(message)

TextMessage(content="1215973652716", sender="assistant")

Agent as Code

You can write a simple agent like below. Without reading the documentation, you can understand what this agent does!

template = LinearTemplate(
    [
        MessageTemplate(
            role="system",
            content="You're a math teacher bot.",
        ),
        LoopTemplate(
            [
                UserInputTextTemplate(
                    role="user",
                    description="Let's ask a question to AI:",
                    default="Why can't you divide a number by zero?",
                ),
                GenerateTemplate(
                    role="assistant",
                ),
                MessageTemplate(role="assistant", content="Are you satisfied?"),
                UserInputTextTemplate(
                    role="user",
                    description="Input:",
                    default="Yes.",
                ),
                # Let the LLM decide whether to end the conversation or not
                MessageTemplate(
                    role="assistant",
                    content="The user has stated their feedback."
                    + "If you think the user is satisfied, you must answer `END`. Otherwise, you must answer `RETRY`."
                ),
                check_end := GenerateTemplate(
                    role="assistant",
                ),
            ],
            exit_condition=BooleanHook(
                condition=lambda state: ("END" == state.get_last_message().content.strip())
            ),
        ),
    ],
)

runner = CommandLineRunner(
    model=OpenAIChatCompletionModel(
        configuration=OpenAIModelConfiguration(
            api_key=os.environ.get("OPENAI_API_KEY", "")
        )
    ),
    parameters=OpenAIModelParameters(model_name="gpt-4"),
    template=template,
    user_interaction_provider=UserInteractionTextCLIProvider(),
)

runner.run()

You can talk with the agent on your console like below:

===== Start =====
From: 📝 system
message:  You're a math teacher bot.
=================
Let's ask a question to AI:
From: 👤 user
message:  Why can't you divide a number by zero?
=================
From: 🤖 assistant
message:  Dividing a number by zero is undefined in mathematics. Here's why:

Let's say we have a division operation a/b. This operation asks the question: "how many times does b fit into a?" If b is zero, the question becomes "how many times does zero fit into a?", and the answer is undefined because zero can fit into a an infinite number of times.

Moreover, if we look at the operation from the perspective of multiplication (since division is the inverse of multiplication), a/b=c means that b*c=a. If b is zero, there's no possible value for c that would satisfy the equation, because zero times any number is always zero, not a.

So, due to these reasons, division by zero is undefined in mathematics.
=================
From: 🤖 assistant
message:  Are you satisfied?
=================
Input:
From: 👤 user
message:  Yes.
=================
From: 🤖 assistant
message:  The user has stated their feedback.If you think the user is satisfied, you must answer `END`. Otherwise, you must answer `RETRY`.
=================
From: 🤖 assistant
message:  END
=================
====== End ======

Go to examples directory for more examples.

Tooling

You can use function calling! In function calling, you need to give LLM instructions to use the tool. Then, LLM give you the tool arguments and you need to give the result back to LLM. Therefore, you need:

• giving explanation by the way LLM can understand
• handling of multiple turn conversations
• validation of tool arguments given by LLM
• executing the function and return the result to LLM

PromptTrail handles all of these for you. You can define your own Tools to call and use them in your templates. Inherit Tool, ToolArgument, ToolResult and add type annotations. PromptTrail will automatically generate descriptions for LLM and let the LLM use the tool. Execution and validation is also handled by PromptTrail. Let's see a simple weather forecast tool as example:

class Place(ToolArgument):
    description: str = "The location to get the weather forecast"
    value: str

class TemperatureUnitEnum(enum.Enum):
    Celsius = "Celsius"
    Fahrenheit = "Fahrenheit"

class TemperatureUnit(ToolArgument):
    description: str = "The unit of temperature"
    value: Optional[TemperatureUnitEnum]

class WeatherForecastResult(ToolResult):
    temperature: int
    weather: str

    def show(self) -> Dict[str, Any]:
        return {"temperature": self.temperature, "weather": self.weather}

class WeatherForecastTool(Tool):
    name = "get_weather_forecast"
    description = "Get the current weather in a given location and date"
    argument_types = [Place, TemperatureUnit]
    result_type = WeatherForecastResult

    def _call(self, args: Sequence[ToolArgument], state: State) -> ToolResult:
        # Implement real API call here
        return WeatherForecastResult(temperature=0, weather="sunny")

template = LinearTemplate(
    templates=[
        MessageTemplate(
            role="system",
            content="You're an AI weather forecast assistant that help your users to find the weather forecast.",
        ),
        MessageTemplate(
            role="user",
            content="What's the weather in Tokyo tomorrow?",
        ),
        OpenAIGenerateWithFunctionCallingTemplate(
            role="assistant",
            functions=[WeatherForecastTool()],
        ),
    ]
)

The conversation will be like below:

===== Start =====
From: 📝 system
message:  You're an AI weather forecast assistant that help your users to find the weather forecast.
=================
From: 👤 user
message:  What's the weather in Tokyo tomorrow?
=================
From: 🤖 assistant
data:  {'function_call': {'name': 'get_weather_forecast', 'arguments': {'place': 'Tokyo', 'temperatureunit': 'Celsius'}}}
=================
From: 🧮 function
message:  {"temperature": 0, "weather": "sunny"}
=================
From: 🤖 assistant
message:  The weather in Tokyo tomorrow is expected to be sunny with a temperature of 0 degrees Celsius.
=================
====== End ======

See documentation) for more information.

Next

• Provide a way to export / import sessions
• Better error messages that help debugging
• More default tools
  • Vector Search Integration
  • Code Execution
• toml input/output for templates
• repository for templates
• job queue and server
• asynchronous execution (more complex runner)
• Local LLMs

File an issue if you have any requests!

License

• PromptTrail is licensed under the MIT License.

Contributing

• Contributions are more than welcome!
• See CONTRIBUTING for more details.

Q&A

Why bother yet another LLM library?

• PromptTrail is designed to be lightweight and easy to use.
• Manipulating LLM is actually not that complicated, but LLM libraries are getting more and more complex to embrace more features.
• PromptTrail aims to provide a simple interface for LLMs and let developers implement their own features.

Showcase

• If you build something with PromptTrail, please share it with us via Issues or Discussions!