applang 0.1.1

A Prompt Programming Language

🍎APPL: A Prompt Programming Language

APPL is A Prompt Programming Language that extends Python to provide a Natural, Intuitive, Convenient, and Efficient (NICE) way to utilize Large Language Models (LLMs) such as GPT in your program.

Key Features

• Readability and maintainability via seamless integration with Python. APPL seamlessly embeds natural language prompts into Python programs, maintaining prompts' readability while inheriting modularity, reusability, dynamism and the ecosystem from the host programming language.
• Flexible prompt engineering. Except for allowing the utilization of Python control flows and the modularized decomposition of prompts, APPL offers prompt coding helpers to facilitate programming prompts in a modularized and maintainable way.
• Automatic parallelization via asynchronous computation. APPL schedules LLM calls asynchronously, leveraging potential independence among them to facilitate efficient parallelization. This offloads the burden of users to manage synchronization manually, with almost no extra work.
• Smooth tool calling integration. APPL provides intuitive ways to transform Python functions into tools that can be called by LLMs, making it easy for users to integrate existing Python libraries and functions with LLMs.
• Tracing and Failure Recovery. APPL traces the execution of LLM calls and supports recovery from failures, which is essential for debugging and error handling in the LLM programming paradigm.
• More Features. APPL also provides a unified interface for multiple LLM backends using litellm, structured generations using instructor, and many other features.

News

• [2024-07-12]: We have improved our tutorial. Please check them out for more detailed usage and examples.

Quick Start

Installation

You can simply install APPL from PyPI using pip:

pip install -U applang

More installation options can be found in the installation guide.

Setup

You need to set up API keys or your own LLM backends to interact with LLMs.

In this guide, we use OpenAI API as the default backend. You can set your OpenAI API key in the .env file in the root directory of your project:

OPENAI_API_KEY=<your openai api key>

or export it as an environment variable:

export OPENAI_API_KEY=<your openai api key>

For setting up other backends, enabling tracing and recovering from traces, please refer to the setup guide.

Hello World

To begin, let's create a simple function that uses LLM to respond to a greeting.

import appl
from appl import gen, ppl

appl.init()  # initialize APPL

@ppl  # the @ppl decorator marks the function as an `APPL function`
def greeting(name: str):
    f"Hello World! My name is {name}."  # Add text to the prompt
    return gen()  # call the default LLM with the current prompt

print(greeting("APPL"))  # call `greeting` as a normal Python function

The prompt for the generation is:

Hello World! My name is APPL.

The output will look like

Nice to meet you, APPL!

In this example, the @ppl decorator (@ stands for a here) marks the hello_world function as an APPL function. Within such a function, the standalone string f"Hello World! My name is {name}." is added to the prompt, and the gen() function calls LLM to generate responses using the current prompt.

Question Answering

Let's then implement a question-answering system using APPL. In this example, the APPL program answers multiple questions about a quotation by first extracting the author's name (inspired by this cookbook). Here is a runnable Colab notebook of this example.

import appl
from appl import AIRole, gen, ppl
from appl.const import NEWLINE

appl.init()

@ppl(ctx="copy")  # copy the context from caller
def get_answer(question: str):
    question  # append to the prompt
    return gen()  # return as a future object

@ppl  # marks APPL function
def answer_questions(quotation: str, questions: list[str]):
    "Extract the name of the author from the quotation below and answer questions."
    quotation  # append to the prompt
    with AIRole():  # assistant message
        f"The name of the author is {gen(stop=NEWLINE)}"  # specify the prefix
    return [get_answer(q) for q in questions]  # parallelize calls

quotation = '"Simplicity is the ultimate sophistication." -- Leonardo da Vinci'
questions = [
    "In what era did the author live?",
    # more questions can be added here
]
for ans in answer_questions(quotation, questions):
    print(ans)

The resulting conversation for the first question would look like (generated responses are in bold):

Role	Message
User	Extract the name of the author from the quotation below and answer questions. "Simplicity is the ultimate sophistication." -- Leonardo da Vinci
Assistant	The name of the author is Leonardo da Vinci.
User	In what era did the author live?
Assistant	Leonardo da Vinci lived during the Renaissance era.

In APPL functions, expression statements are captured as prompts based on the type of its value. Notably, the f-string is processed part by part, so the gen function inside the f-string intuitively uses the contents before that. In this example, The name of the author is serves as a prefix to guide the completion of the author's name.

After the author's name is extracted, the get_answer function is called multiple times in parallel to answer the questions, with the context being copied (detailed in context-management), demonstrating the automatic parallelization feature of APPL.

RoadMap

• Default to exclude """docstring""" from the prompt formation.
• Use FastAPI to build a server for inspecting the traces.
• Add more ... (contributions are welcome!)
  • Examples and tutorials to demonstrate the usage
  • Test cases to increase the coverage

Tutorial and Cookbook

For a more comprehensive tutorial, please refer to the tutorial.

Table of Contents

• Introduction
• Getting Started
• Example: QA with LMs
• APPL Function
• Concurrent LM Calls
• Tool Calls for LMs
• Prompt Coding Helpers
• Using Tracing

Cookbook

For more detailed usage and examples, please refer to the cookbook.

APPL can be used to reproduce some popular LM-based applications easily, such as:

• Wordware's TwitterPersonality[APPL implementation]: analyzes your tweets to determine your Twitter personality.

Citation and Acknowledgment

If you find APPL helpful, please consider citing our paper:

@article{dong2024appl,
  title={APPL: A Prompt Programming Language for Harmonious Integration of Programs and Large Language Model Prompts},
  author={Dong, Honghua and Su, Qidong and Gao, Yubo and Li, Zhaoyu and Ruan, Yangjun and Pekhimenko, Gennady and Maddison, Chris J and Si, Xujie},
  journal={arXiv preprint arXiv:2406.13161},
  year={2024}
}

We would like to thank the open-source community for their contributions, where we learned from or used these libraries in our project, including instructor, LiteLLM, LMQL, Guidance, SGLang and autogen.

License

This project is licensed under the terms of the MIT License.