llfn 0.1.3

No project description provided

LLFn (read: 🐘 Elephant) is a light-weight framework for creating applications using LLMs.
Build anything from a simple text-summarizer to complex AI agents with one common primitive: function.

Core Concept · Installation · Examples · Report Bug · Request Feature

Features

LLFn has only one goal: making it 100x easier to experiment and ship AI applications using the LLMs with minimum boilerplate. Even if you're new to LLMs, you can still get started and be productive in minutes, instead of spending hours learning how to use the framework.

• 🔋 Functionify: turning any prompt into a callable function.
• 📤 Flexible Output: defining LLM result with Python and pydantic types.
• 🧱 Infinitely Composable: any function can call any other function in any order.
• 🛒 Use Any LLM: leveraging LangChain's LLM interface, drop in your favorite LLM with 100% compatibility.
• 🪶 Light-Weight: small core framework of LLFn making it extremely easy to understand and extend.

We draw a lot of inspiration from both FastAPI and LangChain, so if you're familiar with either of them you'd feel right at home with LLFn.

Core Concept: Function is All You Need

⭐️ Defining Your Function

The primary goal of LLFn is to encapsulate everything you do with LLMs into a function. Each function comprises of 3 components: input, prompt, and output. Here's how you can create a simple text translator using LLFn.

from llfn import LLFn

function_prompt = LLFn()

@function_prompt
def translate(text: str, output_language: str) -> str:
    return f"Translate the following text into {output_language} language: {text}"

🥪 Binding It with LLM

To execute this you have to bind the function to an LLM. You can bind any LangChain's supported language models to LLFn functions.

from langchain.chat_models import ChatOpenAI

# Setup LLM and bind it to the function
llm = ChatOpenAI(temperature=0.7, openai_api_key=API_KEY)
translate.bind(llm)

If you don't want to repeat yourself binding all the functions individually, you can bind directly to function_prompt as well.

# Alternatively: bind the llm to all functions
function_prompt.bind(llm)

🍺 Calling the Function

Now you can call your function like you would for any Python function.

translate("Hello welcome. How are you?", "Thai")

The beauty of this construct is that you're able to infinitely compose your applications. LLFn does not make any assumption on how you'd chain up your application logic. The only requirement for a function_prompt is that it returns a prompt string at the end.

And ... that's it! That's just about everything you need to learn about LLFn to start building AI apps with it.

👆 Click to see the full code

import os
from llfn import LLFn
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(
    temperature=0.7,
    model=os.getenv("OPENAI_MODEL"),
    openai_api_key=os.getenv("OPENAI_API_KEY"),
)

function_prompt = LLFn()
function_prompt.bind(llm)

@function_prompt
def translate(text: str, output_language: str) -> str:
    return f"Translate the following text into {output_language} language: {text}"

# Call the function
print(translate("Hello welcome. How are you?", "Thai"))

Installation

LLFn is available on PyPi. You can install LLFn using your favorite Python package management software.

$ pip install llfn # If you use pip
$ poetry add llfn # If you use poetry

Advanced Features

While LLFn is simple and light-weight by design, it does pack some punches above its weight that helps you write AI applications more intuitively.

📐 Structured Function Output

You can create a function to automatically format the response into a desired pydantic object. To do this, you simply declare the output of your function in pydantic, and let the LLFn do the heavy lifting for you.

from pydantic import BaseModel, Field

class AgentLogicOutput(BaseModel):
  tasks: List[str] = Field(..., description="list of tasks to accomplish the given goal")
  reasoning: str  = Field(..., description="why you choose to do these tasks")

@function_prompt
def agent_think(goal: str) -> AgentLogicOutput:
    return f"You're an agent planning 5 tasks to accomplish the following goal: {goal}"

agent_think("Creating online business") # this returns AgentLogicOutput object

This allows for powerful composability like tools, agents, etc. without sacrificing the simplicity and verbosity of our program. Under the hood LLFn injects the output format into your prompt and automatically parse the LLM result into the specified format.

🗝️ Binding and Overriding LLMs

LLFn let you use both LangChain's LLMs and Chat Models to power your functions. There are 2 ways to assign it.

Method 1: Binding LLM to All Your Functions

This is the most convenient way to just get your function to work.

from llfn import LLFn
from langchain.chat_models import ChatOpenAI

llm = ChatOpenAI(temperature=0.7, openai_api_key=API_KEY)

function_prompt = LLFn()
function_prompt.bind(llm)

@function_prompt
def your_function(...):
    ...

Method 2: Binding/Overriding LLM to A Specific Function

If you want to use different LLMs for different tasks, you can do so by binding llm to your function individually.

from langchain.llms import LlamaCpp

llm2 = LlamaCpp(...)

your_function.bind(llm2) # This will override the LLM for this function

🏄‍♂️ Callbacks and Streaming

Callbacks

With LLFn, it's trivial to define callbacks in between execution. Because everything is a function, you can print or call any 3rd party APIs anywhere you want.

For example, if you want to print results of executions in between a series of complex AI agent it would look like this.

...

# Assuming that other functions are already defined with function_prompt
@function_prompt
def agent_plan_and_execute(goal: str) -> str:
    tasks = agent_think(goal)
    print(tasks) # Debug
    results = agent_execute_tasks(tasks)
    print(results) # Debug
    evaluation = agent_evaluate_perf(goal, results)
    print(evaluation) # Debug

Your function can also take a callback function as a parameter. LLFn does not dictate how you should do it. Anything that works for you would do!

Streaming LLM Output

Because LLFn leverages LangChain's LLMs, you can use LangChain Streaming directly.

from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler

llm = ChatOpenAI(
    streaming=True,
    callbacks=[StreamingStdOutCallbackHandler()],
    ...
)

function_prompt.bind(llm)

Examples

Coming soon

Sponsors and Contributors

We currently don't take any monetary donations! However, every issue filed and PR are extremely important to us. Here is the roster of contributors and supporters of the project.

    

Contribution and Feedback

Contributions, feedback, and suggestions are highly encouraged and appreciated! You can contribute to LLFn in the following ways:

• 🔀 Fork the repository and make modifications in your own branch. Then, submit a pull request (PR) with your changes.
• ⬇️ Submit issues (GitHub Issues) to report bugs, suggest new features, or ask questions.

Citation

If you wish to cite LLFn in your research, we encourage the use of CITATION.cff provided for appropriate citation formatting. For more details on the citation file format, please visit the Citation File Format website.

License

LLFn is an open-source software under the MIT license. This license allows use, modification, and distribution of the software. For complete details, please see the LICENSE file in this repository.

Acknowledgements

We would like to express our gratitude to the following projects and communities for their inspiring work and valuable contributions:

• LangChain
• Pydantic
• FastAPI
• OpenAI