Skip to main content

Orchestrate and observe tools built with langchain.

Project description

langchain-prefect

PyPI

Read the accompanying article for background

Orchestrate and observe langchain using Prefect

Large Language Models (LLMs) are interesting and useful  -  building apps that use them responsibly feels like a no-brainer. Tools like Langchain make it easier to build apps using LLMs. We need to know details about how our apps work, even when we want to use tools with convenient abstractions that may obfuscate those details.

Prefect is built to help data people build, run, and observe event-driven workflows wherever they want. It provides a framework for creating deployments on a whole slew of runtime environments (from Lambda to Kubernetes), and is cloud agnostic (best supports AWS, GCP, Azure). For this reason, it could be a great fit for observing apps that use LLMs.

Features

  • RecordLLMCalls is a ContextDecorator that can be used to track LLM calls made by Langchain LLMs as Prefect flows.

Call an LLM and track the invocation with Prefect:

from langchain.llms import OpenAI
from langchain_prefect.plugins import RecordLLMCalls

with RecordLLMCalls():
    llm = OpenAI(temperature=0.9)
    text = (
        "What would be a good company name for a company that makes colorful socks?"
    )
    llm(text)

and a flow run will be created to track the invocation of the LLM:

LLM invocation UI

Run several LLM calls via langchain agent as Prefect subflows:

from langchain.agents import initialize_agent, load_tools
from langchain.llms import OpenAI

from prefect import flow

llm = OpenAI(temperature=0)
tools = load_tools(["llm-math"], llm=llm)
agent = initialize_agent(tools, llm)

@flow
def my_flow():
    agent.run(
        "How old is the current Dalai Lama? "
        "What is his age divided by 2 (rounded to the nearest integer)?"
    )

with RecordLLMCalls(tags={"agent"}):
    my_flow()

LLM agent UI

Find more examples here.

How do I get a Prefect UI?

  • The easiest way is to use the Prefect Cloud UI for free. You can find details on getting setup here.

  • If you don't want to sign up for cloud, you can use the dashboard locally by running prefect server start in your terminal - more details here.

Resources

Installation

pip install langchain-prefect

Requires an installation of Python 3.10+.

Feedback

If you encounter any bugs while using langchain-prefect, feel free to open an issue in the langchain-prefect repository.

Feel free to star or watch langchain-prefect for updates too!

Contributing

If you'd like to help contribute to fix an issue or add a feature to langchain-prefect, please propose changes through a pull request from a fork of the repository.

Here are the steps:

  1. Fork the repository
  2. Clone the forked repository
  3. Install the repository and its dependencies:
pip install -e ".[dev]"
  1. Make desired changes
  2. Add tests
  3. Insert an entry to CHANGELOG.md
  4. Install pre-commit to perform quality checks prior to commit:
pre-commit install
  1. git commit, git push, and create a pull request

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

langchain-prefect-0.0.3.tar.gz (33.8 kB view details)

Uploaded Source

Built Distribution

langchain_prefect-0.0.3-py3-none-any.whl (14.4 kB view details)

Uploaded Python 3

File details

Details for the file langchain-prefect-0.0.3.tar.gz.

File metadata

  • Download URL: langchain-prefect-0.0.3.tar.gz
  • Upload date:
  • Size: 33.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.1 CPython/3.11.4

File hashes

Hashes for langchain-prefect-0.0.3.tar.gz
Algorithm Hash digest
SHA256 cd8528a26c6460944fb406a7b9c646242fa0b99f8f46504db0777295d46c5cf1
MD5 b79e90068f81739b29cb21991252101c
BLAKE2b-256 111ed72c8b8146ff92db235a8e962769ef5c122908f3ac2ab8dafcbd60b976d4

See more details on using hashes here.

File details

Details for the file langchain_prefect-0.0.3-py3-none-any.whl.

File metadata

File hashes

Hashes for langchain_prefect-0.0.3-py3-none-any.whl
Algorithm Hash digest
SHA256 dcf0f4e21c31d55a9fe41dd6e3c3fd55bd51ea4308f80c34cc4a82d76c4b73b7
MD5 75b35e5e265bdbef38f06c0b6533cde7
BLAKE2b-256 8583330cef6b1271c084246505f69caa6dcd9f0c72e70c5d20bc7282926e425b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page