LangGraph Compare
Documentation
Documentation is available at: https://serafinski.github.io/LangGraph-Compare/
Purpose
This Python package facilitates the parsing of run logs generated by LangGraph. During execution, logs are stored in an SQLite database in an encoded format (using msgpack). These logs are then decoded and exported to a json format. Subsequently, the json files are transformed into csv files for further analysis.
Once in csv format, the data can be analyzed using methods from the py4pm library. These methods calculate specific statistics related to the multi-agent infrastructure's performance and enable visualizations of the process behavior and execution flow.
This pipeline provides a streamlined approach for extracting, transforming, and analyzing logs, offering valuable insights into multi-agent systems.
Installation
This package requires Python 3.9 or higher. Check below for more information on creating environment.
If you would like to develop this package, use poetry with Python 3.10 - since 3.10 is the needed minimum by Sphinx. Install needed dependencies with:
poetry install --with dev,test,docs
Prerequisites
This package requires Graphviz to be installed on your system.
Windows
Download the Graphviz installer from the Graphviz website.
macOS
Install Graphviz using Homebrew:
brew install graphviz
Linux
For Debian, Ubuntu, use the following command:
sudo apt-get install graphviz
For Fedora, Rocky Linux, RHEL or CentOS use the following command:
sudo dnf install graphviz
Environment setup
To create virtual environment (using conda), use the following commands:
conda create -n langgraph_compare python=3.9
conda activate langgraph_compare
pip install langgraph_compare
Basic Example
This example is based on the Building a Basic Chatbot from LangGraph documentation.
It will require You to install the following packages (besides langgraph_compare):
pip install python-dotenv langchain-openai
Example:
from dotenv import load_dotenv
from typing import Annotated
from typing_extensions import TypedDict
from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langgraph_compare import *
exp = create_experiment("main")
memory = exp.memory
load_dotenv()
class State(TypedDict):
messages: Annotated[list, add_messages]
graph_builder = StateGraph(State)
llm = ChatOpenAI(model="gpt-4o-mini")
def chatbot(state: State):
return {"messages": [llm.invoke(state["messages"])]}
graph_builder.add_node("chatbot_node", chatbot)
graph_builder.add_edge(START, "chatbot_node")
graph_builder.add_edge("chatbot_node", END)
graph = graph_builder.compile(checkpointer=memory)
print()
run_multiple_iterations(graph, 1, 5, {"messages": [("user", "Tell me a joke")]})
print()
graph_config = GraphConfig(
nodes=["chatbot_node"]
)
prepare_data(exp, graph_config)
print()
event_log = load_event_log(exp)
print_analysis(event_log)
print()
generate_artifacts(event_log, graph, exp)
When You have multiple architectures analyzed, You can use the following code to compare them (by default, it will look in experiments directory):
from langgraph_compare import compare
infrastructures = ["main", "other1", "other2"]
compare(infrastructures)
This should generate a file in a comparison_reports directory, with the name: main_vs_other1_vs_other2.html.
Metadata
Release files for langgraph-compare 0.1.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| langgraph_compare-0.1.4.tar.gz | 46.0 kB | Without provenance | Details for langgraph_compare-0.1.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| langgraph_compare-0.1.4-py3-none-any.whl | Python 3 | none | any | Details for langgraph_compare-0.1.4-py3-none-any.whl |
Total release size: 98.6 kB
Release files / langgraph_compare-0.1.4.tar.gz
| Download URL | langgraph_compare-0.1.4.tar.gz |
|---|---|
| Size | 46.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ea1b6a5317692209feb43910899b709e874ede7e65ffe4fe6128566803b39b08
|
|
BLAKE2b-256 checksum How to use checksums |
f6f8827618ff65c592f8264b4c91a1ff19b981d4e6c24efe69d05c0ccb9e18fa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.4 CPython/3.8.10 Windows/10
|
Release files / langgraph_compare-0.1.4-py3-none-any.whl
| Download URL | langgraph_compare-0.1.4-py3-none-any.whl |
|---|---|
| Size | 52.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9bdd2eb9f38d942dd14ff6dadb5cccd1998fcd3d8d89c1c2537fb8c987e4e90e
|
|
BLAKE2b-256 checksum How to use checksums |
ae8e193b75ddd044a18f472ec84397a2460b3eeac09cf476cd2db89d5653129b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
poetry/1.8.4 CPython/3.8.10 Windows/10
|
Security for langgraph-compare 0.1.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.