Langchain Token Usage
Project description
Langchain Token Usage
This repository contains a python module to track the consumption of tokens of a Large Language Models (LLMs) within a LangChain application.
Installation
You can install LangChain Token Usage via pip:
pip install git+https://github.com/mrtj/langchain_token_usage.git
or in requirements.txt:
langchain-token-usage @ git+https://github.com/mrtj/langchain_token_usage.git
Usage
Token usage tracking is implemented as a LangChain Callback so it is easy to integrate in any LangChain application. The collection of the token usage metrics is LLM specific: currently only OpenAI LLMs are supported.
The metrics collected can be processed locally, or sent to a metrics repository via a Reporter. Current implementation includes sending the metrics to Amazon CloudWatch service.
If you wish to use CloudWatch, you should ensure that you have configured your boto3 client to credentials that are associated to an IAM Role that can write data to CloudWatch Metrics.
Example usage:
from langchain.chains import LLMChain
from langchain.chat_models import ChatOpenAI
from langchain.prompts import PromptTemplate
from langchain_token_usage.handlers import OpenAITokenUsageCallbackHandler
from langchain_token_usage.reporters import CloudWatchTokenUsageReporter
reporter = CloudWatchTokenUsageReporter(
namespace="openai_token_usage",
dimensions={"project": "my_test_project"}
)
handler = OpenAITokenUsageCallbackHandler(reporter)
llm = ChatOpenAI(model="gpt-4", callbacks=[handler])
prompt = PromptTemplate.from_template("1 + {number} = ")
chain = LLMChain(llm=llm, prompt=prompt)
chain.run(number=2)
The above code will perform a single call to an OpenAI GPT-4 backed LLM model, and send token usage metrics and cost estimation to CloudWatch. You can define the desired namespace of your CloudWatch metrics, and add arbitrary metrics dimensions as str -> str
mapping.
Contributing
Contributions are very welcome. To learn more, see the Contributor Guide.
License
Distributed under the terms of the MIT license, Langchain Token Usage is free and open source software.
Issues
If you encounter any problems, please file an issue along with a detailed description.
Credits
This project was generated from @cjolowicz's Hypermodern Python Cookiecutter template.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for langchain_token_usage-0.1.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | d8380eb0556536f819a0ad2ef76037f8da309a514fbffd96265f0628104e409a |
|
MD5 | 1d7ce12de00d9b98fd8aa4926544b4ea |
|
BLAKE2b-256 | 6936a6653e8ed0e8a7669cdd76e88f48eaaa4ddda8fce3952cf263d12a95954c |
Hashes for langchain_token_usage-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 22f3adee1057ac29c1cae163e155a4ab4ca33bd51453b6a57579eafd768ec754 |
|
MD5 | 9d0902446db23481220f3d0fb4b3bef1 |
|
BLAKE2b-256 | db236b0f9f7271b8b7d43abead39eb1d7282bd0ce040bb3ba2e6efa700666397 |