Skip to main content

Langchain Token Usage

Project description

Langchain Token Usage

This repository contains a python module to track the consumption of tokens of a Large Language Models (LLMs) within a LangChain application.

Installation

You can install LangChain Token Usage via pip:

pip install git+https://github.com/mrtj/langchain_token_usage.git

or in requirements.txt:

langchain-token-usage @ git+https://github.com/mrtj/langchain_token_usage.git

Usage

Token usage tracking is implemented as a LangChain Callback so it is easy to integrate in any LangChain application. The collection of the token usage metrics is LLM specific: currently only OpenAI LLMs are supported.

The metrics collected can be processed locally, or sent to a metrics repository via a Reporter. Current implementation includes sending the metrics to Amazon CloudWatch service.

If you wish to use CloudWatch, you should ensure that you have configured your boto3 client to credentials that are associated to an IAM Role that can write data to CloudWatch Metrics.

Example usage:

from langchain.chains import LLMChain
from langchain.chat_models import ChatOpenAI
from langchain.prompts import PromptTemplate

from langchain_token_usage.handlers import OpenAITokenUsageCallbackHandler
from langchain_token_usage.reporters import CloudWatchTokenUsageReporter

reporter = CloudWatchTokenUsageReporter(
    namespace="openai_token_usage",
    dimensions={"project": "my_test_project"}
)
handler = OpenAITokenUsageCallbackHandler(reporter)

llm = ChatOpenAI(model="gpt-4", callbacks=[handler])
prompt = PromptTemplate.from_template("1 + {number} = ")

chain = LLMChain(llm=llm, prompt=prompt)
chain.run(number=2)

The above code will perform a single call to an OpenAI GPT-4 backed LLM model, and send token usage metrics and cost estimation to CloudWatch. You can define the desired namespace of your CloudWatch metrics, and add arbitrary metrics dimensions as str -> str mapping.

Contributing

Contributions are very welcome. To learn more, see the Contributor Guide.

License

Distributed under the terms of the MIT license, Langchain Token Usage is free and open source software.

Issues

If you encounter any problems, please file an issue along with a detailed description.

Credits

This project was generated from @cjolowicz's Hypermodern Python Cookiecutter template.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

langchain_token_usage-0.1.1.tar.gz (9.4 kB view details)

Uploaded Source

Built Distribution

langchain_token_usage-0.1.1-py3-none-any.whl (11.5 kB view details)

Uploaded Python 3

File details

Details for the file langchain_token_usage-0.1.1.tar.gz.

File metadata

  • Download URL: langchain_token_usage-0.1.1.tar.gz
  • Upload date:
  • Size: 9.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.7.0 CPython/3.11.5 Darwin/21.6.0

File hashes

Hashes for langchain_token_usage-0.1.1.tar.gz
Algorithm Hash digest
SHA256 d8380eb0556536f819a0ad2ef76037f8da309a514fbffd96265f0628104e409a
MD5 1d7ce12de00d9b98fd8aa4926544b4ea
BLAKE2b-256 6936a6653e8ed0e8a7669cdd76e88f48eaaa4ddda8fce3952cf263d12a95954c

See more details on using hashes here.

File details

Details for the file langchain_token_usage-0.1.1-py3-none-any.whl.

File metadata

File hashes

Hashes for langchain_token_usage-0.1.1-py3-none-any.whl
Algorithm Hash digest
SHA256 22f3adee1057ac29c1cae163e155a4ab4ca33bd51453b6a57579eafd768ec754
MD5 9d0902446db23481220f3d0fb4b3bef1
BLAKE2b-256 db236b0f9f7271b8b7d43abead39eb1d7282bd0ce040bb3ba2e6efa700666397

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page