To calculate token and translated USD cost of string and message calls to OpenAI, for example when used by AI agents
Project description
TokenCost
Clientside token counting + price estimation for LLM apps and AI agents. Tokencost helps calculate the USD cost of using major Large Language Model (LLMs) APIs by calculating the estimated cost of prompts and completions.
Building AI agents? Check out AgentOps
Features
- LLM Price Tracking Major LLM providers frequently add new models and update pricing. This repo helps track the latest price changes
- Token counting Accurately count prompt tokens before sending OpenAI requests
- Easy integration Get the cost of a prompt or completion with a single function
Example usage:
from tokencost import calculate_prompt_cost, calculate_completion_cost
model = "gpt-3.5-turbo"
prompt = [{ "role": "user", "content": "Hello world"}]
completion = "How may I assist you today?"
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 0.0000135 + 0.000014 = 0.0000275
Installation
Recommended: PyPI:
pip install tokencost
Usage
Cost estimates
Calculating the cost of prompts and completions from OpenAI requests
from openai import OpenAI
client = OpenAI()
model = "gpt-3.5-turbo"
prompt = [{ "role": "user", "content": "Say this is a test"}]
chat_completion = client.chat.completions.create(
messages=prompt, model=model
)
completion = chat_completion.choices[0].message.content
# "This is a test."
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 0.0000180 + 0.000010 = 0.0000280
Calculating cost using string prompts instead of messages:
from tokencost import calculate_prompt_cost
prompt_string = "Hello world"
response = "How may I assist you today?"
model= "gpt-3.5-turbo"
prompt_cost = calculate_prompt_cost(prompt_string, model)
print(f"Cost: ${prompt_cost}")
# Cost: $3e-06
Counting tokens
from tokencost import count_message_tokens, count_string_tokens
message_prompt = [{ "role": "user", "content": "Hello world"}]
# Counting tokens in prompts formatted as message lists
print(count_message_tokens(message_prompt, model="gpt-3.5-turbo"))
# 9
# Alternatively, counting tokens in string prompts
print(count_string_tokens(prompt="Hello world", model="gpt-3.5-turbo"))
# 2
Cost table
Units denominated in USD. All prices can be located in model_prices.json
.
- Prices last updated Jan 30, 2024 from: https://openai.com/pricing and https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json
Model Name | Prompt Cost (USD) | Completion Cost (USD) | Max Prompt Tokens |
---|---|---|---|
gpt-4 | $0.00003000 | $0.00006000 | 8192 |
gpt-4-0314 | $0.00003000 | $0.00006000 | 8192 |
gpt-4-0613 | $0.00003000 | $0.00006000 | 8192 |
gpt-4-32k | $0.00006000 | $0.00012000 | 32768 |
gpt-4-32k-0314 | $0.00006000 | $0.00012000 | 32768 |
gpt-4-32k-0613 | $0.00006000 | $0.00012000 | 32768 |
gpt-4-1106-preview | $0.00001000 | $0.00003000 | 128000 |
gpt-4-0125-preview | $0.00001000 | $0.00003000 | 128000 |
gpt-4-vision-preview | $0.00001000 | $0.00003000 | 128000 |
gpt-3.5-turbo | $0.00000150 | $0.00000200 | 4097 |
gpt-3.5-turbo-0301 | $0.00000150 | $0.00000200 | 4097 |
gpt-3.5-turbo-0613 | $0.00000150 | $0.00000200 | 4097 |
gpt-3.5-turbo-1106 | $0.00000050 | $0.00000150 | 16385 |
gpt-3.5-turbo-0125 | $0.00000050 | $0.00000150 | 16385 |
gpt-3.5-turbo-16k | $0.00000300 | $0.00000400 | 16385 |
gpt-3.5-turbo-16k-0613 | $0.00000300 | $0.00000400 | 16385 |
text-embedding-ada-002 | $0.00000010 | $0.00000000 | 8191 |
text-embedding-3-small | $0.00000002 | $0.00000000 | 8191 |
text-embedding-3-large | $0.00000013 | $0.00000000 | 8191 |
azure/gpt-4-1106-preview | $0.00001000 | $0.00003000 | 128000 |
azure/gpt-4-0613 | $0.00003000 | $0.00006000 | 8192 |
azure/gpt-4-32k-0613 | $0.00006000 | $0.00012000 | 32768 |
azure/gpt-4-32k | $0.00006000 | $0.00012000 | 32768 |
azure/gpt-4 | $0.00003000 | $0.00006000 | 8192 |
azure/gpt-35-turbo-16k-0613 | $0.00000300 | $0.00000400 | 16385 |
azure/gpt-35-turbo-1106 | $0.00000150 | $0.00000200 | 16384 |
azure/gpt-35-turbo-16k | $0.00000300 | $0.00000400 | 16385 |
azure/gpt-35-turbo | $0.00000150 | $0.00000200 | 4097 |
azure/text-embedding-ada-002 | $0.00000010 | $0.00000000 | 8191 |
text-davinci-003 | $0.00000200 | $0.00000200 | 4097 |
text-curie-001 | $0.00000200 | $0.00000200 | 2049 |
text-babbage-001 | $0.00000040 | $0.00000040 | 2049 |
text-ada-001 | $0.00000040 | $0.00000040 | 2049 |
babbage-002 | $0.00000040 | $0.00000040 | 16384 |
davinci-002 | $0.00000200 | $0.00000200 | 16384 |
gpt-3.5-turbo-instruct | $0.00000150 | $0.00000200 | 8192 |
claude-instant-1 | $0.00000160 | $0.00000550 | 100000 |
mistral/mistral-tiny | $0.00000010 | $0.00000040 | 8192 |
mistral/mistral-small | $0.00000060 | $0.00000190 | 8192 |
mistral/mistral-medium | $0.00000270 | $0.00000820 | 8192 |
claude-instant-1.2 | $0.00000010 | $0.00000050 | 100000 |
claude-2 | $0.00000800 | $0.00002400 | 100000 |
claude-2.1 | $0.00000800 | $0.00002400 | 200000 |
text-bison | $0.00000010 | $0.00000010 | 8192 |
text-bison@001 | $0.00000010 | $0.00000010 | 8192 |
text-unicorn | $0.00001000 | $0.00002800 | 8192 |
text-unicorn@001 | $0.00001000 | $0.00002800 | 8192 |
chat-bison | $0.00000010 | $0.00000010 | 4096 |
chat-bison@001 | $0.00000010 | $0.00000010 | 4096 |
chat-bison@002 | $0.00000010 | $0.00000010 | 4096 |
chat-bison-32k | $0.00000010 | $0.00000010 | 32000 |
code-bison | $0.00000010 | $0.00000010 | 6144 |
code-bison@001 | $0.00000010 | $0.00000010 | 6144 |
code-gecko@001 | $0.00000010 | $0.00000010 | 2048 |
code-gecko@002 | $0.00000010 | $0.00000010 | 2048 |
code-gecko | $0.00000010 | $0.00000010 | 2048 |
codechat-bison | $0.00000010 | $0.00000010 | 6144 |
codechat-bison@001 | $0.00000010 | $0.00000010 | 6144 |
codechat-bison-32k | $0.00000010 | $0.00000010 | 32000 |
gemini-pro | $0.00000020 | $0.00000050 | 30720 |
gemini-pro-vision | $0.00000020 | $0.00000050 | 30720 |
palm/chat-bison | $0.00000010 | $0.00000010 | 4096 |
palm/chat-bison-001 | $0.00000010 | $0.00000010 | 4096 |
palm/text-bison | $0.00000010 | $0.00000010 | 8196 |
palm/text-bison-001 | $0.00000010 | $0.00000010 | 8196 |
palm/text-bison-safety-off | $0.00000010 | $0.00000010 | 8196 |
palm/text-bison-safety-recitation-off | $0.00000010 | $0.00000010 | 8196 |
command-nightly | $0.00001500 | $0.00001500 | 4096 |
command | $0.00001500 | $0.00001500 | 4096 |
command-light | $0.00001500 | $0.00001500 | 4096 |
command-medium-beta | $0.00001500 | $0.00001500 | 4096 |
command-xlarge-beta | $0.00001500 | $0.00001500 | 4096 |
openrouter/openai/gpt-3.5-turbo | $0.00000150 | $0.00000200 | 4095 |
openrouter/openai/gpt-3.5-turbo-16k | $0.00000300 | $0.00000400 | 16383 |
openrouter/openai/gpt-4 | $0.00003000 | $0.00006000 | 8192 |
openrouter/anthropic/claude-instant-v1 | $0.00000160 | $0.00000550 | 100000 |
openrouter/anthropic/claude-2 | $0.00001100 | $0.00003260 | 100000 |
openrouter/google/palm-2-chat-bison | $0.00000050 | $0.00000050 | 8000 |
openrouter/google/palm-2-codechat-bison | $0.00000050 | $0.00000050 | 8000 |
openrouter/meta-llama/llama-2-13b-chat | $0.00000020 | $0.00000020 | 4096 |
openrouter/meta-llama/llama-2-70b-chat | $0.00000150 | $0.00000150 | 4096 |
openrouter/meta-llama/codellama-34b-instruct | $0.00000050 | $0.00000050 | 8096 |
openrouter/nousresearch/nous-hermes-llama2-13b | $0.00000020 | $0.00000020 | 4096 |
openrouter/mancer/weaver | $0.00000560 | $0.00000560 | 8000 |
openrouter/gryphe/mythomax-l2-13b | $0.00000180 | $0.00000180 | 8192 |
openrouter/jondurbin/airoboros-l2-70b-2.1 | $0.00001380 | $0.00001380 | 4096 |
openrouter/undi95/remm-slerp-l2-13b | $0.00000180 | $0.00000180 | 6144 |
openrouter/pygmalionai/mythalion-13b | $0.00000180 | $0.00000180 | 4096 |
openrouter/mistralai/mistral-7b-instruct | $0.00000000 | $0.00000000 | 4096 |
j2-ultra | $0.00001500 | $0.00001500 | 8192 |
j2-mid | $0.00001000 | $0.00001000 | 8192 |
j2-light | $0.00000300 | $0.00000300 | 8192 |
dolphin | $0.00002000 | $0.00002000 | 4096 |
chatdolphin | $0.00002000 | $0.00002000 | 4096 |
luminous-base | $0.00003000 | $0.00003300 | 2048 |
luminous-base-control | $0.00003740 | $0.00004120 | 2048 |
luminous-extended | $0.00004500 | $0.00004940 | 2048 |
luminous-extended-control | $0.00005620 | $0.00006180 | 2048 |
luminous-supreme | $0.00017500 | $0.00019250 | 2048 |
luminous-supreme-control | $0.00021870 | $0.00024060 | 2048 |
ai21.j2-mid-v1 | $0.00001250 | $0.00001250 | 8191 |
ai21.j2-ultra-v1 | $0.00001880 | $0.00001880 | 8191 |
amazon.titan-text-lite-v1 | $0.00000030 | $0.00000040 | 8000 |
amazon.titan-text-express-v1 | $0.00000130 | $0.00000170 | 8000 |
anthropic.claude-v1 | $0.00000800 | $0.00002400 | 100000 |
bedrock/us-east-1/anthropic.claude-v1 | $0.00000800 | $0.00002400 | 100000 |
bedrock/us-west-2/anthropic.claude-v1 | $0.00000800 | $0.00002400 | 100000 |
bedrock/ap-northeast-1/anthropic.claude-v1 | $0.00000800 | $0.00002400 | 100000 |
bedrock/eu-central-1/anthropic.claude-v1 | $0.00000800 | $0.00002400 | 100000 |
anthropic.claude-v2 | $0.00000800 | $0.00002400 | 100000 |
bedrock/us-east-1/anthropic.claude-v2 | $0.00000800 | $0.00002400 | 100000 |
bedrock/us-west-2/anthropic.claude-v2 | $0.00000800 | $0.00002400 | 100000 |
bedrock/ap-northeast-1/anthropic.claude-v2 | $0.00000800 | $0.00002400 | 100000 |
bedrock/eu-central-1/anthropic.claude-v2 | $0.00000800 | $0.00002400 | 100000 |
anthropic.claude-v2:1 | $0.00000800 | $0.00002400 | 200000 |
bedrock/us-east-1/anthropic.claude-v2:1 | $0.00000800 | $0.00002400 | 100000 |
bedrock/us-west-2/anthropic.claude-v2:1 | $0.00000800 | $0.00002400 | 100000 |
bedrock/ap-northeast-1/anthropic.claude-v2:1 | $0.00000800 | $0.00002400 | 100000 |
bedrock/eu-central-1/anthropic.claude-v2:1 | $0.00000800 | $0.00002400 | 100000 |
anthropic.claude-instant-v1 | $0.00000160 | $0.00000550 | 100000 |
bedrock/us-east-1/anthropic.claude-instant-v1 | $0.00000080 | $0.00000240 | 100000 |
bedrock/us-west-2/anthropic.claude-instant-v1 | $0.00000080 | $0.00000240 | 100000 |
bedrock/ap-northeast-1/anthropic.claude-instant-v1 | $0.00000220 | $0.00000750 | 100000 |
bedrock/eu-central-1/anthropic.claude-instant-v1 | $0.00000240 | $0.00000830 | 100000 |
cohere.command-text-v14 | $0.00000150 | $0.00000200 | 4096 |
cohere.command-light-text-v14 | $0.00000030 | $0.00000060 | 4000 |
cohere.embed-english-v3 | $0.00000010 | $0.00000000 | 512 |
cohere.embed-multilingual-v3 | $0.00000010 | $0.00000000 | 512 |
meta.llama2-13b-chat-v1 | $0.00000070 | $0.00000100 | 4096 |
meta.llama2-70b-chat-v1 | $0.00000190 | $0.00000250 | 4096 |
sagemaker/meta-textgeneration-llama-2-7b | $0.00000000 | $0.00000000 | 4096 |
sagemaker/meta-textgeneration-llama-2-7b-f | $0.00000000 | $0.00000000 | 4096 |
sagemaker/meta-textgeneration-llama-2-13b | $0.00000000 | $0.00000000 | 4096 |
sagemaker/meta-textgeneration-llama-2-13b-f | $0.00000000 | $0.00000000 | 4096 |
sagemaker/meta-textgeneration-llama-2-70b | $0.00000000 | $0.00000000 | 4096 |
sagemaker/meta-textgeneration-llama-2-70b-b-f | $0.00000000 | $0.00000000 | 4096 |
together-ai-7.1b-20b | $0.00000040 | $0.00000040 | 1000 |
ollama/llama2 | $0.00000000 | $0.00000000 | 4096 |
ollama/llama2:13b | $0.00000000 | $0.00000000 | 4096 |
ollama/llama2:70b | $0.00000000 | $0.00000000 | 4096 |
ollama/llama2-uncensored | $0.00000000 | $0.00000000 | 4096 |
ollama/mistral | $0.00000000 | $0.00000000 | 8192 |
ollama/codellama | $0.00000000 | $0.00000000 | 4096 |
ollama/orca-mini | $0.00000000 | $0.00000000 | 4096 |
ollama/vicuna | $0.00000000 | $0.00000000 | 2048 |
deepinfra/meta-llama/Llama-2-70b-chat-hf | $0.00000070 | $0.00000090 | 4096 |
deepinfra/codellama/CodeLlama-34b-Instruct-hf | $0.00000060 | $0.00000060 | 4096 |
deepinfra/meta-llama/Llama-2-13b-chat-hf | $0.00000030 | $0.00000030 | 4096 |
deepinfra/meta-llama/Llama-2-7b-chat-hf | $0.00000020 | $0.00000020 | 4096 |
deepinfra/mistralai/Mistral-7B-Instruct-v0.1 | $0.00000020 | $0.00000020 | 4096 |
deepinfra/jondurbin/airoboros-l2-70b-gpt4-1.4.1 | $0.00000070 | $0.00000090 | 4096 |
perplexity/pplx-7b-chat | $0.00000000 | $0.00000000 | 8192 |
perplexity/pplx-70b-chat | $0.00000000 | $0.00000000 | 4096 |
perplexity/pplx-7b-online | $0.00000000 | $0.00050000 | 4096 |
perplexity/pplx-70b-online | $0.00000000 | $0.00050000 | 4096 |
perplexity/llama-2-13b-chat | $0.00000000 | $0.00000000 | 4096 |
perplexity/llama-2-70b-chat | $0.00000000 | $0.00000000 | 4096 |
perplexity/mistral-7b-instruct | $0.00000000 | $0.00000000 | 4096 |
perplexity/replit-code-v1.5-3b | $0.00000000 | $0.00000000 | 4096 |
anyscale/mistralai/Mistral-7B-Instruct-v0.1 | $0.00000010 | $0.00000010 | 16384 |
anyscale/HuggingFaceH4/zephyr-7b-beta | $0.00000010 | $0.00000010 | 16384 |
anyscale/meta-llama/Llama-2-7b-chat-hf | $0.00000010 | $0.00000010 | 4096 |
anyscale/meta-llama/Llama-2-13b-chat-hf | $0.00000020 | $0.00000020 | 4096 |
anyscale/meta-llama/Llama-2-70b-chat-hf | $0.00000100 | $0.00000100 | 4096 |
anyscale/codellama/CodeLlama-34b-Instruct-hf | $0.00000100 | $0.00000100 | 16384 |
cloudflare/@cf/meta/llama-2-7b-chat-fp16 | $0.00000190 | $0.00000190 | 3072 |
cloudflare/@cf/meta/llama-2-7b-chat-int8 | $0.00000190 | $0.00000190 | 2048 |
cloudflare/@cf/mistral/mistral-7b-instruct-v0.1 | $0.00000190 | $0.00000190 | 8192 |
cloudflare/@hf/thebloke/codellama-7b-instruct-awq | $0.00000190 | $0.00000190 | 4096 |
voyage/voyage-01 | $0.00000010 | $0.00000000 | 4096 |
voyage/voyage-lite-01 | $0.00000010 | $0.00000000 | 4096 |
Callback handlers
You may also calculate token costs in LLM wrapper/framework libraries using callbacks.
LlamaIndex
pip install `'tokencost[llama-index]'`
To use the base callback handler, you may import it:
from tokencost.callbacks.llama_index import BaseCallbackHandler
and pass to your framework callback handler.
Langchain
(Coming Soon)
Running locally
Installation via GitHub:
git clone git@github.com:AgentOps-AI/tokencost.git
cd tokencost
pip install -e .
Running tests
- Install
pytest
if you don't have it already
pip install pytest
- Run the
tests/
folder while in the parent directory
pytest tests
This repo also supports tox
, simply run python -m tox
.
Contributing
Contributions to TokenCost are welcome! Feel free to create an issue for any bug reports, complaints, or feature suggestions.
License
TokenCost is released under the MIT License.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tokencost-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 93880c9dc4bdc3725386747904a95cef229a740907d16a376af8e3d11f4af0bc |
|
MD5 | 8721ad77ab979c3795f0e365c1f3ab11 |
|
BLAKE2b-256 | 2de86a923cffe2ff7fe59719b46d79fb68767b208a3861dbccc6c46598a0a884 |