To calculate token and translated USD cost of string and message calls to OpenAI, for example when used by AI agents
Project description
TokenCost
Overview
TokenCost is a specialized tool designed for calculating the USD cost of using major Large Language Models (LLMs) APIs.
Features
- LLM Price Tracking Major LLM providers frequently add new models and update pricing. This repo helps track the latest price changes
- Token counting Accurately count prompt tokens before sending OpenAI requests
- Easy integration Get the cost of a prompt or completion with a single function
Example usage:
from tokencost import calculate_prompt_cost, calculate_completion_cost
model = "gpt-3.5-turbo"
prompt = "Hello world"
completion = "How may I assist you today?"
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 30 + 140 = 170
# In TPUs (token price units), which is 1/10,000,000th of a USD.
Installation
Recommended: PyPI:
pip install tokencost
Usage
Counting tokens
from tokencost import count_message_tokens, count_string_tokens
message_prompt = [{ "role": "user", "content": "Hello world"}]
# Counting tokens in prompts formatted as message lists
print(count_message_tokens(message_prompt, model="gpt-3.5-turbo"))
# 9
# Alternatively, counting tokens in string prompts
print(count_string_tokens(prompt="Hello world", model="gpt-3.5-turbo"))
# 2
Cost estimates
Calculating the cost of prompts and completions from OpenAI requests
from openai import OpenAI
client = OpenAI()
model = "gpt-3.5-turbo"
prompt = [{ "role": "user", "content": "Say this is a test"}]
chat_completion = client.chat.completions.create(
messages=prompt, model=model
)
completion = chat_completion.choices[0].message.content
# "This is a test."
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 180 + 100 = 280
from tokencost import USD_PER_TPU
print(f"Cost USD: ${(prompt_cost + completion_cost)/USD_PER_TPU}")
# $2.8e-05
Calculating cost using string prompts instead of messages:
prompt_string = "Hello world"
response = "How may I assist you today?"
model= "gpt-3.5-turbo"
prompt_cost = calculate_prompt_cost(prompt_string, model)
print(f"Cost: ${prompt_cost/USD_PER_TPU}")
# Cost: $3e-06
Counting tokens
from tokencost import count_message_tokens, count_string_tokens
prompt = [{ "role": "user", "content": "Say this is a test"}]
prompt_message_token_count = count_message_tokens(prompt, model)
print(f"{prompt_message_token_count=}")
# prompt_message_token_count=12
completion_string_token_count = count_string_tokens(response, model)
print(f"{completion_string_token_count=}")
# completion_string_token_count=7
Cost table
Units denominated in TPUs (Token Price Units = 1/10,000,000 USD)
Model Name | Prompt Cost | Completion Cost | Max Prompt Tokens |
---|---|---|---|
gpt-3.5-turbo | 15 | 20 | 4097 |
gpt-3.5-turbo-0301 | 15 | 20 | 4097 |
gpt-3.5-turbo-0613 | 15 | 20 | 4097 |
gpt-3.5-turbo-16k | 30 | 40 | 16385 |
gpt-3.5-turbo-16k-0613 | 30 | 40 | 16385 |
gpt-3.5-turbo-1106 | 10 | 20 | 16385 |
gpt-3.5-turbo-instruct | 15 | 20 | 4096 |
gpt-4 | 300 | 600 | 8192 |
gpt-4-0314 | 300 | 600 | 8192 |
gpt-4-0613 | 300 | 600 | 8192 |
gpt-4-32k | 600 | 1200 | 32768 |
gpt-4-32k-0314 | 600 | 1200 | 32768 |
gpt-4-32k-0613 | 600 | 1200 | 32768 |
gpt-4-1106-preview | 100 | 300 | 128000 |
gpt-4-vision-preview | 100 | 300 | 128000 |
text-embedding-ada-002 | 1 | N/A | 8192 |
Running locally
Installation via GitHub:
git clone git@github.com:AgentOps-AI/tokencost.git
cd tokencost
pip install -e .
Running tests
- Install
pytest
if you don't have it already
pip install pytest
- Run the
tests/
folder while in the parent directory
pytest tests
This repo also supports tox
, simply run python -m tox
.
Contributing
Contributions to TokenCost are welcome! Feel free to create an issue for any bug reports, complaints, or feature suggestions.
License
TokenCost is released under the MIT License.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tokencost-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1d209b3c45003c42f9a1fce71f8222b493bc576ceb02414fec7bfd1a6c089cbc |
|
MD5 | 78d1e5423680f642aa9f6222be739752 |
|
BLAKE2b-256 | afae43a678ae05db9b76d1fa40644fbdc2984b69818c466b09c9351f7ffffb7d |