A package to calculate the number of tokens in an image or folder
Project description
imagetoken
Utility to estimate the number of tokens in an image or directory of images for OpenAI models.
Install using pip
pip install image-token
Supported models
- gpt-4.1-mini
- gpt-4.1-nano
- gpt-4.1
- o4-mini
- gpt-4o
- gpt-4o-mini
- gemini-2.5-pro
- gemini-2.5-flash
- gemini-2.5-flash-lite
- gemini-2.5-flash-image-preivew
- gemini-2.0-flash
- gemini-2.0-flash-lite
- gemini-1.5-pro
- gemini-1.5-flash
Usage
To get the number of tokens for a single image
from image_token import get_token
num_tokens = get_token(model_name="gpt-4.1-mini", path=r"kitten.jpg")
To get the number of tokens for a directory of images
from image_token import get_token
num_tokens = get_token(model_name="gpt-4.1-mini", path=r"image_folder")
To get the number of tokens for a URL
from image_token import get_token
num_tokens = get_token(model_name="gpt-4.1-mini", path=r"https://raw.githubusercontent.com/srinathmkce/imagetoken/main/Images/kitten.jpeg")
To get the number of token for multiple URLS
for image_token import get_token
urls = ["https://raw.githubusercontent.com/srinathmkce/imagetoken/main/Images/kitten.jpeg"
,"https://raw.githubusercontent.com/srinathmkce/imagetoken/main/Images/kitten.jpg"
,"https://raw.githubusercontent.com/srinathmkce/imagetoken/main/Images/kitten.png"
]
num_tokens = get_token(model_name="gpt-4.1-mini",path=urls)
To get the estimated cost of generating text from an image or directory of images
from image_token import get_cost
cost = get_cost(model_name="gpt-4.1-nano", system_prompt_tokens=300 * 100, approx_output_tokens=100 * 100, path=r"image_folder")
Langchain integration
You can simulate the langchain OpenAI call and calculate the input token and cost
import base64
from pathlib import Path
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from image_token import simulate_image_token_cost
llm = ChatOpenAI(model="gpt-4.1-nano")
path = str(Path("tests") / "image_folder" / "kitten.jpg")
with open(path, "rb") as image_file:
image_bytes = image_file.read()
image_base64 = base64.b64encode(image_bytes).decode("utf-8")
image_data_url = f"data:image/jpeg;base64,{image_base64}"
messages = [
SystemMessage(content="You are a helpful assistant."),
HumanMessage(
content=[
{"type": "image_url", "image_url": {"url": image_data_url}},
],
),
]
result = simulate_image_token_cost(llm, messages)
print(result)
You can simulate the langchain OpenAI call and calculate the input token and cost using URL
import base64
from pathlib import Path
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage, SystemMessage
from image_token import simulate_image_token_cost
llm = ChatOpenAI(model="gpt-4.1-nano")
image_data_url = "https://raw.githubusercontent.com/srinathmkce/imagetoken/main/Images/kitten.jpeg"
messages = [
SystemMessage(content="You are a helpful assistant."),
HumanMessage(
content=[
{"type": "image_url", "image_url": {"url": image_data_url}},
],
),
]
result = simulate_image_token_cost(llm, messages)
print(result)
Note: simulate_image_token_cost mocks the LangChain OpenAI API and returns the result without making an actual request to the LangChain endpoint. You can simulate the call before executing llm.invoke(messages). The token count and cost calculated are based only on the input tokens. To get the accurate cost, make sure to include the output tokens as well.
Run unit tests
Perform test using poetry package manager
Add the package using the command
poetry add <package_name>
Install required packages using
poetry install
Create an .env file in the project root directory and set the openai key OPENAI_API_KEY
Run pytests with the following command
poetry run pytest -sv tests
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file image_token-0.1.6.tar.gz.
File metadata
- Download URL: image_token-0.1.6.tar.gz
- Upload date:
- Size: 25.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.11.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ce9072c22f0f691da48a308d6b2c1a54cc572b7574974cab1dfe51d8ea96cd02
|
|
| MD5 |
646e4a7ab71b4357b2bf5a56e234a30c
|
|
| BLAKE2b-256 |
982504ade5b251210a3ecddfc8a13db60e51a7d18d575b686bdc07083ca5f586
|
File details
Details for the file image_token-0.1.6-py3-none-any.whl.
File metadata
- Download URL: image_token-0.1.6-py3-none-any.whl
- Upload date:
- Size: 22.4 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.11.5
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
223796aab542156208c11f5771031b78ca42031d853a691f56e7cdbe1da6f2be
|
|
| MD5 |
6d279b81581e60a65b996af90dcbbbc6
|
|
| BLAKE2b-256 |
d797e819bfc5df692cedcec0712c0a50ab0052cd688b4c6473e92ab1f247dcc8
|