ai-tally (Python SDK)

The Python SDK for AI Tally. It records what each AI call in your app costs, which feature made the call, and which of your customers it was for. By default it sends only counts, costs and hashed ids. Your prompts and answers stay in your app.

pip install ai-tally

The SDK has no dependencies, and recording never throws an error into your code. If something inside it fails, it logs the problem and your call goes ahead as normal.

Full documentation: https://ai-tally.com/docs/connect/python-sdk

Quick start

Call aitally.init() once when your app starts. Pass the ingest key from the dashboard's Setup page, or set it as TALLY_KEY.

import aitally

aitally.init("tally_sk_live_...")

# Your existing calls are now recorded. You don't change them:
client.chat.completions.create(model="gpt-4o-mini", messages=[...])     # openai
client.messages.create(model="claude-sonnet-4-5", messages=[...])       # anthropic
client.models.generate_content(model="gemini-2.5-flash", contents=...)  # google-genai
bedrock.converse(modelId="us.amazon.nova-pro-v1:0", messages=[...])     # boto3

init() is safe to call more than once, and it doesn't wait on the network. If the key is missing or the address is refused, it logs one warning and records nothing. If the key is wrong or AI Tally is down, your calls still run normally and the SDK logs a warning.

What gets recorded on its own

The SDK only touches a library you already have installed. It doesn't install any of them.

Anything else needs one line after the call, aitally.record_llm_call(...). That includes Bedrock invoke_model, the old google-generativeai package, and any other client. For LiteLLM, see the LiteLLM section below.

Good to know:

• Choose which libraries to record. aitally.init(key, instrument={"openai", "anthropic"}) records only those two. The names are "openai", "anthropic", "google" and "bedrock". instrument=False turns all of them off.
• Other providers through the openai library. Calls to Groq, Fireworks, Together, DeepSeek, xAI, OpenRouter, Mistral, Azure or Gemini through the openai library are labeled with the real provider, never as OpenAI. A provider AI Tally has no price for shows a blank cost, not a guess. A host the SDK doesn't recognize is labeled openai-compatible and its cost is blank.
• Streaming with OpenAI. OpenAI only reports token counts on a stream if you ask for them with stream_options={"include_usage": True}. Without that, the cost shows as blank. Pass aitally.init(..., instrument_stream_usage=True) and the SDK asks for you.
• Bedrock streams are recorded once your code has read the stream to the end.
• Bedrock inference profiles and provisioned models. If the model can't be read from the ARN (an application inference profile or a provisioned model), the call is recorded with no model and a blank cost.
• Other kinds of cost are one line each: aitally.record_vector_call for vector search, aitally.record_tool_call for your own tool calls, and aitally.record_embedding_call for embeddings not made through a recorded client. They do nothing until init() has run.
• Before your process exits, the SDK sends what it has buffered. Call aitally.flush() to send it sooner. aitally.uninstrument() undoes everything, which helps in tests.
• Calls through the AI Tally proxy are recorded by the proxy, so the SDK skips them and nothing is counted twice.

Remove manual lines you no longer need

If you added aitally.record_llm_call(...) after a Gemini or Bedrock converse call, delete it. init() records those calls now. Until you do, the SDK notices most duplicates (same call, same thread or async task, a few seconds apart), drops the copy and warns you once. Don't rely on that: it can't catch a manual line that runs in a different thread or task, for example under asyncio.gather.

The connection must be secure

The SDK sends your key with every request, so it only connects over https://. Plain http:// works only for your own machine (such as localhost or 127.0.0.1), so local testing still works. To use http:// for anything else, such as another machine on your network, you have to say so: aitally.init(key, endpoint=..., allow_insecure_http=True), or TALLY_ALLOW_INSECURE_HTTP=1.

If the address is refused, the SDK logs one warning and records nothing. It also doesn't follow redirects, so give it the final address.

Cost per customer

Tell the SDK which of your customers a call is for, and AI Tally can show cost per customer. Set the customer once, for example in your request middleware, and every call inside picks it up:

import aitally

aitally.init("tally_sk_live_...")

with aitally.start_trace(feature_tag="support-bot"), aitally.with_account("acct_northwind"):
    client.chat.completions.create(model="gpt-4o", messages=[...])   # recorded for acct_northwind
    aitally.record_tool_call(provider="openai", tool="web_search")

A batch job that works through many customers can pass the customer on each call instead:

import aitally
from aitally.pricing import Usage

with aitally.start_trace():
    for account in accounts:
        aitally.record_llm_call(
            provider="openai", model="gpt-4o",
            usage=Usage(input_tokens=1200, output_tokens=300),
            account_id=account,
        )

with_account(None) clears the customer for a block, for background work that shouldn't inherit it.

Your customer ids stay in your app. The SDK replaces each id with a hash (HMAC-SHA256) made with a key that belongs to your organization only. AI Tally sees the hash, never the id, and the same id gives a different hash for every organization. The SDK first fetches this hashing key when init() runs. Calls made before it arrives are recorded without a customer. If that first fetch fails, calls are still recorded but without a customer until the app restarts.

You can add a readable name so the dashboard shows "Northwind Traders" instead of a short hash:

with aitally.with_account("acct_northwind", label="Northwind Traders"):
    ...

The name is kept apart from your usage data. Names are optional.

Hashing an id yourself

If you use the AI Tally proxy instead of the SDK, send the customer as a hash in the X-Tally-Account-Id-Hash header. Make the hash on your machine with the same key:

from aitally import hash_account
h = hash_account("acct_northwind")

python -m aitally.hash_account acct_northwind   # reads TALLY_KEY

Unlike recording, hash_account raises an error if it can't get the hashing key.

Command line

The SDK comes with a tally command. It uses the same key as aitally.init().

tally login                    # saves your key (not shown on screen) to ~/.config/tally/config.ini
tally health                   # what your key can do, and model calls in the last 24 hours
tally keys                     # your organization's keys, without the secrets (needs a write or admin key)
tally test-event               # sends one empty test event and shows what arrived
tally spend --month 2026-10    # that month's spend by feature and provider

TALLY_KEY overrides the saved key, so CI needs no config file. Every read command takes --json. The key is never printed. A number AI Tally couldn't work out shows as unknown, never 0. If some calls in a month have no price, the total shows as "at least".

LiteLLM

If you call models through LiteLLM, install the extra and add the AI Tally logger:

pip install "ai-tally[litellm]"

import litellm
import aitally
from aitally.integrations.litellm import TallyLiteLLMLogger

aitally.init("tally_sk_live_...", instrument=False)
litellm.callbacks = [TallyLiteLLMLogger()]

litellm.completion(
    model="gemini/gemini-2.5-flash",
    messages=[...],
    litellm_metadata={
        "tally_account": customer_id,          # hashed before it leaves your app
        "tally_feature": "title_search.main",  # shows as the feature
        "tally_session": session_id,           # optional
        "tally_account_label": "Northwind",    # optional, readable name
    },
)

• Use instrument=False. LiteLLM sometimes uses the openai library, so without it a call could be counted twice. With it, direct provider calls outside LiteLLM aren't recorded.

• Use litellm_metadata, not metadata. LiteLLM sends metadata on to the provider in some cases. It never sends litellm_metadata.

• Every call needs tally_account. Without it the cost counts toward your total but not toward any customer. with_account doesn't apply to LiteLLM calls.

• Streams need stream_options={"include_usage": True}, except for Anthropic and Gemini. Without it the cost is blank.

• Shut down before you exit. A normal script is fine. In an async script, await ashutdown() before asyncio.run(main()) returns, or the last calls are lost:

  from aitally.integrations.litellm import ashutdown, shutdown
  
  shutdown(timeout_s=5)         # batch jobs, after your event loop has finished
  await ashutdown(timeout_s=5)  # inside a running loop, such as a server's shutdown hook
  

• Supported providers. Calls that aren't streamed get token counts only for OpenAI, Anthropic, Gemini (the gemini/ route, not Vertex) and Mistral. Non-streamed calls to other providers show a blank cost.

• The extra installs LiteLLM 1.105.1, the version the logger was tested with. On other versions it still runs, but token counts come out blank.

Keeping replay samples on your own server

To try other models on your real traffic without sending your prompts to AI Tally, the SDK can save a small sample of calls on your own server:

aitally.init(replay_capture="local", replay_dir="/var/lib/tally/replay")

It saves only while an admin has turned on local capture for your organization. By default it saves 1 in 100 calls that aren't streamed (replay_sample_rate) and keeps them for 7 days (replay_retention_days). Nothing is sent to AI Tally. The files hold real prompts and answers, so treat that folder like any other customer data.

The SDK only deletes its own sample files, in a folder it marked as its own. If an admin turns local capture off, the SDK deletes the samples. To delete them yourself:

tally eval purge --samples /var/lib/tally/replay                      # everything
tally eval purge --samples /var/lib/tally/replay --before 2026-10-01  # older days only

Or from code: aitally.purge_local_samples(replay_dir, before=...).

Comparing models on those samples

pip install openai anthropic   # the providers you want to test
tally eval --samples /var/lib/tally/replay --models gpt-4o-mini,claude-haiku-5-5 --max-spend-usd 5

Each sample is run again on each model you list, using your own provider keys (OPENAI_API_KEY, ANTHROPIC_API_KEY). A judge model then compares each answer with the one your app actually gave. You get one row per model with its cost, the judge's cost, how often it won (with a 95% range), how often it refused, and how many samples were scored, skipped or failed. Skipped and failed samples never count as losses.

The models you compare, and the judge, must be OpenAI or Anthropic models. Only samples from your OpenAI and Anthropic calls are used. Samples that used tools or images are skipped.

• --max-spend-usd is a hard limit on what the run can spend. The command shows the worst case and asks before it starts. --yes skips the question.
• --judge picks the judge model. Otherwise it uses your organization's judge.
• Answers are capped at 1024 tokens (--max-output-tokens, --judge-max-output-tokens). An answer cut off at the cap counts as failed.
• --format json also checks that each answer is valid JSON. --json prints the results as JSON.

tally eval only runs while local capture is on. If your organization has turned it off, it also deletes the samples. aitally.run_local_eval(...) does the same from code.

Setup help for your coding agent (MCP)

aitally.init() records model calls by itself. Vector search, tool calls and cost per customer need a few lines in your code. The onboarding MCP server gives your coding agent (Claude Code, Cursor) the right lines for your app. It runs on your machine, your code never leaves it, and it only suggests code. It never edits your files.

pip install "ai-tally[mcp]"

This installs the tally-onboarding-mcp command. Give it your ingest key as TALLY_KEY. Only the coverage_report tool uses the key. The others work offline.

Claude Code

claude mcp add ai-tally-onboarding --env TALLY_KEY=tally_sk_... -- tally-onboarding-mcp

To share it with your team, add it to .mcp.json in your project. ${TALLY_KEY} is read from each person's own environment, so the key stays out of the file:

{
  "mcpServers": {
    "ai-tally-onboarding": {
      "command": "tally-onboarding-mcp",
      "args": [],
      "env": { "TALLY_KEY": "${TALLY_KEY}" }
    }
  }
}

Cursor

Add this to ~/.cursor/mcp.json. A project .cursor/mcp.json works too, but never commit a file that holds your key:

{
  "mcpServers": {
    "ai-tally-onboarding": {
      "command": "tally-onboarding-mcp",
      "args": [],
      "env": { "TALLY_KEY": "tally_sk_..." }
    }
  }
}

If your agent can't find tally-onboarding-mcp, give the full path instead, for example "command": "/path/to/.venv/bin/tally-onboarding-mcp".

The tools

The server never guesses. It won't pick a customer id for you: tell it how your app knows the customer, or the customer part stays empty. Anything it can't fill in from your code is left as a visible <FILL:...> for you to complete. It also won't suggest a record_llm_call for a call init() already records, since that would count the cost twice.

coverage_report settings

coverage_report is the only tool that connects to anything. It asks AI Tally which kinds of cost are arriving for your organization.

The tool reads the key when it needs it and never saves, shows or logs it. If AI Tally can't be reached, every kind of cost shows as unknown with the reason. A kind of cost only shows as covered when real data has arrived. Pass wired with the kinds you just set up and the report says "waiting for the first event" instead of "not set up".

Node and TypeScript apps

The SDK is Python, but the MCP server also helps with Node apps. Give detect_stack your package.json and it suggests setups that need no SDK:

<ingest> is https://ingest.ai-tally.com. The proxy passes your provider key on to the provider and doesn't store it, but the key does pass through AI Tally. OpenTelemetry never sends it. An admin has to turn the proxy on first, under Settings > API keys. Until then it answers 403.

As in Python, the customer id is hashed in your app and only the hash is sent. Use one setup per call: if a call goes through the proxy and OpenTelemetry, it's counted twice, and detect_stack warns you. There's no Node recipe yet for Gemini, Bedrock, Mistral, Cohere, Groq or Node vector databases. detect_stack lists them as gaps.

License

MIT. See LICENSE.