datasette-llm-accountant 0.1a0

Accounting for LLM token usage

datasette-llm-accountant

Accounting for LLM token usage

Installation

Install this plugin in the same environment as Datasette.

datasette install datasette-llm-accountant

Overview

This plugin provides a library for other Datasette plugins to interact with LLMs via a layer that does accounting and cost tracking. It wraps the llm library with automatic cost calculation and spending limits.

Usage

Basic Usage

from datasette_llm_accountant import LlmWrapper

# In your Datasette plugin
llm = LlmWrapper(datasette)

# Get an async model wrapped with accounting
model = llm.get_async_model("gpt-4o-mini")

# Use with automatic reservation (defaults to 50 cents)
response = await model.prompt("Write a poem about a pirate")

Manual Reservations

You can manually reserve an amount and track spending across multiple prompts:

# Reserve $4.50 for multiple prompts
async with model.reserve(usd=4.50) as tx:
    response1 = await tx.prompt("First question")
    response2 = await tx.prompt("Follow-up question")
    # Total spending is tracked cumulatively

You can also specify reservations in nanocents (1/1,000,000,000 of a cent):

async with model.reserve(nanocents=50_000_000_000) as tx:  # 50 cents
    response = await tx.prompt("a poem about a pirate")

Cost Protection

If a prompt exceeds your reservation, a ReservationExceededError is raised:

from datasette_llm_accountant import ReservationExceededError

try:
    async with model.reserve(usd=0.01) as tx:  # Very small reservation
        # This might exceed the reservation
        response = await tx.prompt("Write a very long story...")
except ReservationExceededError as e:
    print(f"Exceeded budget: {e}")

Creating Accountant Plugins

Other plugins can implement accountants to track and limit LLM spending. Create a plugin that implements the register_llm_accountants hook:

from datasette import hookimpl
from datasette_llm_accountant import Accountant, Tx, InsufficientBalanceError

class DatabaseAccountant(Accountant):
    """Tracks spending in a database."""

    async def reserve(self, nanocents: int) -> Tx:
        # Check if user has sufficient balance
        # If not, raise InsufficientBalanceError
        # Otherwise, create and return a transaction ID
        tx_id = await self.create_reservation(nanocents)
        return Tx(tx_id)

    async def settle(self, tx: Tx, nanocents: int):
        # Record the actual amount spent for this transaction
        await self.record_settlement(tx, nanocents)

    async def rollback(self, tx: Tx):
        # Optional: Release/cancel the reservation
        # Default implementation settles for 0
        await self.cancel_reservation(tx)

@hookimpl
def register_llm_accountants(datasette):
    return [DatabaseAccountant(datasette)]

Multiple Accountants

The system supports multiple accountants. When a reservation is made:

1. All accountants are called in sequence to reserve the amount
2. If any accountant fails, all previous reservations are rolled back
3. When the transaction completes, all accountants are settled with the actual cost

This allows for multiple layers of accounting (e.g., per-user limits, per-project limits, global limits).

Cost Calculation

Costs are automatically calculated based on token usage and current pricing data from llm-prices.com. The library:

• Tracks input and output tokens separately
• Handles cached input tokens when available
• Converts costs to nanocents for precise accounting
• Works with 80+ models across multiple providers

API Reference

LlmWrapper

wrapper = LlmWrapper(datasette)
model = wrapper.get_async_model("model-id")

AccountedModel

# Direct prompt with auto-reservation
response = await model.prompt("text", usd=0.5)

# Manual reservation
async with model.reserve(usd=1.0) as tx:
    response = await tx.prompt("text")

Accountant Base Class

class Accountant(ABC):
    async def reserve(self, nanocents: int) -> Tx: ...
    async def settle(self, tx: Tx, nanocents: int): ...
    async def rollback(self, tx: Tx): ...  # Optional override

Exceptions

• InsufficientBalanceError - Accountant cannot reserve the requested amount
• ReservationExceededError - Actual cost exceeds reserved amount
• ModelPricingNotFoundError - Pricing data not available for model

Development

To set up this plugin locally, first checkout the code. Then create a new virtual environment:

cd datasette-llm-accountant
python -m venv venv
source venv/bin/activate

Now install the dependencies and test dependencies:

pip install -e '.[test]'

To run the tests:

python -m pytest