Lutflow — AI Cost Intelligence for LLM inference

The Lutflow SDK gives developers a first, honest handle on what LLM inference costs — and how to keep it in check. Track and enforce your provider spend in-process (OpenAI · Anthropic · Gemini), with budget strategies that fit your workflow.

pip install lutflow

Offline-first — no account, no backend needed to start.

What this SDK is today

Budget enforcement acts only on the current Python process (raise / warn / callback / SIGKILL of os.getpid()) — it is not cluster-level workload termination.

The connected Lutflow platform — fleet-wide cost intelligence — is in private development.

In-process budget enforcement

Wrap your provider client; Lutflow meters every call and enforces the limit locally. This runs entirely in your process — no account, no network.

from lutflow import Client, BudgetStrategy
import openai

client = Client(
    tenant_id="acme",
    budget_usd=10.00,
    on_budget_exceeded=BudgetStrategy.RAISE_ERROR,
)
wrapped = client.wrap(openai.OpenAI())
response = wrapped.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

print(f"Spent:     ${client.accumulated_cost_usd:.4f}")
print(f"Remaining: ${client.remaining_budget_usd:.4f}")

When the budget is exceeded, the configured strategy fires in this process:

A context-manager form is also available:

from lutflow import budget_session

with budget_session(budget_usd=0.50, tenant_id="acme") as session:
    wrapped = session.wrap(openai.OpenAI())
    ...

GPU-time pricing (self-hosted models)

from lutflow import Client, PricingMode

client = Client(
    tenant_id="acme",
    budget_usd=5.00,
    pricing_mode=PricingMode.GPU_TIME,
    gpu_type="nvidia-l4",
)
client.start_gpu_timer()
# ... run inference ...
cost = client.stop_gpu_timer()

lutflow measure — cost & energy metrics from a telemetry export

A small offline utility: point it at a GPU telemetry export and get a per-concurrency breakdown of cost and energy per token. Read-only, no network.

lutflow measure --from-telemetry telemetry.jsonl --price-per-hr 0.85

Installation

pip install lutflow            # Core + CLI (measure + in-process enforcement)
pip install lutflow[openai]    # + OpenAI wrapper
pip install lutflow[anthropic] # + Anthropic wrapper
pip install lutflow[gemini]    # + Google Gemini wrapper
pip install lutflow[all]       # All providers

Supported providers: OpenAI, Anthropic, Google Gemini, and self-hosted (vLLM / TGI / BentoML via GPU-time pricing).

Links

• Website: lutflow.dev
• PyPI: pypi.org/project/lutflow
• Support: oscarmatiasg@lutflow.com

License

Business Source License 1.1 (BSL 1.1)

• Free for: internal use, development, testing, evaluation, non-commercial use
• Commercial license required for: offering GPU cost management as a service
• Change Date: March 29, 2030 (converts to Apache 2.0)

See LICENSE for full terms. For commercial licensing: licensing@lutflow.com