lutflow 0.1.0a1

GPU Financial Firewall — budget enforcement for AI inference

Lutflow — GPU Financial Firewall

Budget enforcement for AI inference workloads. Track costs, enforce budgets, kill runaway GPU spending in real time.

pip install lutflow

5-Line Quickstart

from lutflow import Client
import openai

client = Client(tenant_id="acme", budget_usd=10.00)
wrapped = client.wrap(openai.OpenAI())
response = wrapped.chat.completions.create(model="gpt-4o", messages=[{"role": "user", "content": "Hello!"}])

Lutflow tracks every token, calculates cost, and terminates the process if the budget is exceeded.

CLI

# GPU price lookup (powered by TinyFish)
lutflow lookup --gpu nvidia-l4

# Deploy with budget enforcement
lutflow deploy --model TinyLlama/TinyLlama-1.1B-Chat-v1.0 --budget 0.10

# Live dashboard
lutflow watch

# Check deployment status
lutflow status <deployment_id>

# Manual kill
lutflow kill <deployment_id> --reason overspend

Installation

pip install lutflow            # Core + CLI
pip install lutflow[openai]    # With OpenAI wrapper
pip install lutflow[anthropic] # With Anthropic wrapper
pip install lutflow[all]       # All providers + Kafka transport

SDK Usage

Token-Based Pricing (API Providers)

from lutflow import Client, BudgetStrategy

client = Client(
    tenant_id="my-tenant",
    api_key="lut_xxx",
    budget_usd=10.00,
    on_budget_exceeded=BudgetStrategy.RAISE_ERROR,
)

wrapped = client.wrap(openai.OpenAI())
response = wrapped.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}],
)

print(f"Spent: ${client.accumulated_cost_usd:.4f}")
print(f"Remaining: ${client.remaining_budget_usd:.4f}")

GPU-Time Pricing (Self-Hosted Models)

from lutflow import Client, PricingMode

client = Client(
    tenant_id="my-tenant",
    budget_usd=5.00,
    pricing_mode=PricingMode.GPU_TIME,
    gpu_type="nvidia-l4",
)

client.start_gpu_timer()
# ... run inference ...
cost = client.stop_gpu_timer()

GPU Price Lookup (TinyFish)

from lutflow.tinyfish import TinyFishClient

tf = TinyFishClient()  # Uses TINYFISH_API_KEY env var
result = tf.fetch_all_prices(gpu_type="nvidia-l4")

best = result.cheapest
print(f"Cheapest: {best.provider} at ${best.spot_usd_per_hour:.2f}/hr")

Budget Strategies

Strategy	Behavior
SELF_KILL	Sends SIGKILL to the process (default)
RAISE_ERROR	Raises BudgetExceededError
WARN_ONLY	Logs warning, continues execution
CALLBACK	Calls a custom function you provide

Supported Providers

• OpenAI — GPT-4o, GPT-4, GPT-3.5, embeddings
• Anthropic — Claude 3.5, Claude 3 Opus/Sonnet/Haiku
• Google — Gemini 1.5 Pro/Flash
• Self-hosted — vLLM, TGI, BentoML (GPU-time pricing)

Environment Variables

Variable	Description
LUTFLOW_ENDPOINT	API endpoint (default: https://api.lutflow.dev)
TINYFISH_API_KEY	TinyFish API key for live GPU pricing

Links

• Website: lutflow.dev
• Docs: docs.lutflow.dev/sdk/python
• GitHub: github.com/Lutflow/lutflow

License

Business Source License 1.1 (BSL 1.1)

• Free for: internal use, development, testing, evaluation, non-commercial use
• Commercial license required for: offering GPU cost management as a service
• Change Date: March 29, 2030 (converts to Apache 2.0)

See LICENSE for full terms. For commercial licensing: [e-mail]