Skip to main content

Transform LiteLLM database data into CloudZero AnyCost CBF format

Project description

ll2cz - LiteLLM to CloudZero ETL Tool

Transform LiteLLM database data into CloudZero AnyCost CBF format for cost tracking and analysis.

Features

  • Extract usage data from LiteLLM PostgreSQL database
  • Transform data into CloudZero Billing Format (CBF)
  • Cost comparison analysis between SpendLogs and user tables (v0.4.0+)
  • Dual data source support with --source option: transaction-level SpendLogs or daily aggregated user tables
  • Analysis mode with beautiful terminal output using Rich
  • Multiple output options: CSV files or direct CloudZero API streaming
  • Built with modern Python tools: uv, ruff, pytest, polars, httpx

Installation

From PyPI (Recommended)

# Install with uv (recommended)
uv add ll2cz

# Or install with pip
pip install ll2cz

From Source

git clone https://github.com/Cloudzero/cloudzero-litellm-toolkit.git
cd cloudzero-litellm-toolkit
uv sync

Configuration

ll2cz supports configuration files to avoid repeating common settings. You can store your database connection, CloudZero API credentials, and other settings in ~/.ll2cz/config.yml.

Create Configuration File

# Create an example configuration file
ll2cz config example

# Check current configuration status
ll2cz config status

This creates ~/.ll2cz/config.yml with the following structure:

database_url: postgresql://user:password@host:5432/litellm_db
cz_api_key: your-cloudzero-api-key
cz_connection_id: your-connection-id

Configuration Priority

CLI arguments always take priority over configuration file values:

  1. CLI arguments (highest priority)
  2. Configuration file (~/.ll2cz/config.yml)
  3. Default values (lowest priority)

CLI Commands

# Show version
ll2cz --version

# Show help
ll2cz --help

# Configuration management
ll2cz config example     # Create example config file
ll2cz config status      # Show current config status
ll2cz config edit        # Interactively edit configuration

Usage

Transform Mode

Transform LiteLLM data to CloudZero CBF format:

# Display data on screen (formatted table)
ll2cz transform --input "postgresql://user:pass@host:5432/litellm_db" --screen

# Export to CSV file
ll2cz transform --input "postgresql://user:pass@host:5432/litellm_db" --output data.csv

# Limit records for screen display
ll2cz transform --screen --limit 25

Transmit Mode

Send data directly to CloudZero AnyCost API:

# Send today's data
ll2cz transmit day

# Send specific day's data (DD-MM-YYYY format)
ll2cz transmit day 15-01-2024

# Send current month's data
ll2cz transmit month

# Send specific month's data (MM-YYYY format)
ll2cz transmit month 01-2024

# Send all available data (batched by day)
ll2cz transmit all

# Test mode - show payloads without sending (5 records only)
ll2cz transmit day --test

# Use append mode (sum operation instead of replace_hourly)
ll2cz transmit day --append

# Specify timezone for date handling
ll2cz transmit day --timezone "US/Eastern"

# Limit number of records to process
ll2cz transmit month --limit 1000

Analysis Mode

Analyze your LiteLLM database data:

# General data analysis
ll2cz analyze data --limit 10000

# Show raw table data
ll2cz analyze data --show-raw --table all

# Show specific table only
ll2cz analyze data --show-raw --table user

# CZRN (CloudZero Resource Name) analysis
ll2cz analyze czrn --limit 10000

# Spend analysis
ll2cz analyze spend --limit 10000

# Database schema analysis
ll2cz analyze schema --output schema_docs.md

# Refresh cache from server (use cache commands)
ll2cz cache refresh

# Save analysis to JSON
ll2cz analyze data --json analysis.json

Cache Management

Manage local data cache for offline operation:

# Check cache status (local only)
ll2cz cache status

# Check cache status with remote server verification
ll2cz cache status --remote-check

# Clear local cache
ll2cz cache clear

# Force refresh cache from server
ll2cz cache refresh

Data Transformation

The tool transforms LiteLLM usage logs into CloudZero's CBF format with the following mappings:

  • spendcost
  • total_tokensusage_quantity
  • model, user_id, call_typedimensions
  • metadata fields → additional dimensions
  • Duration calculated from startTime and endTime

Technology Stack

This project follows modern Python best practices:

  • Python 3.12 - Latest Python version
  • uv - Fast Python package manager
  • Polars - High-performance DataFrames (instead of pandas)
  • httpx - Modern HTTP client (instead of requests)
  • Rich - Beautiful terminal output and formatting
  • Typer - Modern CLI framework with rich help formatting
  • PyYAML - YAML configuration file support
  • Pathlib - Modern filesystem path operations
  • pytest - Testing framework
  • ruff - Fast Python linter and formatter

Development

# Setup development environment
uv sync --dev

# Run tests
uv run pytest

# Run linting
uv run ruff check src/ tests/

# Fix linting issues
uv run ruff check --fix src/ tests/

# Build package
uv build

Requirements

  • Python ≥ 3.12
  • PostgreSQL database with LiteLLM data
  • CloudZero API key and connection ID (for streaming mode)

License

Apache 2.0 - see LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ll2cz-0.5.1.tar.gz (66.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ll2cz-0.5.1-py3-none-any.whl (78.3 kB view details)

Uploaded Python 3

File details

Details for the file ll2cz-0.5.1.tar.gz.

File metadata

  • Download URL: ll2cz-0.5.1.tar.gz
  • Upload date:
  • Size: 66.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.11

File hashes

Hashes for ll2cz-0.5.1.tar.gz
Algorithm Hash digest
SHA256 53f67479013a74a3581b18701c33bd3802436af7a1e5d6e3ac87309e31b2cc5c
MD5 54bd190475e719ddd9e01debfb733f3d
BLAKE2b-256 ca30697d00803ea8b96cfe38db69d98a4c91f0ff2827fef41a6abdc53c7c2bfc

See more details on using hashes here.

File details

Details for the file ll2cz-0.5.1-py3-none-any.whl.

File metadata

  • Download URL: ll2cz-0.5.1-py3-none-any.whl
  • Upload date:
  • Size: 78.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.11

File hashes

Hashes for ll2cz-0.5.1-py3-none-any.whl
Algorithm Hash digest
SHA256 cbc7514449724741d83f95981f5eb49f61c458ab63c63c36fc3dacfadee6c069
MD5 2fed60801a120dc2eb8f00a6a581464a
BLAKE2b-256 144945906c35c0361f375ea85ec1b6b532125ea6edb7dd3b9eb03a3539a8435f

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page