Skip to main content

Transform LiteLLM database data into CloudZero AnyCost CBF format

Project description

ll2cz - LiteLLM to CloudZero ETL Tool

Transform LiteLLM database data into CloudZero AnyCost CBF format for cost tracking and analysis.

Features

  • Extract usage data from LiteLLM PostgreSQL database
  • Transform data into CloudZero Billing Format (CBF)
  • Cost comparison analysis between SpendLogs and user tables (v0.4.0+)
  • Dual data source support with --source option: transaction-level SpendLogs or daily aggregated user tables
  • Analysis mode with beautiful terminal output using Rich
  • Multiple output options: CSV files or direct CloudZero API streaming
  • Built with modern Python tools: uv, ruff, pytest, polars, httpx

Installation

From PyPI (Recommended)

# Install with uv (recommended)
uv add ll2cz

# Or install with pip
pip install ll2cz

From Source

git clone https://github.com/Cloudzero/cloudzero-litellm-toolkit.git
cd cloudzero-litellm-toolkit
uv sync

Configuration

ll2cz supports configuration files to avoid repeating common settings. You can store your database connection, CloudZero API credentials, and other settings in ~/.ll2cz/config.yml.

Create Configuration File

# Create an example configuration file
ll2cz config example

# Check current configuration status
ll2cz config status

This creates ~/.ll2cz/config.yml with the following structure:

database_url: postgresql://user:password@host:5432/litellm_db
cz_api_key: your-cloudzero-api-key
cz_connection_id: your-connection-id

Configuration Priority

CLI arguments always take priority over configuration file values:

  1. CLI arguments (highest priority)
  2. Configuration file (~/.ll2cz/config.yml)
  3. Default values (lowest priority)

CLI Commands

# Show version
ll2cz --version

# Show help
ll2cz --help

# Configuration management
ll2cz config example     # Create example config file
ll2cz config status      # Show current config status
ll2cz config edit        # Interactively edit configuration

Usage

Transform Mode

Transform LiteLLM data to CloudZero CBF format:

# Display data on screen (formatted table)
ll2cz transform --input "postgresql://user:pass@host:5432/litellm_db" --screen

# Export to CSV file
ll2cz transform --input "postgresql://user:pass@host:5432/litellm_db" --output data.csv

# Limit records for screen display
ll2cz transform --screen --limit 25

Transmit Mode

Send data directly to CloudZero AnyCost API:

# Send today's data
ll2cz transmit day

# Send specific day's data (DD-MM-YYYY format)
ll2cz transmit day 15-01-2024

# Send current month's data
ll2cz transmit month

# Send specific month's data (MM-YYYY format)
ll2cz transmit month 01-2024

# Send all available data (batched by day)
ll2cz transmit all

# Test mode - show payloads without sending (5 records only)
ll2cz transmit day --test

# Use append mode (sum operation instead of replace_hourly)
ll2cz transmit day --append

# Specify timezone for date handling
ll2cz transmit day --timezone "US/Eastern"

# Limit number of records to process
ll2cz transmit month --limit 1000

Analysis Mode

Analyze your LiteLLM database data:

# General data analysis
ll2cz analyze data --limit 10000

# Show raw table data
ll2cz analyze data --show-raw --table all

# Show specific table only
ll2cz analyze data --show-raw --table user

# CZRN (CloudZero Resource Name) analysis
ll2cz analyze czrn --limit 10000

# Spend analysis
ll2cz analyze spend --limit 10000

# Database schema analysis
ll2cz analyze schema --output schema_docs.md

# Refresh cache from server (use cache commands)
ll2cz cache refresh

# Save analysis to JSON
ll2cz analyze data --json analysis.json

Cache Management

Manage local data cache for offline operation:

# Check cache status (local only)
ll2cz cache status

# Check cache status with remote server verification
ll2cz cache status --remote-check

# Clear local cache
ll2cz cache clear

# Force refresh cache from server
ll2cz cache refresh

Data Transformation

The tool transforms LiteLLM usage logs into CloudZero's CBF format with the following mappings:

  • spendcost
  • total_tokensusage_quantity
  • model, user_id, call_typedimensions
  • metadata fields → additional dimensions
  • Duration calculated from startTime and endTime

Technology Stack

This project follows modern Python best practices:

  • Python 3.12 - Latest Python version
  • uv - Fast Python package manager
  • Polars - High-performance DataFrames (instead of pandas)
  • httpx - Modern HTTP client (instead of requests)
  • Rich - Beautiful terminal output and formatting
  • Typer - Modern CLI framework with rich help formatting
  • PyYAML - YAML configuration file support
  • Pathlib - Modern filesystem path operations
  • pytest - Testing framework
  • ruff - Fast Python linter and formatter

Development

# Setup development environment
uv sync --dev

# Run tests
uv run pytest

# Run linting
uv run ruff check src/ tests/

# Fix linting issues
uv run ruff check --fix src/ tests/

# Build package
uv build

Requirements

  • Python ≥ 3.12
  • PostgreSQL database with LiteLLM data
  • CloudZero API key and connection ID (for streaming mode)

License

Apache 2.0 - see LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ll2cz-0.6.0.tar.gz (66.4 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ll2cz-0.6.0-py3-none-any.whl (78.3 kB view details)

Uploaded Python 3

File details

Details for the file ll2cz-0.6.0.tar.gz.

File metadata

  • Download URL: ll2cz-0.6.0.tar.gz
  • Upload date:
  • Size: 66.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.11

File hashes

Hashes for ll2cz-0.6.0.tar.gz
Algorithm Hash digest
SHA256 e1f1702470c66f754c72a6468db38b6757852d284a3854f66c9ab09a49383cc6
MD5 e5cb4f2613699cba3301f08629c66bea
BLAKE2b-256 ab04574aafa30d9813d0657a85349a4aea546e4cc7e70f4cfe79ba178f0c6b0d

See more details on using hashes here.

File details

Details for the file ll2cz-0.6.0-py3-none-any.whl.

File metadata

  • Download URL: ll2cz-0.6.0-py3-none-any.whl
  • Upload date:
  • Size: 78.3 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.12.11

File hashes

Hashes for ll2cz-0.6.0-py3-none-any.whl
Algorithm Hash digest
SHA256 d25fc5620015fcc185d908a46a1e3742c1821c58eab57bccf7b84e2e878c5b2a
MD5 2d63fafc13f32b0d8477c94639213ab1
BLAKE2b-256 cc851d9e058d802c196b2816c18b2104e29157d8af01f254ed00bf4de6b2b22c

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page