csvpredict 0.0.2

SDK for the CSVPredict API - analyze and visualize CSV data

CSVPredict SDK

A Python SDK for the CSVPredict API - analyze and visualize your tabular data with ease.

Features

• 📊 Generate graphs - Automatically create histograms, correlations, and statistical visualizations
• 🔍 Inspect data - Get comprehensive statistics for numeric, string, datetime, and boolean columns
• 🐼 DataFrame support - Works with pandas, Polars DataFrames, and Polars LazyFrames
• 📓 Jupyter integration - Display graphs directly in notebooks with customizable sizing
• 🔄 Multiple output formats - Export to Polars, pandas, numpy, or raw dictionaries
• 🎯 Flexible filtering - Case-insensitive substring matching for columns, datatypes, and partitions

Installation

pip install csvpredict

Or with uv:

uv add csvpredict

Quick Start

import polars as pl
from csvpredict import CSVPredict

# Initialize the SDK
sdk = CSVPredict()

# Load your data
df = pl.read_csv("sales_data.csv")

# Generate graphs
graphs = sdk.generate_graphs(df)
graphs.display()  # Show in Jupyter

# Get statistics
result = sdk.inspect(df)
print(result.stats.overall.height)  # Number of rows
print(result.stats.summary.numeric["price"])  # Price statistics

API Reference

Initializing the SDK

from csvpredict import CSVPredict

sdk = CSVPredict(
    api_key="your-api-key",               # Required for production (or set CSVPREDICT_API_KEY env var)
    base_url="https://csvpredict.com/api/v1",  # API server URL (default)
    timeout=60.0,                          # Request timeout in seconds
)

# For local development, no API key required
sdk = CSVPredict(base_url="http://localhost:8000/_/")

---

Generating Graphs

Generate statistical visualizations from your data.

Basic Usage

import polars as pl
from csvpredict import CSVPredict

sdk = CSVPredict()
df = pl.read_csv("data.csv")

# Generate graphs with default settings (SVG format)
graphs = sdk.generate_graphs(df)

Options

graphs = sdk.generate_graphs(
    df,
    partition_by=["category"],     # Generate separate graphs per category
    extension=".png",              # Output format: .svg, .png, .jpg, .jpeg
    transparent=True,              # Transparent background
    window_size=7,                 # Rolling window size for time series
    null_values=["N/A", "NULL"],   # Strings to treat as null
    dpi=300,                       # Resolution for raster formats
    font="Arial",                  # Font family
    language="en",                 # Language for labels
)

Working with GraphResult

The generate_graphs() method returns a GraphResult object with many ways to access your graphs:

Display in Jupyter

# Display all graphs in a grid
graphs.display()

# Display with custom size
graphs.display(width=500, height=400)

# Display in 3 columns
graphs.display(columns=3)

# Filter by name (case-insensitive substring match)
graphs.display("histogram")           # Show all histograms
graphs.display("price")               # Show price-related graphs
graphs.display("correlation")         # Show correlation matrices

# Filter by extension
graphs.display(extension=".png")

# Display a single graph
graphs.display_one("price_histogram.svg", width=600)

Access Individual Graphs

# Get list of all graph names
print(graphs.names)
# ['price_histogram.svg', 'quantity_histogram.svg', 'correlation_matrix.svg', ...]

# Get raw bytes for a specific graph
svg_bytes = graphs["price_histogram.svg"]

# Check if a graph exists
if graphs.contains("price_histogram.svg"):
    print("Found it!")

# Get with default
data = graphs.get("missing.svg", default=None)

# Number of graphs
print(graphs.count())  # 15

Iterate and Filter

# Iterate over all graphs
for name, data in graphs:
    print(f"{name}: {len(data)} bytes")

# Filter by pattern (case-insensitive)
for name, data in graphs.filter("histogram"):
    print(name)

# Filter by extension
for name, data in graphs.filter(extension=".svg"):
    print(name)

Convert to PIL Image (Raster Only)

# Convert PNG/JPG to PIL Image for further processing
img = graphs.to_pil("price_histogram.png")
img.show()
img.save("modified.png")

Save to Disk

# Save as ZIP file
graphs.save("graphs.zip")

# Extract all to a directory
extracted_files = graphs.extract("./output_graphs/")
print(extracted_files)
# [Path('output_graphs/price_histogram.svg'), ...]

---

Inspecting Data

Get comprehensive statistics about your dataset.

Basic Usage

result = sdk.inspect(df)

# Access overall statistics
print(result.stats.overall.height)          # Number of rows
print(result.stats.overall.width)           # Number of columns
print(result.stats.overall.null_count)      # Total null values
print(result.stats.overall.duplicate_count) # Duplicate rows
print(result.stats.overall.column_names)    # List of columns

Options

result = sdk.inspect(
    df,
    partition_by=["category", "region"],  # Partition data
    window_size=5,                         # Rolling window size
    round_digits=3,                        # Decimal precision
    null_values=["N/A", ""],               # Strings to treat as null
)

Accessing Summary Statistics

Summary statistics are organized by data type:

# Numeric columns
result.stats.summary.numeric["price"]           # Statistics for 'price' column
result.stats.summary.numeric_count["price"]     # Value counts for 'price'
result.stats.summary.numeric_rolling["price"]   # Rolling statistics (time series)

# String columns
result.stats.summary.string["product_name"]
result.stats.summary.string_count["product_name"]

# Datetime columns
result.stats.summary.datetime["created_at"]
result.stats.summary.datetime_count["created_at"]

# Boolean columns
result.stats.summary.boolean["is_active"]
result.stats.summary.boolean_count["is_active"]

# Duration columns
result.stats.summary.duration["processing_time"]
result.stats.summary.duration_count["processing_time"]

Filtering Statistics

All filters use case-insensitive substring matching:

# Filter by column name
result.stats.summary.get_stats_for_column("price")
# Returns: {'price': DataFrame, 'unit_price': DataFrame, 'total_price': DataFrame}

# Filter by column name - get value counts
result.stats.summary.get_counts_for_column("status")
# Returns: {'status': DataFrame, 'order_status': DataFrame}

# Filter by column name - get rolling stats
result.stats.summary.get_rolling_for_column("price")

# Filter by datatype
result.stats.summary.filter(datatype="numeric")
# Returns: {'numeric': {'price': DataFrame, 'quantity': DataFrame, ...}}

# Filter by both column and datatype
result.stats.summary.filter(column="price", datatype="count")
# Returns: {'numeric_count': {'price': DataFrame, 'unit_price': DataFrame}}

# Access datatype directly
result.stats.summary["numeric"]        # All numeric statistics
result.stats.summary["string_count"]   # All string value counts

Correlation Matrix

# Get the full correlation matrix
corr = result.stats.correlation.matrix
print(corr)

# Get correlation between two specific columns
corr_value = result.stats.correlation.get_correlation("price", "quantity")
print(f"Correlation: {corr_value}")

# Get column names in the correlation matrix
print(result.stats.correlation.columns)

Working with Partitioned Data

When you use partition_by, data is split into groups:

result = sdk.inspect(df, partition_by=["category"])

# Check if data is partitioned
print(result.is_partitioned)  # True

# List available partitions
print(result.partitions)
# ['_electronics', '_clothing', '_food']

# Access a specific partition
electronics_stats = result["_electronics"]
print(electronics_stats.overall.height)
print(electronics_stats.summary.numeric["price"])

# Iterate over all partitions
for partition_name, stats in result.items():
    print(f"{partition_name}: {stats.overall.height} rows")

# Filter partitions by name
matching = result.filter_partitions("electronics")
# Returns: {'_electronics': Statistics(...)}

# Filter across partitions, columns, and datatypes
filtered = result.filter(
    partition="electronics",
    column="price",
    datatype="numeric"
)

Non-Partitioned Data

For non-partitioned data, use .stats directly:

result = sdk.inspect(df)  # No partition_by

# Access statistics directly
stats = result.stats
print(stats.overall.height)
print(stats.summary.numeric["price"])

---

Output Formats

All statistics objects support multiple output formats:

Polars (Default)

# DataFrames are Polars by default
df = result.stats.summary.numeric["price"]
print(type(df))  # <class 'polars.DataFrame'>

# Explicitly convert to Polars
polars_data = result.stats.to_polars()
polars_lazy = result.stats.to_polars_lazy()  # LazyFrame for large data

Pandas

# Convert everything to pandas
pandas_data = result.stats.to_pandas()

# Convert specific statistics
pandas_df = result.stats.summary.get_stats_for_column_pandas("price")
pandas_counts = result.stats.summary.get_counts_for_column_pandas("status")

# Correlation matrix with index
corr_df = result.stats.correlation.to_pandas_indexed()

NumPy

# Convert to numpy arrays
numpy_data = result.stats.to_numpy()

# Correlation matrix as numpy array
corr_array = result.stats.correlation.to_numpy()

Dictionary

# Convert to plain Python dictionaries
dict_data = result.stats.to_dict()

# Useful for JSON serialization
import json
json.dumps(result.stats.overall.to_dict())

---

DataFrame Compatibility

The SDK works with multiple DataFrame types:

Polars DataFrame

import polars as pl

df = pl.read_csv("data.csv")
result = sdk.inspect(df)
graphs = sdk.generate_graphs(df)

Polars LazyFrame

import polars as pl

# LazyFrames are collected automatically
lf = pl.scan_csv("data.csv")
result = sdk.inspect(lf)
graphs = sdk.generate_graphs(lf)

Pandas DataFrame

import pandas as pd

df = pd.read_csv("data.csv")
result = sdk.inspect(df)
graphs = sdk.generate_graphs(df)

---

Browser Inspection

Open an interactive inspection view in your browser:

# Inspect and open browser
sdk.inspect_in_browser(df)

# With partitioning
sdk.inspect_in_browser(df, partition_by=["category"])

# Show in Jupyter notebook instead of browser
sdk.inspect_in_browser(df, show_in_ipynb=True)

Standalone Visualization Functions

You can also use the standalone show() and display() functions:

from csvpredict import CSVPredict, show, display

sdk = CSVPredict()
result = sdk.inspect(df)

# Open in browser with live reload
show(result)

# Or display inline in Jupyter notebook
display(result)
display(result, height=600)  # Custom height

---

Complete Example

import polars as pl
from csvpredict import CSVPredict

# Initialize SDK
sdk = CSVPredict(base_url="http://localhost:8000/_/")

# Load data
df = pl.read_csv("sales_data.csv")

# Generate and display graphs
graphs = sdk.generate_graphs(df, extension=".svg")
graphs.display(width=400, columns=2)

# Filter to show only histograms
graphs.display("histogram", width=300)

# Save graphs to disk
graphs.save("sales_graphs.zip")

# Inspect data
result = sdk.inspect(df)

# Overall statistics
print(f"Rows: {result.stats.overall.height}")
print(f"Columns: {result.stats.overall.width}")
print(f"Null values: {result.stats.overall.null_count}")

# Numeric statistics for price column
price_stats = result.stats.summary.numeric["price"]
print(price_stats)

# Get all price-related statistics
price_data = result.stats.summary.get_stats_for_column("price")
for col_name, stats_df in price_data.items():
    print(f"\n{col_name}:")
    print(stats_df)

# Correlation analysis
print("\nCorrelation Matrix:")
print(result.stats.correlation.matrix)

# Get specific correlation
corr = result.stats.correlation.get_correlation("price", "quantity")
print(f"\nPrice-Quantity correlation: {corr}")

# Export to different formats
pandas_stats = result.stats.to_pandas()
numpy_stats = result.stats.to_numpy()
dict_stats = result.stats.to_dict()

# Partitioned analysis
result = sdk.inspect(df, partition_by=["category"])
for partition, stats in result.items():
    print(f"\n{partition}: {stats.overall.height} rows")
    print(stats.summary.numeric["price"])

---

Jupyter Notebook Tips

Optimal Display Settings

# For high-DPI displays
graphs.display(width=600, height=450, columns=2)

# For presentations
graphs.display(width=800, height=600, columns=1)

# Quick overview
graphs.display(width=300, height=200, columns=4)

Filtering in Notebooks

# Show only correlation graphs
graphs.display("correlation")

# Show only a specific column's graphs
graphs.display("price")

# Show histograms at smaller size
graphs.display("histogram", width=250, columns=3)

---

Error Handling

from csvpredict import CSVPredict

sdk = CSVPredict()

try:
    result = sdk.inspect(df)
except ValueError as e:
    print(f"API error: {e}")
except Exception as e:
    print(f"Unexpected error: {e}")

---

License

MIT License - see LICENSE file for details.