BidViz

A powerful, configurable backend visualization data transformation library designed to bridge the gap between raw data and frontend charting libraries. BidViz supports both pandas and Polars DataFrames, providing comprehensive tools for data cleaning, transformation, and formatting optimized for analytics dashboards and data visualization applications.

NEW: High-performance Polars support for 2-10x faster transformations on large datasets!

Features

• Dual DataFrame Support: Works with both pandas and Polars DataFrames
• High Performance: Polars support for 2-10x faster transformations on large datasets
• 12+ Chart Type Transformations: Support for KPI cards, bar charts, line charts, pie charts, heatmaps, funnels, tables, and more
• Automatic Data Cleaning: NaN/null handling, type conversion, and sanitization
• Human-Readable Formatting: Intelligent label generation from column names
• Built-in Pagination: Server-side pagination for data tables
• Frontend-Ready Output: JSON-serializable structures optimized for charting libraries
• Statistical Analysis: Correlation matrices with heatmap data generation
• Highly Configurable: Custom column mappings, formatting rules, and transformation behaviors
• Framework Agnostic: Works with any frontend (React, Vue, Angular) and any charting library (Chart.js, D3, Plotly, Recharts)

Installation

pip install bidviz

For development:

pip install bidviz[dev]

Quick Start

With Pandas

import pandas as pd
from bidviz import ChartTransformer

# Initialize the transformer
transformer = ChartTransformer()

# Sample data
df = pd.DataFrame({
    'vendor': ['Vendor A', 'Vendor B', 'Vendor C'],
    'revenue': [125000, 98000, 112000]
})

# Transform to bar chart
result = transformer.transform_to_bar_chart(
    df=df,
    x_column='vendor',
    y_column='revenue'
)

print(result)
# {
#     "chart_type": "bar_chart",
#     "data": [
#         {"x": "Vendor A", "y": 125000, "label": "Vendor A"},
#         {"x": "Vendor B", "y": 98000, "label": "Vendor B"},
#         {"x": "Vendor C", "y": 112000, "label": "Vendor C"}
#     ],
#     "x_label": "Vendor",
#     "y_label": "Revenue"
# }

With Polars (High Performance)

import polars as pl
from bidviz.polars import ChartTransformer

# Initialize the Polars transformer
transformer = ChartTransformer()

# Sample data with Polars
df = pl.DataFrame({
    'vendor': ['Vendor A', 'Vendor B', 'Vendor C'],
    'revenue': [125000, 98000, 112000]
})

# Transform to bar chart (2-10x faster for large datasets!)
result = transformer.transform_to_bar_chart(
    df=df,
    x_column='vendor',
    y_column='revenue'
)

# Same output format as pandas version
print(result)

Performance Comparison

When to use Polars:

• Working with datasets > 10K rows
• Need faster API response times
• Building high-throughput data pipelines
• Want to leverage modern multi-core processors

Supported Chart Types

Usage Examples

KPI Cards

# Single-row DataFrame with metrics
df = pd.DataFrame({
    'total_orders': [150],
    'revenue': [45000.50],
    'satisfaction_rate': [94.2]
})

result = transformer.transform_to_kpi_cards(df)
# Returns list of KPI cards with labels, values, and keys

Line Chart with Time Series

df = pd.DataFrame({
    'date': pd.date_range('2024-01-01', periods=30),
    'orders': [152, 168, 145, ...]  # 30 values
})

result = transformer.transform_to_line_chart(
    df=df,
    x_column='date',
    y_column='orders',
    series_name='Daily Orders'
)

Multi-Line Chart

df = pd.DataFrame({
    'date': pd.date_range('2024-01-01', periods=30),
    'vendor_a': [...],
    'vendor_b': [...],
    'vendor_c': [...]
})

result = transformer.transform_to_multi_line_chart(
    df=df,
    x_column='date',
    y_columns=['vendor_a', 'vendor_b', 'vendor_c'],
    series_names=['Vendor A', 'Vendor B', 'Vendor C']
)

Data Table with Pagination

df = pd.DataFrame({
    'order_id': range(1, 1001),
    'customer': [f'Customer {i}' for i in range(1, 1001)],
    'amount': [...]  # 1000 values
})

result = transformer.transform_to_data_table(
    df=df,
    page=1,
    page_size=50
)
# Returns paginated data with metadata

Correlation Heatmap

df = pd.DataFrame({
    'revenue': [...],
    'orders': [...],
    'rating': [...],
    'shipping_days': [...]
})

result = transformer.transform_to_correlation_heatmap(df)
# Auto-detects numeric columns and generates correlation matrix

Integration with Web Frameworks

FastAPI

from fastapi import FastAPI, Query
from bidviz import ChartTransformer
import pandas as pd

app = FastAPI()
transformer = ChartTransformer()

@app.get("/api/charts/revenue")
async def get_revenue_chart(chart_type: str = Query("bar")):
    # Fetch data from database
    df = get_revenue_data()

    if chart_type == "bar":
        return transformer.transform_to_bar_chart(
            df, x_column='vendor', y_column='revenue'
        )
    elif chart_type == "line":
        return transformer.transform_to_line_chart(
            df, x_column='date', y_column='revenue'
        )

Flask

from flask import Flask, jsonify
from bidviz import ChartTransformer

app = Flask(__name__)
transformer = ChartTransformer()

@app.route('/api/charts/sales')
def sales_chart():
    df = get_sales_data()
    result = transformer.transform_to_pie_chart(
        df, label_column='category', value_column='sales'
    )
    return jsonify(result)

Frontend Integration

React with Recharts

import { BarChart, Bar, XAxis, YAxis } from 'recharts';

function RevenueChart() {
  const [chartData, setChartData] = useState(null);

  useEffect(() => {
    fetch('/api/charts/revenue?chart_type=bar')
      .then(res => res.json())
      .then(data => setChartData(data));
  }, []);

  if (!chartData) return <div>Loading...</div>;

  return (
    <BarChart data={chartData.data}>
      <XAxis dataKey="x" label={chartData.x_label} />
      <YAxis label={chartData.y_label} />
      <Bar dataKey="y" fill="#8884d8" />
    </BarChart>
  );
}

Chart.js

const response = await fetch('/api/charts/revenue?chart_type=line');
const chartData = await response.json();

new Chart(ctx, {
  type: 'line',
  data: {
    labels: chartData.data.map(d => d.x),
    datasets: [{
      label: chartData.series_name,
      data: chartData.data.map(d => d.y)
    }]
  }
});

Data Handling

NaN Handling

All transformations automatically handle pandas NaN values:

• Numeric NaN → null in JSON
• String/Empty NaN → null in JSON
• All computations work safely with NaN values

Type Conversion

Label Formatting

Automatic snake_case to Title Case conversion:

• total_gmv → "Total Gmv"
• customer_id → "Customer Id"
• avg_days_to_ship → "Avg Days To Ship"

Error Handling

from bidviz.exceptions import TransformationError

try:
    result = transformer.transform_to_bar_chart(df, 'category', 'value')
except TransformationError as e:
    print(f"Error: {e.message}")
    print(f"Chart Type: {e.chart_type}")
    print(f"DataFrame Shape: {e.df_shape}")
    print(f"Missing Columns: {e.missing_columns}")

Development

Setup Development Environment

# Clone the repository
git clone https://github.com/aghabidareh/bidviz.git
cd bidviz

# Create virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install development dependencies
pip install -e ".[dev]"

Running Tests

# Run all tests with coverage
pytest

# Run specific test file
pytest tests/test_transformer.py

# Run with verbose output
pytest -v

# Generate HTML coverage report
pytest --cov=bidviz --cov-report=html

Code Quality

# Format code with black
black bidviz tests

# Sort imports
isort bidviz tests

# Check code style
flake8 bidviz tests

# Type checking
mypy bidviz

Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add some amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Please make sure to update tests as appropriate and adhere to the existing code style.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Changelog

See CHANGELOG.md for a list of changes.

Roadmap

• Advanced value formatting pipeline
• Data validation framework
• Aggregation pipeline
• Multi-chart dashboard builder
• Real-time streaming support
• Export & report generation (PDF, Excel)
• Caching layer
• Plugin system
• Multi-language support
• Performance profiling tools

Acknowledgments

• Built with pandas
• Inspired by the need for seamless backend-to-frontend data transformation
• Thanks to all contributors and users of this library

Contact

Mohammad Amin Khara - kharama8709@gmail.com

Project Link: https://github.com/aghabidareh/bidviz