datawash-inspector 0.1.5

An automated end-to-end data cleaning, preprocessing, and EDA pipeline.

DataWash 🔬

DataWash is an automated, end-to-end Python data pipeline that cleans, prunes, prepares, and visualizes your datasets with zero manual effort. It includes a built-in interactive Streamlit dashboard for zero-code data exploration!

Installation

You can install datawash via pip:

pip install datawash-inspector

Quick Start

It only takes a few lines of code to completely sanitize, optimize, and encode your dataset for machine learning.

from datawash import DataPipeline

# 1. Initialize Pipeline
pipe = DataPipeline()

# 2. Load and Auto-Clean
pipe.load_data('your_data.csv')
pipe.sanitize_garbage().auto_type_correct()
pipe.handle_missing_values(strategy='auto')

# 3. Export Clean Data
pipe.save_data('cleaned_data.csv')

The Interactive Dashboard

datawash comes with a fully automated EDA (Exploratory Data Analysis) dashboard powered by Streamlit. You can launch it directly from your script to visually explore your data!

# Launch the dashboard to see Heatmaps, Bar Charts, and Health Reports!
pipe.run_app()

Features

• Memory Optimization: Automatically downcasts large numbers to save memory (up to 50%+ reduction).
• Auto-Cleaning: Automatically converts pure whitespace strings or garbage text ("N/A", "?", "-") to NaN values.
• Smart Imputation: Dynamically imputes missing numeric values with the median and categorical values with the mode.
• Statistical EDA: Calculates an advanced unified correlation matrix supporting numerical, categorical, and mixed variable types (Cramér's V, Eta, Spearman).
• Feature Pruning: Automatically identifies and drops highly redundant features (e.g., > 90% correlation) and high-cardinality IDs.

License

This project is licensed under the MIT License.