dfxpy: The DataFrame Workflow Accelerator ⚡

dfxpy is a high-performance Python library designed to reduce the entire data preparation and analysis workflow into a few powerful, deterministic operations. It helps data scientists and engineers go from raw, messy datasets to model-ready features in seconds, not hours.

🎯 Why dfxpy?

While pandas is the industry standard for data manipulation, cleaning a dataset for machine learning often requires writing repetitive boilerplate code. dfxpy acts as an accelerator, automating the "grunt work" while providing intelligent diagnostics and insights.

• Fast & Deterministic: Optimized operations without the unpredictability of AI.
• Production-Ready: Type-hinted, modular, and follows PEP8 standards.
• Lightweight: Minimal dependencies—built on top of pandas, numpy, and scikit-learn.
• Self-Contained: Load and analyze data without needing to import multiple libraries.

✨ Key Features

🛠️ Auto-Pilot Cleaning (auto)

One-line pipeline for column normalization (snake_case), duplicate removal, smart type inference (Object → Numeric/Datetime), and missing value imputation.

🔍 Deep Audit (audit)

Intelligent dataset diagnostics that detect:

• ID-like columns (uniqueness checks)
• High cardinality categoricals
• Multicollinearity (correlated features)
• Data skewness and low-variance columns

🚀 ML Preparation (prepare)

Transform your data into X and y instantly. Automatically handles categorical target encoding (LabelEncoding), feature encoding (One-Hot), and optional scaling.

📉 Smart EDA (eda)

Generates structured, human-readable summaries of shapes, nulls, unique counts, and correlation matrices.

🧹 Outlier Management (outliers)

Detect and handle outliers using the IQR (Interquartile Range) method with options to remove or cap.

📊 One-Click Reports (report)

Generate beautiful, standalone HTML EDA reports with distributions and missing value summaries—zero dependencies required.

🔄 Dataset Comparison (compare)

Instantly track changes between two versions of a dataset. Detects shape shifts, column changes, and cell-level value modifications.

⚖️ Smart Balancing (balance)

Handle imbalanced datasets with ease using Oversampling, Undersampling, or synthetic interpolation.

🛡️ Data Contracts (validate)

Enforce rigorous data contracts with schema validation. Ensure your pipeline only processes data that meets your quality standards.

🏗️ Workflow Pipelines (pipeline)

Build, save, and reuse complex cleaning workflows with the new chainable pipeline engine.

💡 ML Model Advisor (suggest)

Get instant recommendations on which ML algorithms (XGBoost, LightGBM, etc.) to use based on your dataset's specific characteristics.

🕵️ Explainable Cleaning (analyze_cleaning)

Understand exactly what happened to your data. Logs every transformation and repair for full transparency.

📦 Installation

Install the latest version via pip:

pip install dfxpy

For detailed usage and parameter definitions, see the Full API Reference.

🚀 Quick Start

Python API

import dfxpy as dfx

# 1. Load data directly
df = dfx.load("raw_data.csv")

# 2. Run auto-clean (names, types, nulls, encoding)
df_clean = dfx.auto(df)

# 3. Get intelligent insights
dfx.audit(df_clean)

# 4. Prepare for Machine Learning
X, y = dfx.prepare(df_clean, target="outcome")

Command Line Interface (CLI)

# Analyze a dataset instantly
dfxpy analyze data.csv

# Prepare data for ML via CLI
dfxpy prepare data.csv --target price --output cleaned_features.csv

🧠 Pandas vs. dfxpy

🤝 Contributing

We welcome contributions! Please see our CONTRIBUTING.md for guidelines on how to submit issues, feature requests, and pull requests.

📄 License

dfxpy is licensed under the MIT License.

Built with ❤️ for the Data Science Community.