masterclean 1.3.0

Automated Data Cleaning, Validation and Analytics Toolkit

🚀 MasterClean

Automated Data Cleaning, Validation & Analytics Toolkit for Python.

MasterClean is a professional Python package that automates:

• data cleaning
• preprocessing
• validation
• profiling
• visualization
• reporting
• analytics

using simple CLI commands or Python API.

Designed for:

• Data Analysts
• Data Scientists
• ML Engineers
• Researchers
• Students
• Automation workflows

---

✨ Features

🧹 Advanced Data Cleaning

• Missing value handling
• Duplicate row removal
• Empty string cleanup
• Whitespace cleanup
• Column standardization
• Datetime conversion
• Smart categorical filling
• Automatic preprocessing pipeline

---

⚡ Datatype Optimization

• Integer optimization
• Float optimization
• Boolean conversion
• Category optimization
• Datetime detection
• Memory usage reduction

---

🛡 Advanced Validation Engine

• Negative value detection
• Outlier detection
• Invalid boolean detection
• Email validation
• Phone validation
• Duplicate percentage warnings
• Missing value percentage analysis
• Mixed datatype detection

---

📊 Advanced Profiling

• Dataset health score
• Missing value summaries
• Datatype analytics
• Memory usage analysis
• Numeric statistics
• Categorical summaries
• Dataset overview metrics

---

📈 Interactive Visualization Engine

• Plotly dashboards
• Histograms
• Boxplots
• Pie charts
• Correlation heatmaps
• Missing value charts
• Interactive analytics dashboards

---

📄 Reporting System

• Unified HTML analytics dashboard
• Validation summaries
• Dataset overview cards
• Interactive visualizations
• Automated report generation

---

🖥 Professional CLI Toolkit

MasterClean now supports multiple commands.

Full Automated Pipeline

masterclean clean data.csv

Runs:

• cleaning
• optimization
• validation
• profiling
• visualization
• reporting
• exporting

---

Validation Only

masterclean validate data.csv

---

Dataset Profiling

masterclean profile data.csv

---

Dashboard Generation

masterclean dashboard data.csv

---

Show Version

masterclean version

---

📦 Installation

Install from PyPI

pip install masterclean

---

Upgrade to Latest Version

pip install --upgrade masterclean

---

🐍 Python Usage

from masterclean import *

df, file_extension = read_file("data.csv")

df = clean_data(df)

df = optimize_dtypes(df)

warnings = validate_data(df)

profile = generate_profile(df)

charts = generate_charts(df)

generate_report(
    df=df,
    warnings=warnings,
    profile=profile,
    charts=charts
)

export_data(
    df,
    "cleaned_data",
    file_extension
)

---

📂 Supported File Formats

Format	Supported
CSV	✅
XLSX	✅
XLS	✅

---

🔄 Same-Format Export System

MasterClean automatically preserves output format.

Input	Output
CSV	cleaned_data.csv
XLSX	cleaned_data.xlsx
XLS	cleaned_data.xlsx

---

📊 Example Validation Output

VALIDATION WARNINGS
========================================

⚠ Negative values found in 'salary' (3 rows)

⚠ Invalid email values found in 'email' (5 rows)

⚠ High duplicate rows detected (14.2%)

⚠ Mixed datatypes detected in 'age'

---

🏗 Architecture

Read
   ↓
Clean
   ↓
Optimize
   ↓
Validate
   ↓
Profile
   ↓
Visualize
   ↓
Report
   ↓
Export

---

📁 Project Structure

masterclean/
│
├── cleaner.py
├── validator.py
├── datatypes.py
├── profiler.py
├── visualizer.py
├── report.py
├── exporter.py
├── reader.py
├── cli.py
├── __init__.py
│
tests/
│
├── test_cleaner.py
├── test_validator.py
├── test_reader.py
├── test_report.py
├── test_visualizer.py
│
.github/workflows/
│
└── tests.yml

---

🧪 Testing

Run tests using:

python -m pytest

---

🔄 CI/CD

MasterClean uses GitHub Actions for:

• automated testing
• dependency validation
• continuous integration

---

🛣 Roadmap

Future improvements planned:

• Streamlit web application
• AI-powered cleaning suggestions
• Large dataset optimization
• Schema validation engine
• Cloud deployment support
• Plugin architecture
• Real-time analytics dashboards

---

🤝 Contributing

Contributions are welcome.

You can:

• report bugs
• suggest features
• improve documentation
• submit pull requests

---

📄 License

MIT License

---

👨‍💻 Author

Mohamed Faisal Maraicar N

GitHub: https://github.com/MohamedFaisal-11/masterclean