✨ Welcome to VeriData ✨
An intuitive, high-level toolkit for cleaning and validating pandas DataFrames that accelerates data science and machine learning workflows.
🤔 Why VeriData?
Data scientists spend approximately 80% of their time on data cleaning and preprocessing tasks. This is time that could be better spent on analysis, modeling, and generating insights. DataCleanX solves this problem by automating the most common data cleaning workflows with simple, intuitive functions.
Key Benefits:
- ⏰ Save Time: Transform hours of repetitive cleaning code into a single function call
- 🎯 Enforce Best Practices: Built-in validation and robust statistical methods
- 📖 Improve Readability: Replace complex pandas chains with clear, expressive functions
- 🔧 Stay Flexible: Use the all-in-one
clean()function or individual modules as needed
🚀 Example in Action
See the power of DataCleanX in transforming messy, real-world data into analysis-ready datasets.
Before VeriData:
import pandas as pd
import numpy as np
# A sample of messy, real-world data
data = {
'age': [25, 34, 999, np.nan, 28],
'salary': [50000, 80000, 5000000, 75000, np.nan],
'department': [' Engineering!! ', 'Sales', 'MARKETING', 'sales', 'Engineering'],
'review': ['Excellent work!', 'N/A', 'Great performance.', '', 'Outstanding!!!']
}
df_messy = pd.DataFrame(data)
print(df_messy)
age salary department review
0 25.0 50000.0 Engineering!! Excellent work!
1 34.0 80000.0 Sales N/A
2 999.0 5000000.0 MARKETING Great performance.
3 NaN 75000.0 sales
4 28.0 NaN Engineering Outstanding!!!
After VeriData:
import veridata as vd
# Clean the entire DataFrame with one powerful function
df_clean = vd.clean(df_messy)
print(df_clean)
age salary department review
0 25.0 50000.0 Engineering excellent work
1 34.0 80000.0 Sales great performance
3 29.0 75000.0 Sales
4 28.0 68333.3 Engineering outstanding
Notice how DataCleanX automatically handled the outliers, missing values, text inconsistencies, and rare categories - all in one line of code!
💡 The Philosophy: Engine vs. Dashboard
Think of pandas as a powerful car engine - it gives you incredible control and performance, but requires mechanical expertise to operate effectively. VeriData is like having a smart car dashboard - it uses that same powerful engine under the hood, but provides an intuitive, user-friendly interface that lets you focus on where you're going rather than how the engine works.
VeriData doesn't replace pandas; it makes pandas more accessible and efficient for everyday data cleaning tasks.
🎯 Key Features
- 🧹 Comprehensive Cleaning: Powerful one-liner
clean()function for complete data preprocessing workflows - 📊 Missing Value Imputation: Handle missing data with strategies including mean, median, mode, and advanced KNN imputation
- 🎯 Robust Outlier Detection: Detect and remove outliers using statistical methods (IQR, MAD) and isolation techniques
- 📝 Advanced Text Cleaning: Automatically lowercase, remove punctuation, digits, whitespace, and common stopwords
- 🏷️ Smart Categorical Handling: Group rare categories and encode features for machine learning (one-hot, label encoding)
- ✅ Schema Validation: Enforce data types and automatically detect schemas to prevent downstream errors
- 📈 Data Quality Visualizations: Generate comprehensive reports with heatmaps, distribution plots, and correlation matrices
- 🔧 Duplicate Detection: Intelligent identification and removal of duplicate records
- 🎨 Flexible API: Use individual functions for granular control or the all-in-one solution for speed
🚀 Getting Started
Installation
Install VeriData directly from PyPI:
pip install veridata
Quick Usage
import veridata as vd
import pandas as pd
# Load your messy data
df = pd.read_csv('your_messy_data.csv')
# Option 1: Clean everything at once
df_clean = vd.clean(df)
# Option 2: Use individual functions for more control
df_filled = vd.handle_missing(df, strategy='median')
df_no_outliers = vd.remove_outliers(df_filled, method='iqr')
df_final = vd.clean_text(df_no_outliers, column='text_column')
# Generate a data quality report
report = vd.generate_report(df)
print(report)
📚 Documentation
For comprehensive guides, tutorials, and detailed API documentation, explore our documentation:
- 📦 Installation Guide - Detailed setup instructions and requirements
- 📖 User Guide - Step-by-step tutorials and best practices
- 🔍 API Reference - Complete function documentation with examples
- 🤝 Contributing Guide - How to contribute to DataCleanX development
🤝 Contributing
We welcome contributions from the community! Whether you're fixing bugs, adding new features, improving documentation, or sharing feedback, your involvement helps make VeriData better for everyone.
Ready to contribute? Check out our Contributing Guide to get started. We've made the process as smooth as possible and provide guidance for contributors of all experience levels.
📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
Metadata
Release files for VeriData 0.1.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| veridata-0.1.0.tar.gz | 12.4 kB | Without provenance | Details for veridata-0.1.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| veridata-0.1.0-py3-none-any.whl | Python 3 | none | any | Details for veridata-0.1.0-py3-none-any.whl |
Total release size: 24.2 kB
Release files / veridata-0.1.0.tar.gz
| Download URL | veridata-0.1.0.tar.gz |
|---|---|
| Size | 12.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
671e603aa51333f798adf466d1cbbba425f789534ff35ddeb2570cf47af1aa78
|
|
BLAKE2b-256 checksum How to use checksums |
b7968fd8ff56260f220efcec78669dc50fa8a8b7cea3c60e515a93225f6262bb
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.7
|
Release files / veridata-0.1.0-py3-none-any.whl
| Download URL | veridata-0.1.0-py3-none-any.whl |
|---|---|
| Size | 11.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
9e172b4afc9889c29d35bf88519d4035c1c114fd9718a9e9f2252a89793b57c2
|
|
BLAKE2b-256 checksum How to use checksums |
f260cc9b0e6e9be1f49f188cb4eeea9916a0bd127eaf6d0da14259694df663fc
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.11.7
|
Security for VeriData 0.1.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.