mlevalu 2.0.0

Zero-code ML Evaluation, Debugging, and Explanation Toolkit

Smart Eval

Zero-code ML Evaluation, Debugging, and Explanation Toolkit

A production-ready Python library that makes ML model evaluation, debugging, and interpretation effortless.

Think beyond metrics → auto-evaluation + intelligence + actionable insights.

🎯 Features

✨ Automatic Task Detection

Detects whether your problem is classification or regression automatically.

📊 Comprehensive Metrics

• Classification: Accuracy, Precision, Recall, F1-Score, Confusion Matrix, Class Imbalance Detection
• Regression: MAE, MSE, RMSE, R² Score, MAPE

🧠 Intelligent Insights Engine

Generates human-readable insights that go beyond raw numbers:

• Detects overfitting / underfitting
• Identifies class imbalance issues
• Highlights false positive / false negative patterns
• Suggests actionable improvements

🔧 Model Debugging Engine

Comprehensive debugging to catch common issues:

• Missing value detection
• Feature scaling issues
• Overfitting detection (train vs test gap)
• Data quality assessment
• Feature analysis and statistics

🎨 Beautiful Terminal Output

Colored, formatted output with:

• Professional report layout
• Confusion matrices
• Color-coded warnings and suggestions
• Clear visual hierarchy

💻 CLI Support

Evaluate models directly from the command line:

smart-eval data.csv --true-col y_true --pred-col y_pred

🚀 Quick Start

Installation

pip install smart-eval

Or from source:

git clone https://github.com/your-org/smart_eval
cd smart_eval
pip install -e .

Basic Usage

import smart_eval

# Classification example
y_true = [0, 1, 1, 0, 1, 1, 0, 0, 1, 0]
y_pred = [0, 1, 0, 0, 1, 1, 0, 0, 1, 0]

# One line evaluation with full insights
results = smart_eval.evaluate(y_true, y_pred)

Output Example

======================================================================
  SMART EVALUATION REPORT
======================================================================

Task Information
──────────────
  Task Type........... classification
  Number of Classes... 2
  Is Binary........... True

Performance Metrics
──────────────
  Accuracy.............. 0.8000
  Precision............. 0.8000
  Recall................ 0.8000
  F1 score.............. 0.8000

Summary
──────────────
  Good model performance with room for improvement

Metrics Interpretation
──────────────
  f1_score: Good balance between precision and recall

Regression Example

import smart_eval
import numpy as np

# Regression example
y_true = np.array([3.0, -0.5, 2.0, 7.0])
y_pred = np.array([2.5, 0.0, 2.0, 8.0])

results = smart_eval.evaluate(y_true, y_pred)
# Outputs: MAE, RMSE, R², MAPE and intelligent insights

Model Debugging

import smart_eval
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

# Train model
X = np.random.randn(200, 10)
y = np.random.randint(0, 2, 200)
X_train, X_test, y_train, y_test = train_test_split(X, y)

model = RandomForestClassifier()
model.fit(X_train, y_train)

# Debug the model
debug_report = smart_eval.debug(model, X_train, y_train, X_test, y_test)
# Detects overfitting, scaling issues, missing values, etc.

Getting Only Metrics (No Output)

# Quick evaluation - returns metrics only
metrics = smart_eval.quick_eval(y_true, y_pred)
print(metrics["accuracy"])  # 0.8

📚 API Reference

Main Functions

evaluate(y_true, y_pred, verbose=True)

Main evaluation function with automatic task detection.

Parameters:

• y_true: Ground truth labels/values
• y_pred: Model predictions
• verbose: Print colored output (default: True)

Returns: Dictionary with task_info, metrics, and insights

debug(model=None, X_train=None, y_train=None, X_test=None, y_test=None, verbose=True)

Debug model for common issues.

Parameters:

• model: Trained model (optional)
• X_train: Training features
• y_train: Training labels
• X_test: Test features
• y_test: Test labels
• verbose: Print output (default: True)

Returns: Debug report with warnings and recommendations

quick_eval(y_true, y_pred)

Quick evaluation returning only metrics without output.

Returns: Dictionary of metrics

Core Classes (Advanced Usage)

from smart_eval.core.detector import TaskDetector
from smart_eval.core.metrics import MetricsCalculator
from smart_eval.core.insights import InsightGenerator
from smart_eval.debugger import ModelDebugger

# Task detection
detector = TaskDetector()
task_info = detector.detect(y_true, y_pred)

# Calculate metrics
calculator = MetricsCalculator()
metrics = calculator.calculate_classification_metrics(y_true, y_pred)

# Generate insights
generator = InsightGenerator()
insights = generator.generate_classification_insights(y_true, y_pred, metrics, num_classes=2, is_binary=True)

# Debug model
debugger = ModelDebugger()
report = debugger.debug(model, X_train, y_train, X_test, y_test)

🎯 Classification Features

Automatic Detection

• Binary classification
• Multi-class classification
• Class imbalance warnings

Metrics

• Accuracy: Overall correctness
• Precision: False positive rate
• Recall: False negative rate
• F1-Score: Balance between precision and recall
• Confusion Matrix: Per-class performance

Insights Generated

• ✓ Overfitting detection
• ✓ Class imbalance warnings with severity levels
• ✓ Precision vs Recall analysis
• ✓ Model constant prediction detection
• ✓ Per-class performance hints
• ✓ Dataset size adequacy analysis

📈 Regression Features

Metrics

• MAE: Mean Absolute Error
• MSE: Mean Squared Error
• RMSE: Root Mean Squared Error
• R² Score: Coefficient of determination
• MAPE: Mean Absolute Percentage Error

Insights Generated

• ✓ R² score interpretation with severity
• ✓ Negative R² detection (model worse than baseline)
• ✓ Outlier prediction detection
• ✓ Systematic bias detection
• ✓ Constant prediction detection
• ✓ Over/underfitting heuristics

🔍 Debugging Features

• ✓ Missing value detection
• ✓ Feature scaling analysis
• ✓ Overfitting/underfitting detection
• ✓ Feature statistics (mean, std, min, max)
• ✓ Constant feature detection
• ✓ High variance feature detection
• ✓ Train/test performance gap analysis

💡 Use Cases

1. Quick Model Validation: Evaluate models in notebooks instantly
2. Model Comparison: Compare different models side-by-side
3. Debugging: Identify why models underperform
4. Production Monitoring: Monitor model performance over time
5. Data Quality: Verify data preprocessing pipeline
6. Teaching: Teach ML concepts with immediate visual feedback

🛠️ CLI Usage

Basic Evaluation

smart-eval predictions.csv --true-col target --pred-col prediction

Save Results to JSON

smart-eval predictions.csv -o results.json

Quiet Mode (no colors)

smart-eval predictions.csv --quiet

Custom Column Names

smart-eval data.csv --true-col ground_truth --pred-col model_output

📋 CSV Format Example

Your CSV file should have columns for true labels and predictions:

y_true,y_pred
0,0
1,1
1,0
0,0
1,1

🔗 Integration Examples

With scikit-learn

import smart_eval
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y)
model = RandomForestClassifier()
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
smart_eval.evaluate(y_test, y_pred)

With XGBoost

import smart_eval
import xgboost as xgb

model = xgb.XGBClassifier()
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
smart_eval.evaluate(y_test, y_pred)

With TensorFlow/Keras

import smart_eval
from tensorflow import keras

model = keras.Sequential([...])
model.fit(X_train, y_train)

y_pred = model.predict(X_test)
smart_eval.evaluate(y_test, y_pred)

With PyTorch

import smart_eval
import torch

# ... train model ...
y_pred = model(X_test).argmax(dim=1).numpy()
smart_eval.evaluate(y_test.numpy(), y_pred)

🎨 Customization

Disable Colors

from smart_eval.visualizer import ColorCodes
ColorCodes.disable()

smart_eval.evaluate(y_true, y_pred)

Access Raw Data

results = smart_eval.evaluate(y_true, y_pred, verbose=False)

# Access individual components
metrics = results["metrics"]
insights = results["insights"]
task_info = results["task_info"]

🐛 Troubleshooting

ImportError: No module named 'smart_eval'

pip install smart-eval
# or from source:
pip install -e .

"Column not found" in CLI

Check that your column names match your CSV:

smart-eval data.csv --true-col correct_name --pred-col prediction_name

No insights generated

Ensure you have sufficient data (minimum 10 samples recommended)

📝 Requirements

• Python >= 3.7
• numpy >= 1.19.0
• scikit-learn >= 0.24.0
• pandas >= 1.1.0

📄 License

MIT License - see LICENSE file for details

🤝 Contributing

Contributions are welcome! Please see CONTRIBUTING.md

🙏 Acknowledgments

Built with ❤️ for the ML community

📞 Support

• 📧 Email: [e-mail]
• 🐛 Issues: GitHub Issues
• 💬 Discussions: GitHub Discussions

---

Smart Eval - Making ML evaluation intelligent and effortless ⚡