Serverlex ⚡

The Production-Safe Machine Learning Preprocessing, Diagnostics, & Serving Platform

Serverlex is an end-to-end Python library that takes raw, dirty data and transforms it into model-ready features, baseline benchmarks, tuned algorithms, and live production endpoints—with mathematical guarantees against data leakage, train/serve skew, and corrupted inference.

---

🚀 Key Highlights & Guarantees

• Zero Data Leakage: Pipeline parameters (medians, modes, one-hot category vocabularies, scaler stats) are strictly computed on training splits and locked during inference.
• Exact Train/Serve Parity: Predict on single records (dict), small micro-batches, or large DataFrames with exact prediction equivalence and zero categorical drift.
• Degenerate-Safe Outlier Cleaning: Cell-wise IQR and Z-score handling protects binary flags, class targets, and degenerate discrete distributions from collapsing bounds.
• Enterprise Serving Security: Cryptographic SHA-256 sidecar verification and HMAC-SHA256 authenticity checks prevent untrusted model deserialization.
• Statistical Drift Monitoring: Combines Kolmogorov-Smirnov, Chi-Square, Wasserstein distance, and finite-sample corrected Population Stability Index (PSI) to eliminate small-batch false alarms.
• High-Throughput Serving: Sub-10ms p50 single-record inference latency ready for microservices and API gateways.

---

📦 Installation

Install the official package from PyPI:

pip install serverlex

Or Install Latest Development Branch:

pip install git+https://github.com/Uwais-ML/Serverlex.git

---

🏗️ End-to-End Architecture

flowchart TD
    subgraph 1. Ingestion & Quality Audit
        A["Raw Data (CSV / DataFrame / JSON)"] --> B["DataGuard: Pre-Flight Safety Audit"]
        B --> C["Clean: Sanitation Engine"]
        C <--> D["Visualize: Server-Safe EDA"]
    end

    subgraph 2. Leakage-Free Preprocessing
        C --> E["MLPipeline: fit() / transform()"]
        E --> F["FeatureSelector: Mutual Info & Collinearity"]
    end

    subgraph 3. Modeling & Optimization
        F --> G["MLPipeline.evaluate_baseline_models()"]
        G --> H["ModelHyperparameter: Unbiased Nested CV"]
        G --> I["EnsembleBuilder: Stacking & Voting"]
    end

    subgraph 4. Diagnostics & Live Serving
        H & I --> J["ModelEvaluator: Metrics Dict & Curves"]
        H & I --> K["ModelExplainer: Feature Importances"]
        E & H --> L["ModelDeployer: Signed Artifact Bundle"]
        L --> M["Live Raw Inference: deployer.predict(record)"]
        L --> N["DriftDetector: KS / Chi2 / Binned PSI Alert"]
    end

---

⚡ 60-Second Quickstart

import pandas as pd
from serverlex import (
    DataGuard,
    MLPipeline,
    ModelHyperparameter,
    ModelDeployer,
    DriftDetector
)

# 1. Load raw dataset
df = pd.read_csv("customers.csv")

# 2. Automated Quality Audit (catches ID columns, target leakage, class imbalance)
guard = DataGuard(df)
audit_report = guard.run_audit(target="churn")

# 3. Fit Pipeline on Training Data (Zero Data Leakage)
train_df = df.iloc[:800].copy()
test_df = df.iloc[800:].copy()

pipeline = MLPipeline()
pipeline.fit(
    train_df,
    target="churn",
    encode="onehot",
    scale="standard",
    data_time=["signup_date", "%Y-%m-%d"],
    numeric_string_cols=["monthly_bill"]
)

X_train, y_train = pipeline.transform(train_df)
X_test, y_test = pipeline.transform(test_df)

# 4. Benchmark Baseline Models (Stratified CV)
leaderboard = pipeline.evaluate_baseline_models(cv=5)

# 5. Tune Best Estimator with Nested Cross-Validation
from sklearn.ensemble import RandomForestClassifier
tuner = ModelHyperparameter()
param_grid = tuner.get_default_param_grid("random_forest")

tune_res = tuner.tune_nested_cv(
    estimator=RandomForestClassifier(random_state=42),
    param_grid=param_grid,
    X=X_train,
    y=y_train,
    outer_cv=5,
    inner_cv=3
)
best_model = tune_res["best_estimator"]

# 6. Bundle & Cryptographically Sign for Production Serving
deployer = ModelDeployer(pipeline=pipeline, model=best_model)
deployer.save("churn_bundle.joblib", compute_hash=True, hmac_key="production_secret_key")

# Later in Production: Load and predict directly on raw payload!
loaded_deployer = ModelDeployer.load("churn_bundle.joblib", hmac_key="production_secret_key")
raw_payload = {
    "signup_date": "2026-04-01",
    "monthly_bill": "$89.50",
    "device": "laptop"
}
prediction = loaded_deployer.predict(raw_payload, strict=True)
print("Predicted Churn:", prediction[0])

---

🛠️ Deep-Dive Module Guides

1. serverlex.preprocessing.clean.Clean

Robust sanitation that cleans data without corrupting valid features:

from serverlex import Clean

cleaner = Clean(df)

# Cell-wise IQR outlier handling with automatic binary & target protection:
cleaned_df = cleaner.total_clean(
    outlier_method="IQR",
    outlier_param="m",           # 'm' = mean imputation, 'median' = median, 'r' = row drop
    corr_target="churn",         # Automatically excludes target from outlier clipping
    numeric_string_cols=["bill"],# Strips '$' and ',' safely
    null_threshold=0.7           # Drops columns with > 70% missing
)

2. serverlex.diagnostics.dataguard.DataGuard

Pre-flight health and leakage auditor:

from serverlex import DataGuard

guard = DataGuard(df, target="churn")
report = guard.run_audit(leakage_threshold=0.90, id_threshold=0.95)
# Returns DataFrame with Status ('PASS', 'WARN', 'FAIL') and actionable remediation advice.

3. serverlex.serving.deployer.ModelDeployer

Enterprise model bundle deployer with schema auto-resilience and security:

from serverlex import ModelDeployer

deployer = ModelDeployer(pipeline=pipeline, model=model)

# Cryptographic signing with SHA-256 or HMAC
deployer.save("bundle.joblib", hmac_key="secure_key")

# Safe loading (verifies authenticity before unpickling)
live_deployer = ModelDeployer.load("bundle.joblib", hmac_key="secure_key")

# Strict mode: raises ValueError on unseen categories or invalid types
preds = live_deployer.predict({"city": "Kolkata"}, strict=True)

4. serverlex.diagnostics.drift.DriftDetector

Finite-sample corrected distribution drift monitor:

from serverlex import DriftDetector

detector = DriftDetector(reference_data=train_df, alpha=0.01, psi_threshold=0.20)
drift_report = detector.detect_drift(production_batch_df)

# Automatically guards against small sample size noise (n < 50)
# Flags missing features and constant shifts with High severity
print(drift_report[["Feature", "Type", "PSI", "Drift_Detected", "Severity"]])

5. serverlex.diagnostics.evaluation.ModelEvaluator

Multi-task model diagnostic engine with server-safe plotting:

from serverlex import ModelEvaluator

evaluator = ModelEvaluator()
metrics = evaluator.evaluate(best_model, X_test, y_test)
# Returns structured dictionary: {"accuracy": ..., "f1_weighted": ..., "f1_macro": ..., "roc_auc": ...}

# Headless server plotting (saves file, doesn't block server)
evaluator.plot_confusion_matrix(y_test, preds, save_path="confusion_matrix.png")

---

📦 Complete Module Directory

Subpackage	Class	Key Capabilities
serverlex.preprocessing.clean	Clean	Headers, deduplication, cell-wise IQR/Z-score clipping, binary protection.
serverlex.preprocessing.pipeline	MLPipeline	Leakage-free fit/transform, datetime extraction, encoding, scaling.
serverlex.serving.deployer	ModelDeployer	Pipeline-model artifact bundling, HMAC signing, strict-mode inference.
serverlex.diagnostics.dataguard	DataGuard	Target leakage ($r \ge 0.90$), unique ID detection, cardinality audit.
serverlex.diagnostics.drift	DriftDetector	Dual-significance KS, Chi-Square, Wasserstein, and binned PSI drift.
serverlex.diagnostics.evaluation	ModelEvaluator	Metrics dictionary return, OvR multiclass ROC, PR curve, headless plots.
serverlex.diagnostics.explainability	ModelExplainer	Gini importances, Permutation feature importance ranking.
serverlex.modeling.selector	FeatureSelector	Mutual Information feature ranking, collinearity pruning.
serverlex.modeling.hyperparameter	ModelHyperparameter	Non-nested GridSearchCV vs. unbiased Nested Cross-Validation.
serverlex.modeling.ensemble	EnsembleBuilder	Soft/Hard Voting and Stacking ensembles with cross-validated meta-learners.
serverlex.visualization.eda	Visualize	EDA plots (save_path support) and live plt.ion() interactive cleaning.

---

🧪 Testing & Verification

Serverlex features a comprehensive automated test suite with 35 unit and regression tests covering train/serve parity, outlier preservation, crypto authenticity, and drift edge cases:

# Run the test suite:
pytest -v tests/

Reproduce real-world benchmark assertions:

python benchmarks/reproduce_benchmarks.py

---

🏆 Real-World Kaggle Verification

Serverlex has been validated on benchmark datasets:

• Customer Churn (muhammadshahidazeem/customer-churn-dataset):
  • DataGuard detected CustomerID as 100% unique identifier and pruned it.
  • Cell-wise outlier clipping preserved churn distribution with zero label alteration.
  • Baseline Gradient Boosting achieved 99.42% accuracy.
• Walmart Store Sales (mikhail1681/walmart-sales):
  • Extracted temporal features from dates (year, month, day, dayofweek).
  • Random Forest Regressor trained to $R^2 = 0.9455$ ($\text{RMSE} = 133,727$).

---

📄 License

Serverlex is licensed under the MIT License.