TinyShift

TinyShift is a lightweight, sklearn-compatible Python library designed for data drift detection, outlier identification, and MLOps monitoring in production machine learning systems. The library provides modular, easy-to-use tools for detecting when data distributions or model performance change over time, with comprehensive visualization capabilities.

For enterprise-grade solutions, consider Nannyml.

Features

• Data Drift Detection: Categorical and continuous data drift monitoring with multiple distance metrics
• Outlier Detection: HBOS, PCA-based and SPAD outlier detection algorithms
• Classification Model Evaluation: Calibration curves, confusion matrices, score distributions, and production confidence analysis
• Time Series Analysis: Seasonality decomposition, trend analysis, forecasting diagnostics, and forecast stabilization
• Decomposed Forecasting: DTL-based non-seasonal and DMSTL-based multi-seasonal forecasting for panel and long-horizon series
• Probabilistic Demand Forecasting: Two-stage discrete or continuous forecasts, calibrated distributions, forecast evaluation, and inventory optimization
• Forecast Stability: Metrics and interpolation methods for stable forecasting

Technologies Used

• Python 3.10+
• Scikit-learn 1.3.0+
• Pandas 2.3.0+
• NumPy
• SciPy
• Statsmodels 0.14.5+
• Plotly 5.22.0+ (optional, for plotting)

📦 Installation

Install the core package with pip:

pip install tinyshift

Or with uv:

uv add tinyshift

Optional module extras

TinyShift now separates optional capabilities into extras so you can install only what you need. Some functions also use lazy importing so optional dependencies are loaded only when they are actually used, helping keep the import surface lightweight and avoiding unnecessary dependency overhead.

• series: forecasting and series-specific dependencies

pip install "tinyshift[series]"
# or
uv add "tinyshift[series]"

• plot: interactive plotting and export support

pip install "tinyshift[plot]"
# or
uv add "tinyshift[plot]"

• notebook: notebook support

pip install "tinyshift[notebook]"
# or
uv add "tinyshift[notebook]"

• all: install all optional extras

pip install "tinyshift[all]"
# or
uv add "tinyshift[all]"

Development installation

Clone the repository and install from source:

git clone https://github.com/HeyLucasLeao/tinyshift.git
cd tinyshift
pip install -e ".[dev]"

With uv:

uv sync --extra dev

📖 Quick Start

1. Categorical Data Drift Detection

TinyShift provides sklearn-compatible drift detectors that follow the familiar fit() and score() pattern:

import pandas as pd
from tinyshift.drift import CatDrift

# Load your data
df = pd.read_csv("data.csv")
reference_data = df[df["date"] < '2024-07-01']
analysis_data = df[df["date"] >= '2024-07-01'] 

# Initialize and fit the drift detector
detector = CatDrift(
    freq="D",                    # Daily frequency
    func="chebyshev",           # Distance metric
    drift_limit="auto",         # Automatic threshold detection
    method="expanding"          # Comparison method
)

# Fit on reference data
detector.fit(reference_data)

# Score new data for drift
drift_scores = detector.predict(analysis_data)
print(drift_scores)

Available distance metrics for categorical data:

• "chebyshev": Maximum absolute difference between distributions
• "jensenshannon": Jensen-Shannon divergence
• "psi": Population Stability Index

2. Continuous Data Drift Detection

For numerical features, use the continuous drift detector:

from tinyshift.drift import ConDrift

# Initialize continuous drift detector
detector = ConDrift(
    freq="W",                   # Weekly frequency  
    func="ws",                  # Wasserstein distance
    drift_limit="auto",
    method="expanding"
)

# Fit and score
detector.fit(reference_data)
drift_predicts = detector.predict(analysis_data)

3. Outlier Detection

TinyShift includes sklearn-compatible outlier detection algorithms:

from tinyshift.outlier import SPAD, HBOS, PCAReconstructionError

# SPAD (Simple Probabilistic Anomaly Detector)
spad = SPAD(plus=True)
spad.fit(X_train)

outlier_scores = spad.decision_function(X_test)
outlier_labels = spad.predict(X_test)

# HBOS (Histogram-Based Outlier Score)
hbos = HBOS(dynamic_bins=True)
hbos.fit(X_train, nbins="fd")
scores = hbos.decision_function(X_test)
outlier_labels = hbos.predict(X_test)

# PCA-based outlier detection
pca_detector = PCAReconstructionError(n_components=None)
pca_detector.fit(X_train)
pca_scores = pca_detector.decision_function(X_test)
pca_outlier_labels = pca_detector.predict(X_test)

4. Binary Classification Model Evaluation

Evaluate and visualize classification model performance for production deployment:

from tinyshift.plot import (
    reliability_curve,
    score_distribution, 
    confusion_matrix,
    efficiency_curve,
    beta_confidence_analysis
)

# Model calibration assessment
reliability_curve(
    clf=classifier,
    X=X_test,
    y=y_test,
    model_name="RandomForestClassifier",
    n_bins=15
)

# Analyze prediction confidence patterns
score_distribution(clf, X_test, nbins=20)

# Performance evaluation with interactive confusion matrix
confusion_matrix(clf, X_test, y_test, percentage_by_class=True)

# Conformal prediction analysis
efficiency_curve(conformal_classifier, X_test)

# Production deployment confidence analysis
beta_confidence_analysis(
    alpha=95, 
    beta_param=5, 
    fig_type=None
)

5. Time Series Analysis and Diagnostics

TinyShift provides comprehensive time series analysis capabilities:

from tinyshift.plot import seasonal_decompose
from tinyshift.series import (
    trend_significance, 
    foreca, 
    sample_entropy,
    permutation_entropy,
    theoretical_limit,
    hurst_exponent,
    hampel_filter,
    bollinger_bands
)

seasonal_decompose(
    time_series, 
    periods=[7, 365],  # Weekly and yearly patterns
    width=1200, 
    height=800
)

# Test for significant trends
r_squared, p_value = trend_significance(time_series)

# Assess forecastability
forecastability = foreca(time_series)
print(f"Forecastability (Omega): {forecastability}")

# Measure complexity and regularity
complexity = sample_entropy(time_series, m=2, tolerance=0.2)
print(f"Sample Entropy: {complexity}")

# Measure ordinal complexity
perm_entropy = permutation_entropy(time_series, m=3, delay=1, normalize=True)
print(f"Permutation Entropy: {perm_entropy}")

# Calculate theoretical predictability limit
theo_limit = theoretical_limit(time_series, m=3, delay=1)
print(f"Theoretical Limit (Πmax): {theo_limit}")

# Detect long-term memory
hurst, p_value = hurst_exponent(time_series)
print(f"Hurst Exponent: {hurst}, P-value: {p_value}")

# Outlier detection in time series
outliers = hampel_filter(time_series, window_size=5)
outliers = bollinger_bands(time_series, window_size=20)

# Plot lag analysis with PAMI (Permutation Auto-Mutual Information)
from tinyshift.plot import pami
pami(time_series, nlags=20, m=3, delay=1, normalize=False)

6. Forecast Accuracy Metrics

TinyShift also includes forecast evaluation utilities in the series metrics module, implemented in tinyshift/series/metric.py. This module provides functions such as wape, pbias, score, rmae, and fva_rmae to compare forecasting models using aggregate error, bias, and baseline-relative performance:

import pandas as pd
from tinyshift.series import wape, pbias, score, rmae, fva_rmae

# Example evaluation dataframe
# df must contain actual values in the 'y' column and model predictions as columns

wape_df = wape(df, models=["model_a", "model_b"], id_col="unique_id", target_col="y")
pbias_df = pbias(df, models=["model_a", "model_b"], id_col="unique_id", target_col="y")
score_df = score(df, models=["model_a", "model_b"], id_col="unique_id", target_col="y")
rmae_df = rmae(df, models=["model_a", "model_b"], baseline_col="naive", id_col="unique_id", target_col="y")

# Single-series Forecast Value Added (FVA) analysis
fva = fva_rmae(y_true, y_pred, nlags=1, baseline_type="naive")
print(f"FVA RMAE: {fva}")

These utilities cover:

• wape: weighted absolute percentage error for overall accuracy
• pbias: percent bias to detect over- or under-forecasting
• score: composite score combining WAPE and absolute bias
• economic_loss: financial loss from understock and overstock costs
• rmae: relative mean absolute error versus a baseline model
• fva_rmae: lead-time-aware RMAE for Forecast Value Added analysis

7. Forecast Stability and Interpolation

TinyShift includes forecast stability metrics and interpolation methods:

from tinyshift.series import (
    forecast_instability,          # Period-over-period forecast instability
    macv, mach,           # Mean Absolute Change metrics
    mascv, masch,         # Mean Absolute Scaled Change metrics
    rmsscv, rmssch,       # Root Mean Squared Scaled Change metrics
    vi, hpi, hfi          # Interpolation methods
)

# Calculate forecast stability metrics
vertical_stability = macv(y_hat, y_hat_t_minus_1)
horizontal_stability = mach(y_hat) 

# Calculate period-over-period forecast variability (instability)
# `df` should contain `unique_id`, `ds` (ordered dates) and model forecast columns.
# Example: `forecast_instability(df, models=["model_a", "model_b"], ds_col="ds")`
instability_scores = forecast_instability(df, models=["model_a"], ds_col="ds")

# Scaled stability metrics
scaled_v_stability = mascv(y_train, y_hat, y_hat_t_minus_1, seasonality=12)
scaled_h_stability = masch(y_train, y_hat, seasonality=12)

# Apply forecast stabilization techniques
# Vertical Interpolation
stable_forecast = vi(y_hat, anchor, w_s=0.3)

# Horizontal Partial Interpolation
smooth_forecast = hpi(y_hat, w_s=0.4)

# Horizontal Full Interpolation
fully_stable_forecast = hfi(y_hat, w_s=0.5)

8. Preprocessing, Features and Forecasting

These responsibilities are exposed through focused packages:

• filter_features_by_vif — remove highly correlated features using VIF filtering
• FeatureResidualizer — residualize correlated predictors while preserving information
• RobustGaussianScaler — robust scaling with winsorization and power transforms
• DTLWrapper — decomposed LOWESS trend plus ML residual forecasting for non-seasonal data
• DMSTLWrapper — decomposed MSTL forecasting wrapper for panel/multi-seasonal data
• TwoStageForecasterWrapper — configurable Negative Binomial or Gamma predictive distributions and inventory optimization on top of MLForecast
• relative_strength_index, standardize_returns, fourier_seasonality, estimate_history_length — feature engineering helpers for time-series models

Use tinyshift.preprocessing for data transforms and tinyshift.forecasting for estimators and predictive distributions.

9. Advanced Modeling Tools

from tinyshift.preprocessing import FeatureResidualizer, filter_features_by_vif
from tinyshift.stats import BootstrapBCA

# Detect multicollinearity
mask = filter_features_by_vif(X_train, threshold=5.0)
selected_columns = X_train.columns[mask]
X_train = X_train.loc[:, selected_columns].astype(float)
X_test = X_test.loc[:, selected_columns].astype(float)

# Residualize correlated features using the training fit
residualizer = FeatureResidualizer(corrcoef=0.70)
X_train.loc[:, :] = residualizer.fit_transform(X_train)

X_test.loc[:, :] = residualizer.transform(X_test)

# Bootstrap confidence intervals
confidence_interval = BootstrapBCA.compute_interval(
    data,
    confidence_level=0.95,
    statistic=np.mean,
    n_resamples=1000,
    random_state=42,
)

10. Decomposed Forecasting with DTL and DMSTL

TinyShift includes decomposed forecasting wrappers for non-seasonal and multi-seasonal panel data. DTLWrapper extracts a robust LOWESS trend and models residuals with MLForecast:

from tinyshift.forecasting import DTLWrapper
from mlforecast import MLForecast
from sklearn.ensemble import RandomForestRegressor

def residual_model_callable(nlags, freq):
    return MLForecast(
        models=[RandomForestRegressor(random_state=42)],
        lags=nlags,
        freq=freq,
    )

model = DTLWrapper(
    residual_model_callable=residual_model_callable,
    freq="D",
    nlags="auto",
    pami_params={"max_tau": 48, "m": 3, "delay": 1},
    trend_frac=0.2,
    robust=True,
)
model.fit(df, id_col="unique_id", time_col="ds", target_col="y")
preds = model.predict(h=14, stabilization_method="hfi", w_s=0.2)

For multiple seasonalities, use DMSTLWrapper:

from tinyshift.forecasting import DMSTLWrapper
from mlforecast import MLForecast
from sklearn.ensemble import RandomForestRegressor

def residual_model_callable(nlags, freq):
    return MLForecast(
        models=[RandomForestRegressor(random_state=42)],
        lags=nlags,
        freq=freq,
    )

model = DMSTLWrapper(
    residual_model_callable=residual_model_callable,
    freq="D",
    season_length="auto",
    seasonal_detection_params={"top_k": 2, "noise_threshold_factor": 1.5},
    nlags="auto",
    pami_params={"max_tau": 48, "m": 3, "delay": 1},
    log_transform=True,
)

model.fit(df, id_col="unique_id", time_col="ds", target_col="y")

preds = model.predict(h=14, stabilization_method="hfi", w_s=0.2)
print(preds.head())

11. Two-Stage Probabilistic Demand Forecasting

TwoStageForecasterWrapper separates the point forecast from uncertainty calibration. An MLForecast model estimates the conditional mean (lambda_t), while temporal cross-validation calibrates global, horizon, series, and series-by-horizon distribution parameters with hierarchical shrinkage. The default Negative Binomial family supports discrete demand and inventory decisions; GammaFamily supports strictly positive continuous targets.

import pandas as pd
from mlforecast import MLForecast
from sklearn.ensemble import RandomForestRegressor
from tinyshift.forecasting import (
    FirstStageForecasterEvaluator,
    NewsvendorOptimizer,
    TwoStageForecasterEvaluator,
    TwoStageForecasterWrapper,
)

fcst = MLForecast(
    models=[RandomForestRegressor(random_state=42)],
    freq="D",
    lags=[1, 7, 14],
)

model = TwoStageForecasterWrapper(fcst)
model.fit(
    df_train,
    id_col="unique_id",
    time_col="ds",
    target_col="y",
    h=14,
    n_windows=5,
)

# Forecast once and derive all probabilistic views from the same object
forecast = model.predict_distribution(h=14)
forecast_df = forecast.to_frame()
quantiles = forecast.ppf([0.50, 0.95])
distribution = forecast.distribution

# Newsvendor-optimal inventory using shortage and holding costs
stock_plan = NewsvendorOptimizer.optimize(
    forecast_df, distribution, underage_cost=10.0, overage_cost=2.0
)

# Exact probabilities from P(Y=0) through P(Y=10)
probabilities = forecast.pmf(range(11))

# Expected value of stocking each additional discrete inventory unit
marginal_value = NewsvendorOptimizer.marginal_benefit(
    forecast_df,
    distribution,
    underage_cost=10.0,
    overage_cost=2.0,
    max_k=10,
)

probability_below_five = forecast.cdf(5)
median = forecast.ppf(0.50)

# Cost columns may be supplied without exogenous forecast features.
costs = pd.DataFrame({"cu": [10.0] * len(forecast), "co": [2.0] * len(forecast)})
stock_plan = NewsvendorOptimizer.optimize(
    forecast_df,
    distribution,
    underage_cost="cu",
    overage_cost="co",
    cost_df=costs,
)

# Continuous alternative; cdf/ppf/interval share the same interface.
from tinyshift.forecasting import GammaFamily

continuous_model = TwoStageForecasterWrapper(fcst, distribution=GammaFamily())

Evaluate only held-out or rolling-origin predictions after joining their actual targets. The first-stage evaluator covers conditional-mean diagnostics and its calibration table; the two-stage evaluator evaluates symmetric quantile pairs as central intervals using MWIS, empirical coverage, and interval width:

mean_metrics = FirstStageForecasterEvaluator.evaluate(backtest_df)
calibration = FirstStageForecasterEvaluator.calibration_table(
    backtest_df, n_bins=10
)
probabilistic_metrics = TwoStageForecasterEvaluator.evaluate(
    backtest_df, quantiles=(0.05, 0.50, 0.95)
)

Persist the fitted wrapper so its base forecaster, selected family, and calibrated per-series dispersion parameters remain together:

import joblib

joblib.dump(model, "two_stage_forecaster.joblib")
restored_model = joblib.load("two_stage_forecaster.joblib")

Only load joblib files from trusted sources.

Negative Binomial targets must contain non-negative integer counts; Gamma targets must be strictly positive. Install the series extra to use this wrapper: pip install "tinyshift[series]".

📁 Project Structure

tinyshift/
├── association_mining/          # Market basket analysis tools
│   ├── README.md                # Module documentation
│   ├── __init__.py              # Package exports
│   ├── analyzer.py              # Transaction pattern analysis
│   └── encoder.py               # Data encoder
├── drift/                       # Data drift detection
│   ├── README.md                # Module documentation
│   ├── __init__.py              # Package exports
│   ├── base.py                  # Base drift detection classes
│   ├── categorical.py           # CatDrift for categorical features
│   └── continuous.py            # ConDrift for numerical features
├── examples/                    # Jupyter notebook examples
│   ├── dmstl.ipynb              # Multi-seasonal forecasting example
│   ├── dtl.ipynb                # Trend/residual forecasting example
│   ├── drift.ipynb              # Drift detection examples
│   ├── outlier.ipynb            # Outlier detection demos
│   ├── power_analysis.ipynb     # Statistical power analysis
│   ├── series.ipynb             # Time series analysis
│   ├── solver.ipynb             # Probabilistic decision example
│   ├── transaction_analyzer.ipynb  # Transaction analysis examples
│   ├── ts_diagnostics.ipynb     # Time series diagnostics
│   └── tsf.ipynb                # Probabilistic forecasting example
├── features/                    # Feature-engineering helpers
│   ├── README.md                # Package documentation
│   ├── __init__.py              # Package exports
│   └── time_series.py           # Time-series feature functions
├── forecasting/                 # Forecasting estimators and distributions
│   ├── README.md                # Package documentation
│   ├── __init__.py              # Public forecasting API
│   ├── dmstl/                   # Multi-seasonal decomposed forecasting
│   ├── dtl/                     # LOWESS trend/residual forecasting
│   └── probabilistic/           # Distributions, calibration and decisions
├── preprocessing/               # Sklearn-compatible data transforms
│   ├── README.md                # Package documentation
│   ├── __init__.py              # Package exports
│   ├── multicollinearity.py     # VIF-based feature filtering
│   ├── residualizer.py          # Correlated-feature residualization
│   └── scaler.py                # Robust Gaussian scaling
├── outlier/                     # Outlier detection algorithms
│   ├── README.md                # Module documentation
│   ├── __init__.py              # Package exports
│   ├── base.py                  # Base outlier detection classes
│   ├── hbos.py                  # Histogram-Based Outlier Score
│   ├── pca.py                   # PCA-based outlier detection
│   └── spad.py                  # Simple Probabilistic Anomaly Detector
├── plot/                        # Visualization capabilities
│   ├── README.md                # Module documentation
│   ├── __init__.py              # Package exports
│   ├── calibration.py           # Binary classification model evaluation
│   ├── correlation.py           # Correlation analysis plots
│   ├── diagnostic.py            # Time series diagnostics plots
│   └── power.py                 # Power analysis and related plots
├── series/                      # Time series analysis tools
│   ├── README.md                # Module documentation
│   ├── __init__.py              # Package exports
│   ├── diagnostic.py            # Time series diagnostics and decomposition
│   ├── forecastability.py       # Forecast quality and complexity metrics
│   ├── interpolation.py         # Forecast stabilization methods
│   ├── metric.py                # Forecast accuracy and stability metrics
│   ├── outlier.py               # Time series outlier detection
│   └── stability.py             # Forecast stability metrics
└── stats/                       # Statistical utilities
    ├── __init__.py              # Package exports
    ├── bootstrap_bca.py         # Bootstrap confidence intervals
    ├── statistical_interval.py  # Statistical interval estimation
    └── utils.py                 # General statistical utilities

Development Setup

git clone https://github.com/HeyLucasLeao/tinyshift.git
cd tinyshift
pip install -e ".[all]"

📋 Requirements

• Python: 3.10+
• Core Dependencies:
  • pandas (>2.3.0)
  • scikit-learn (>1.3.0)
  • statsmodels (>=0.14.5)
• Optional Dependencies:
  • plotly (>5.22.0) - for visualization
  • kaleido (<=0.2.1) - for static plot export
  • nbformat (>=5.10.4) - for notebook support

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Inspired by Nannyml