Skip to main content

Temporal EDA for feature drift, PCA, breakpoints, and trends

Project description

TEDA - Temporal Exploratory Data Analysis

A Python package for temporal feature analysis and drift detection.

Features

1. Feature Overview Table

  • Comprehensive statistics for all features
  • Missing values, zero rates, unique percentages
  • Volatility and PSI (Population Stability Index) tracking
  • Correlation with target variable

2. Monthly PSI Tracking

  • Track Population Stability Index over time
  • Automatic threshold warnings

3. 3D PCA Visualization

  • Monthly mean projection in PCA space
  • Travel distance bar chart showing monthly movement

4. Structural Breakpoint Detection

  • Uses Ruptures library for changepoint detection
  • Monthly aggregation with PELT/Binseg/Dynp algorithms
  • Visual breakpoint markers on time series

5. Feature Importance Evolution

  • Track feature importance over time
  • Multiple models: RandomForest & XGBoost

6. HP Filter Decomposition

  • Hodrick-Prescott filter for trend extraction
  • Seasonal pattern detection
  • Residual analysis

Installation

pip install teda

Or install from source:

git clone https://github.com/tergelitu/teda.git
cd teda
pip install -e .

Requirements

  • Python 3.8+
  • pandas >= 1.3.0
  • numpy >= 1.21.0
  • plotly >= 5.0.0
  • scikit-learn >= 1.0.0
  • xgboost >= 1.5.0
  • ruptures >= 1.1.0
  • statsmodels >= 0.13.0
  • ipywidgets >= 7.6.0

📖 Quick Start

See full example usage in teda-demo.ipynb

import pandas as pd
import teda as te

# Load your data
df = pd.read_csv('your_data.csv')

# 1. Feature Overview
te.overview_table(df, date_col='date', target_col='target', exclude_cols=['id'])

# 2. PSI Tracking
te.plot_monthly_psi(df, date_col='date', feature_col='col')

# 3. 3D PCA Visualization
te.plot_monthly_pca(df, date_col='date', exclude_cols=['id', 'target'])

# 4. Structural Breakpoints
te.detect_change_points(df, date_col='date', feature_col='col', model='l2')

# 5. Feature Importance Evolution
te.plot_monthly_importance(df, date_col='date', target_col='target', feature_name='col')

# 6. HP Filter Decomposition
te.plot_hp_decomposition(df, date_col='date', feature_col='col')

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Author

Tergel Munkhbaatar

Use Cases

  • Model Monitoring: Track feature drift in production ML systems
  • Data Quality: Identify distribution shifts and anomalies
  • Feature Engineering: Understand temporal patterns and breakpoints
  • Exploratory Analysis: Quick overview of time-series feature behavior

Star ⭐ this repo if you find it useful!

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

teda_temporal-0.1.0.tar.gz (9.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

teda_temporal-0.1.0-py3-none-any.whl (8.6 kB view details)

Uploaded Python 3

File details

Details for the file teda_temporal-0.1.0.tar.gz.

File metadata

  • Download URL: teda_temporal-0.1.0.tar.gz
  • Upload date:
  • Size: 9.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.4

File hashes

Hashes for teda_temporal-0.1.0.tar.gz
Algorithm Hash digest
SHA256 7bae96a778c3aec7fab0e8d2c90420d4a207d730a20b0f06d52c3a11f3ade960
MD5 fc6e8d16d089a99e348c6feed01c57f9
BLAKE2b-256 7c4ba225de0dc22d5b4c7f316e3ee6cb2686733775a90b1c69a4a5799ab83890

See more details on using hashes here.

File details

Details for the file teda_temporal-0.1.0-py3-none-any.whl.

File metadata

  • Download URL: teda_temporal-0.1.0-py3-none-any.whl
  • Upload date:
  • Size: 8.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.4

File hashes

Hashes for teda_temporal-0.1.0-py3-none-any.whl
Algorithm Hash digest
SHA256 7525b03c78d158c3580752dcd7a5822b6170a25eaf7d19491a9ece7b9fd561b9
MD5 d1b740a5dc24ffd848d350911ff542f0
BLAKE2b-256 a1924b9609a9bf9e9c2f124d5440ee2a697fbb280f681f3e1cb3664e42f0be20

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page