qfeaturelib 0.5.0

High-performance feature engineering library for quantitative investment

QFeatureLib

English | 中文

QFeatureLib is a high-performance, production-grade feature engineering library for quantitative investment. It focuses on financial time series processing with strict handling of future function avoidance, computational efficiency, and rigorous sample splitting.

Key Features

• Zero Future Function: All time-series operations use shift=1 by default to prevent data leakage. The library raises FutureFunctionError if you accidentally try to use future information.
• High Performance: Pure NumPy implementation with vectorized operations, 10-100x faster than pandas.
• Memory Efficient: Uses views instead of copies, supports in-place operations for large-scale panel data.
• Quantitative Finance Focused: Specialized for financial scenarios - suspended stock handling, industry neutralization, market cap neutralization, etc.

Installation

pip install qfeaturelib

For development:

pip install qfeaturelib[dev]

Quick Start

import numpy as np
from qfeaturelib import PanelData
from qfeaturelib.standardization import rolling_zscore, cs_zscore
from qfeaturelib.splitting import RollingWindowSplitter

# Create panel data (T=100 days, N=50 stocks, F=5 features)
values = np.random.randn(100, 50, 5)
dates = np.arange(100)
tickers = [f'STOCK_{i:02d}' for i in range(50)]

panel = PanelData(values, dates, tickers)

# Time-series standardization (rolling Z-score with shift=1 to prevent leakage)
zscore_values = rolling_zscore(
    panel.values[..., 0],  # First feature
    window=20,
    shift=1,  # Use past 20 days only, excluding current moment
)

# Cross-sectional standardization (Z-score across all stocks each day)
cs_values = cs_zscore(panel.values[..., 0])

# Sample splitting for backtesting
splitter = RollingWindowSplitter(
    n_samples=100,
    train_ratio=0.6,
    val_ratio=0.2,
    test_ratio=0.2,
)

for split in splitter.split():
    train_data = zscore_values[split.train]
    val_data = zscore_values[split.val]
    test_data = zscore_values[split.test]
    # Train your model...

Core Modules

1. Time-Series Standardization

Operations along the time dimension with rolling windows:

from qfeaturelib.standardization import (
    rolling_zscore,      # Rolling Z-Score
    rolling_robust_zscore,  # Robust Z-Score using Median/MAD
    rolling_minmax,      # Rolling Min-Max scaling
)

# Parameters explained
result = rolling_zscore(
    data,
    window=20,      # Rolling window size
    shift=1,        # Window end offset (shift=1 excludes current moment)
    outlier_method="squash",  # Outlier handling: 'truncate' or 'squash'
    outlier_bounds=(0.01, 0.99),  # Quantile bounds for outliers
)

For long-format pandas frames with multiple groups (e.g. one row per (ticker, date)), the grouped wrappers — added in 0.2 — standardise each ticker against its own rolling statistics, so a ticker's outliers cannot leak into another ticker's z-scores:

from qfeaturelib import grouped_rolling_zscore, grouped_rolling_minmax

out = grouped_rolling_zscore(
    df,
    group_col="ticker",
    feature_cols=["ret", "vol"],
    # Optional: per-group feature subset
    # group_feature_map={"AAPL": ["ret"], "MSFT": ["ret", "vol"]},
    window=20,
    shift=1,          # default; shift=0 emits FutureFunctionWarning
)

2. Cross-Sectional Standardization

Operations across all assets at each time point:

from qfeaturelib.standardization import (
    cs_zscore,           # Cross-sectional Z-Score
    cs_robust_zscore,    # Cross-sectional robust Z-Score
    cs_minmax,           # Cross-sectional Min-Max
    cs_rank,             # Cross-sectional rank (percentile)
)

# Support for group-wise operations
result = cs_zscore(data, groups=industry_labels)

3. Sample Splitting Engine

Time-series aware train/validation/test splitting:

from qfeaturelib.splitting import (
    RollingWindowSplitter,
    ExpandingWindowSplitter,
    DateAwareRollingSplitter,      # NEW in 0.2 — slice by unique calendar dates
    DateAwareExpandingSplitter,    # NEW in 0.2 — expanding-train variant
)

# Rolling window (fixed training size)
rolling_splitter = RollingWindowSplitter(
    n_samples=1000,
    train_ratio=0.6,
    val_ratio=0.2,
    test_ratio=0.2,
    step=100,  # Roll forward 100 samples each iteration
    gap=0,     # Gap between train/val/test to prevent leakage
)

# Expanding window (growing training size)
expanding_splitter = ExpandingWindowSplitter(
    n_samples=1000,
    train_ratio=0.6,
    val_ratio=0.2,
    test_ratio=0.2,
    step=50,   # Expand by 50 samples each iteration
)

# Use split.apply() to split multiple arrays consistently
for split in rolling_splitter.split():
    (X_train, X_val, X_test), (y_train, y_val, y_test) = split.apply([X, y])

# Multi-asset panels: slice by unique calendar dates so no date is
# split across train / val / test (the row-index splitters above
# silently leak in this case).
date_splitter = DateAwareRollingSplitter(
    dates=df["date"].to_numpy(),
    train_ratio=0.7, val_ratio=0.1, test_ratio=0.2,
)
for split in date_splitter.split():
    (X_tr, X_va, X_te), (y_tr, y_va, y_te) = split.apply([X, y], axis=0)

# Label uses future N periods? Use the per-boundary gaps to embargo
# trading days between consecutive folds and avoid label leakage.
# (Available on all four splitters; ``train_val_gap`` /
# ``val_test_gap`` override ``gap`` independently.)
date_splitter = DateAwareRollingSplitter(
    dates=df["date"].to_numpy(),
    train_ratio=0.7, val_ratio=0.1, test_ratio=0.2,
    train_val_gap=5,   # e.g. label = future 5-day return
    val_test_gap=5,
)

# NEW in 0.5 — size-mode + anchors. Instead of guessing ratios so the
# OOS slice ends up at 2023Q1..2025Q2, you can pin it directly. Window
# sizes use calendar units (D/W/M/Q/Y); gaps stay in trading days.
date_splitter = DateAwareRollingSplitter(
    dates=df["date"].to_numpy(),     # datetime64, int(YYYYMMDD) or str supported
    train_size="3Y", val_size="6M", test_size="2Q", step="1Q",
    test_start_anchor="2023Q1",
    test_end_anchor="2025Q2",
    train_val_gap=5, val_test_gap=5,  # 5 trading-day embargo
)
# → 9 rolling folds whose test windows tile 2023Q1..2025Q2.

3a. Rolling Prediction Aggregation

When you have one prediction array per rolling fold, stitch them into a single out-of-sample series:

from qfeaturelib import aggregate_rolling_predictions

oos = aggregate_rolling_predictions(
    predictions=fold_predictions,     # list of (test_pred, ...) per fold
    datetime_labels=fold_datetime,    # list of [train_dates, val_dates, test_dates] per fold
    method="last",                    # 'last' | 'mean' on overlapping dates
)

3b. 3-D Sliding-Window Patches (sequence models)

Vectorised (N, n_steps, F) patch builder for transformer / RNN inputs, ~3× faster than a hand-rolled Python loop and cross-asset-safe:

from qfeaturelib import make_sliding_patches

bundle = make_sliding_patches(
    X=features,                  # (T, F)
    y=labels,                    # optional (T,)
    n_steps=20, step_size=1,
    dates=df["date"].to_numpy(),
    assets=df["code"].to_numpy(),  # build patches per ticker; windows
                                   # cannot straddle two tickers
)
# bundle.X: (K, n_steps, F), bundle.y, bundle.dates, bundle.assets, bundle.row_idx

4. Missing Value Imputation

from qfeaturelib.imputation import (
    ffill,          # Forward fill
    ffill_limit,    # Forward fill with limit (prevents stale data filling)
    cs_median_fill, # Cross-sectional median fill
    cs_mean_fill,   # Cross-sectional mean fill
)

# Forward fill with maximum 5 consecutive fills
result = ffill_limit(data, limit=5)

5. Feature Neutralization

Remove effects of control factors via regression residuals:

from qfeaturelib.neutralization import (
    neutralize,
    industry_neutralize,
    size_neutralize,
)

# Industry neutralization
neutralized = industry_neutralize(feature, industry_labels)

# Size (market cap) neutralization
neutralized = size_neutralize(feature, log_market_cap)

# Custom control factors
neutralized = neutralize(feature, control_factors, method="ols")

6. Macro Indicators

Special handling for macro-economic indicators without asset dimension:

from qfeaturelib import (
    macro_rolling_zscore,
    adapt_macro_to_panel,
)

# Direct standardization of 1D macro data
gdp_zscore = macro_rolling_zscore(gdp_growth, window=12, shift=1)

# Broadcast to panel format for combination with asset features
gdp_panel = adapt_macro_to_panel(gdp_growth, n_assets=50)  # (T,) -> (T, N)

7. Factor / Feature Correlation Analysis

Vectorised correlation computation for large factor panels, with an optional PyTorch / GPU backend, plus generic visualisation that treats grouping as opt-in:

import numpy as np
from qfeaturelib.correlation import (
    cross_section_corr,
    lagged_autocorr,
    build_sample_keys,
    plot_correlation_heatmap,
    plot_autocorr_heatmap,
)

# X has shape (N, K) — N stacked (date, asset) rows, K factors
X = np.random.randn(100_000, 30).astype("float32")

# Same-period K x K correlation matrix.
# backend="auto" picks torch only on CUDA; otherwise falls back to numpy.
corr = cross_section_corr(X, method="pearson", backend="auto", device="cuda:0")

# Plain heatmap (no grouping)
plot_correlation_heatmap(corr, title="My factors")

# Grouped heatmap (factor family, industry, ...). The library is
# group-agnostic: pass any sequence of labels and you get coloured
# group bands + legend automatically.
groups = ["momentum"] * 10 + ["value"] * 10 + ["growth"] * 10
plot_correlation_heatmap(
    corr,
    groups=groups,
    sort_by_groups=True,
    title="Factors by family",
    group_legend_title="Family",
)

# Multi-lag cross-asset autocorrelation
time_idx = np.repeat(np.arange(200), 500)        # (date_idx, asset_idx)
entity_idx = np.tile(np.arange(500), 200)
keys, base = build_sample_keys(time_idx, entity_idx)
ac = lagged_autocorr(
    X, sample_keys=keys, base=base,
    lags=(1, 5, 10, 20), method="pearson",
    backend="auto", device="cuda:0",
)
plot_autocorr_heatmap(ac, lags=[1, 5, 10, 20])

Installation extras:

pip install qfeaturelib[gpu]   # adds torch for GPU acceleration
pip install qfeaturelib[viz]   # adds matplotlib for the plotting helpers

Performance Benchmarks

On standard test data (T=5000, N=1000, F=50):

Operation	Pandas	QFeatureLib	Speedup
Rolling Z-Score	~5s	~0.1s	50x
Cross-sectional Z-Score	~2s	~0.02s	100x
Rolling Rank	~10s	~0.5s	20x

Design Principles

1. Safety First: Default shift=1 prevents accidental future function usage
2. Vectorization: All core computations use NumPy vectorized operations
3. Memory Efficiency: Return views instead of copies, support in-place operations
4. Type Safety: Full type annotations, passes mypy strict mode

Related Projects

• AssetPanelForest - Supervised clustering for panel data
• MASFactorMiner - Factor mining and analysis
• GeneralBacktest - Backtesting framework

License

MIT License - see LICENSE file for details.

Contributing

Contributions are welcome! Please see CONTRIBUTING.md for guidelines.

Changelog

See CHANGELOG.md for version history and changes.

Support

• GitHub Issues: https://github.com/ElenYoung/QFeatureLib/issues
• Documentation: https://github.com/ElenYoung/QFeatureLib#readme

---

Note: This library is part of a quantitative finance ecosystem. When implementing features, consider compatibility with downstream projects.