dsa-metrics 0.1.0

Fast, vectorized regression error and directional/trading metrics for forecasts and time series.

DSA Metrics

dsa_metrics is a small pandas-first package for fast regression error analysis.

It supports:

• one dataframe or one CSV that already contains timestamps, predictions, and true values
• separate prediction and truth dataframes or CSVs with different column names
• global metrics or metrics grouped per timestamp

Supported metrics:

Error metrics:

• mae
• mse
• rmse
• mape
• wape
• smape
• accuracy

Directional / trading-style metrics (following the DSA model_performance report logic):

• hit_rate — directional accuracy: fraction of steps where the predicted move matches the realised move (alias: directional_accuracy, da)
• mock_pnl — total mock PnL: take the predicted direction each step, earn the realised return (alias: pnl)
• pnl_mean — mean per-step mock PnL (alias: mean_pnl)
• pnl_std — sample standard deviation (ddof=1) of the per-step mock PnL (alias: std_pnl)
• pnl_sharpe — Sharpe ratio of the per-step mock PnL: pnl_mean / pnl_std (alias: sharpe, sharpe_ratio)
• pnl_t_stat — one-sample t-statistic of the per-step mock PnL against the null hypothesis mean = 0 (alias: t_stat, tstat)

Notes:

• mape, wape, and smape are returned as fractions, not percentages
• accuracy is defined as 1 - smape
• hit_rate is returned as a fraction in [0, 1] (multiply by 100 for a percentage)
• The directional / PnL metrics treat pred_column and true_column as level series and difference consecutive rows internally: target_return = target - lag(target), predicted_return = pred - lag(pred), direction = sign(predicted_return), ts_pnl = direction * target_return. Because they depend on row order, pass the data time-ordered and use mode="global" over a single series (not mode="per_timestamp").

Install

pip install -e .

Quick Start

import pandas as pd

from dsa_metrics import calculate_metrics

df = pd.DataFrame(
    {
        "ds": ["2025-01-31", "2025-02-28", "2025-03-31"],
        "y_pred": [101.0, 98.0, 110.0],
        "y": [100.0, 100.0, 105.0],
    }
)

global_metrics = calculate_metrics(
    df,
    timestamp_column="ds",
    pred_column="y_pred",
    true_column="y",
    metrics=["mae", "rmse", "smape", "accuracy"],
    mode="global",
)

per_timestamp_metrics = calculate_metrics(
    df,
    timestamp_column="ds",
    pred_column="y_pred",
    true_column="y",
    metrics=["mae", "smape"],
    mode="per_timestamp",
)

Separate Prediction And Truth Sources

import pandas as pd

from dsa_metrics import calculate_metrics_from_sources

predictions = pd.DataFrame(
    {
        "forecast_date": ["2025-01-31", "2025-02-28", "2025-03-31"],
        "series_id": ["pe", "pe", "pe"],
        "forecast": [101.0, 98.0, 110.0],
    }
)

truth = pd.DataFrame(
    {
        "actual_date": ["2025-01-31", "2025-02-28", "2025-03-31"],
        "product_id": ["pe", "pe", "pe"],
        "actual": [100.0, 100.0, 105.0],
    }
)

result = calculate_metrics_from_sources(
    predictions,
    truth,
    pred_timestamp_column="forecast_date",
    true_timestamp_column="actual_date",
    pred_column="forecast",
    true_column="actual",
    pred_key_columns="series_id",
    true_key_columns="product_id",
    metrics=["mae", "wape", "accuracy"],
    mode="global",
)

If your sources are CSV files, you can pass file paths instead of dataframes.

Public API

from dsa_metrics import available_metrics, calculate_metrics, calculate_metrics_from_sources

calculate_metrics(...)

• use when predictions and truth already live in the same dataframe or CSV

calculate_metrics_from_sources(...)

• use when predictions and truth come from separate dataframes or CSVs
• align them by timestamp and optional key columns before computing metrics

available_metrics()

• returns the metric names supported by the package