tlf-correlation-engine

Country-agnostic correlation analysis — part of TLF ("The Living Facts").

Computes Pearson, Spearman, or Kendall correlations across any pandas DataFrame's numeric columns, along with p-values and observation counts, and produces a sorted, filterable long-form report of the strongest and most statistically significant relationships in a dataset.

Unlike tlf-census-stats, this package has no dependency on a specific country schema — it works on any DataFrame with 2+ numeric columns.

Install

pip install tlf-correlation-engine

Or from source, inside the TLF-Data-Analysis monorepo:

cd tlf-correlation-engine
pip install -e ".[dev]"

Usage

import pandas as pd
from tlf_correlation_engine import CorrelationEngine

df = pd.read_csv("census_data.csv")

engine = CorrelationEngine(df, method="pearson")  # or "spearman" / "kendall"

# Coefficient matrix (like df.corr(), but validated numeric-only)
engine.matrix()

# P-value matrix, aligned with matrix()
engine.pvalue_matrix()

# One specific pair, with n and p-value
engine.pairwise("literacy_rate", "urban_population")

# Long-form report of all pairs, strongest relationship first
engine.report()

# Only strong (|r| >= 0.5) and statistically significant (p < 0.05) pairs
engine.report(threshold=0.5, significant_only=True)

Methods

CLI

tlf-correlation-engine --data census.csv --method spearman --output report --export csv --export-path out.csv

Run with no flags at all for a fully interactive walkthrough (file path → sheet selection → method → output type → report filters → export format):

tlf-correlation-engine

For unattended/scripted runs, --yes disables all prompting and fails loudly (rather than silently guessing) if something required — like --data — is missing:

tlf-correlation-engine --data census.csv --yes --method pearson --output matrix

CLI flags

Errors

• InvalidMethodError — unsupported method value
• InsufficientDataError — fewer than 2 numeric columns in the DataFrame

License

MIT — see LICENSE.