pysmartcor

pysmartcor detects variable types and selects a suitable correlation method for each pair. It supports continuous, count, binary, ordinal, and categorical variables and returns the estimate, inference, selected method, and rationale.

Install

The package is on PyPI:

pip install pysmartcor

Install the optional plotting dependencies when needed:

pip install 'pysmartcor[viz]'

Quick start

import numpy as np
from pysmartcor import smart_cor

rng = np.random.default_rng(42)
education_years = rng.integers(8, 22, size=300)
income = 1500 * education_years + rng.normal(0, 8000, size=300)

result = smart_cor(income, education_years, x_name="income", y_name="education_years", verbose=False)
print(result)

Smart Correlation
  Estimate: 0.5478
  Method:   Pearson Correlation
  Variables: income (continuous) x education_years (count)
  N: 300
  p-value: < 0.001  (exact null distribution of r, equivalent to the t-test on r (scipy.stats.pearsonr))
  H0: rho = 0
  95% CI: [0.4633, 0.6225]  (Fisher z (scipy.stats.pearsonr))
  One variable is continuous and the other is a count (treated as continuous); Pearson correlation selected.
  Alternatives: spearman, kendall

smart_cor() detected the variable types, selected Pearson correlation, and reported the estimate with its confidence interval, p-value, and the reasoning — no method choice needed from you.

Load the example data

The package includes gss_2024_casestudy.csv. This CSV is frozen for reproducibility. The examples below read the bundled copy and do not download data.

from importlib.resources import as_file, files
import pandas as pd

csv_resource = files("pysmartcor").joinpath("data/gss_2024_casestudy.csv")
with as_file(csv_resource) as csv_path:
    gss = pd.read_csv(csv_path)

Correlate one pair

from pysmartcor import smart_cor

result = smart_cor(
    gss["coninc"],
    gss["age"],
    x_name="coninc",
    y_name="age",
    verbose=False,
)

print(result)
result.estimate
result.method
result.p_value
result.ci_lower, result.ci_upper

Set assume_latent_normal to control pairs that can use a latent-variable method:

smart_cor(gss["degree"], gss["happy"], assume_latent_normal="auto")
smart_cor(gss["degree"], gss["happy"], assume_latent_normal=True)
smart_cor(gss["degree"], gss["happy"], assume_latent_normal=False)

The default, "auto", tests the assumption for each affected pair. A binary-by-binary table is saturated, so automatic selection uses phi. Set the argument to True to request tetrachoric correlation.

Build a matrix

from pysmartcor import smart_cormat

columns = ["age", "coninc", "degree", "happy", "sex", "region"]
matrix = smart_cormat(
    gss[columns],
    assume_latent_normal=False,
    verbose=False,
)

matrix.correlations
matrix.methods
matrix.types
matrix.to_long()

Use smart_cor_df() when a script needs plain pandas DataFrames:

from pysmartcor import smart_cor_df

plain = smart_cor_df(gss[columns], assume_latent_normal=False)
plain["correlations"]
plain["methods"]

Compare methods

from pysmartcor import compare_methods

comparison = compare_methods(
    gss["degree"],
    gss["happy"],
    assume_latent_normal=False,
    bootstrap=False,
    verbose=False,
)
comparison.results

Plot a matrix

Plotting requires the optional viz dependencies.

from pysmartcor.viz import cor_heatmap, method_heatmap

cor_heatmap(matrix)
method_heatmap(matrix)

Methods

The package implements Pearson, Spearman, Kendall's tau, point-biserial, rank-biserial, phi, tetrachoric, Yule's Q, polychoric, polyserial, Cramer's V, Theil's U, Tschuprow's T, and Goodman-Kruskal's gamma.

Required dependencies are NumPy, SciPy, and pandas. Matplotlib and seaborn are optional plotting dependencies.

Paper

The accompanying paper, smartcor: Intelligent Correlation Method Selection for Mixed Variable Types by M. Harshvardhan and Pritam Ranjan (2026), is available as an arXiv preprint: arXiv:2607.22285 (doi:10.48550/arXiv.2607.22285). The vignettes of the R package and the articles on the package website cover the same material.

Authors

M. Harshvardhan (maintainer) and Pritam Ranjan.

License

GPL (>= 3)