Skip to main content

Phi_K correlation analyzer library

Project description

Phi_K is a practical correlation constant that works consistently between categorical, ordinal and interval variables. It is based on several refinements to Pearson’s hypothesis test of independence of two variables. Essentially, the contingency test statistic of two variables is interpreted as coming from a rotated bi-variate normal distribution, where the tilt is interpreted as Phi_K.

The combined features of Phi_K form an advantage over existing coefficients. First, it works consistently between categorical, ordinal and interval variables. Second, it captures non-linear dependency. Third, it reverts to the Pearson correlation coefficient in case of a bi-variate normal input distribution. These are useful features when studying the correlation matrix of variables with mixed types.

For details on the methodology behind the calculations, please see our publication. Emphasis is paid to the proper evaluation of statistical significance of correlations and to the interpretation of variable relationships in a contingency table, in particular in case of low statistics samples. The presented algorithms are easy to use and available through this public Python library.

Example notebooks

Static link

Google Colab link

basic tutorial

basic on colab

advanced tutorial (detailed configuration)

advanced on colab

spark tutorial

no spark available


The entire Phi_K documentation including tutorials can be found at read-the-docs. See the tutorials for detailed examples on how to run the code with pandas. We also have one example on how calculate the Phi_K correlation matrix for a spark dataframe.

Check it out

The Phi_K library requires Python >= 3.7 and is pip friendly. To get started, simply do:

$ pip install phik

or check out the code from out GitHub repository:

$ git clone
$ pip install -e PhiK/

where in this example the code is installed in edit mode (option -e).

You can now use the package in Python with:

import phik

Congratulations, you are now ready to use the PhiK correlation analyzer library!

Quick run

As a quick example, you can do:

import pandas as pd
import phik
from phik import resources, report

# open fake car insurance data
df = pd.read_csv( resources.fixture('fake_insurance_data.csv.gz') )

# Pearson's correlation matrix between numeric variables (pandas functionality)

# get the phi_k correlation matrix between all variables

# get global correlations based on phi_k correlation matrix

# get the significance matrix (expressed as one-sided Z)
# of the hypothesis test of each variable-pair dependency

# contingency table of two columns
cols = ['mileage','car_size']

# normalized residuals of contingency test applied to cols

# show the normalized residuals of each variable-pair

# generate a phik correlation report and save as test.pdf
report.correlation_report(df, pdf_file_name='test.pdf')

For all available examples, please see the tutorials at read-the-docs.

Contact and support

Please note that support is (only) provided on a best-effort basis.

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

phik-0.12.2.tar.gz (602.7 kB view hashes)

Uploaded source

Built Distributions

phik-0.12.2-cp310-cp310-win_amd64.whl (703.8 kB view hashes)

Uploaded cp310

phik-0.12.2-cp310-cp310-macosx_11_0_arm64.whl (975.2 kB view hashes)

Uploaded cp310

phik-0.12.2-cp310-cp310-macosx_10_13_x86_64.whl (812.3 kB view hashes)

Uploaded cp310

phik-0.12.2-cp39-cp39-win_amd64.whl (685.8 kB view hashes)

Uploaded cp39

phik-0.12.2-cp39-cp39-macosx_11_0_arm64.whl (730.7 kB view hashes)

Uploaded cp39

phik-0.12.2-cp39-cp39-macosx_10_13_x86_64.whl (694.7 kB view hashes)

Uploaded cp39

phik-0.12.2-cp38-cp38-win_amd64.whl (677.3 kB view hashes)

Uploaded cp38

phik-0.12.2-cp38-cp38-macosx_11_0_arm64.whl (674.0 kB view hashes)

Uploaded cp38

phik-0.12.2-cp38-cp38-macosx_10_13_x86_64.whl (667.5 kB view hashes)

Uploaded cp38

phik-0.12.2-cp37-cp37m-win_amd64.whl (673.5 kB view hashes)

Uploaded cp37

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page