collection of utility functions for correlation analysis
Project description
korr
collection of utility functions for correlation analysis
Table of Contents
Installation
The korr
git repo is available as PyPi package
pip install korr
Usage
Check the examples folder for notebooks.
Compute correlation matrix and its p-values
- pearson -- Pearson/Sample correlation (interval- and ratio-scale data)
- kendall -- Kendall's tau rank correlation (ordinal data)
- spearman -- Spearman rho rank correlation (ordinal data)
- mcc -- Matthews correlation coefficient between binary variables
EDA, Dig deeper into results
- flatten -- A table (pandas) with one row for each correlation pairs with the variable indicies, corr., p-value. For example, try to find "good" cutoffs with
corr_vs_pval
and then look up the variable indicies withflatten
afterwards. - slice_yx -- slice a correlation and p-value matrix of a (y,X) dataset into a (y,x_i) vector and (x_j, x_k) matrices
- corr_vs_pval -- Histogram to find p-value cutoffs (alpha) for a) highly correlated pairs, b) unrelated pairs, c) the mixed results.
- bracket_pval -- Histogram with more fine-grained p-value brackets.
- corrgram -- Correlogram, heatmap of correlations with p-values in brackets
Utility functions
- confusion -- Confusion matrix. Required for Matthews correlation (mcc) and is a bitter faster than sklearn's
Parameter Stability
- bootcorr -- Estimate multiple correlation matrices based on bootstrapped samples. From there you can assess how stable correlation estimates are (how sensitive against in-sample variation). For example, stable estimates are good candidates for modeling, and unstable correlation pairs are good candidates for P-hacking and non-reproducibility.
Variable Selection, Search Functions
- mincorr -- From all estimated correlation pairs, pick a given
n=3,5,..
of variables with low and insignificant correlations among each other. (See binsel package for an application.) find_best
-- Find the N "best", i.e. high and most significant, correlationsfind_worst
-- Find the N "worst", i.e. insignificant/random and low, correlations- find_unrelated -- Return variable indicies of unrelated pairs (in terms of insignificant p-value)
Commands
- Check syntax:
flake8 --ignore=F401
- Run Unit Tests:
python -W ignore -m unittest discover
- Remove
.pyc
files:find . -type f -name "*.pyc" | xargs rm
- Remove
__pycache__
folders:find . -type d -name "__pycache__" | xargs rm -rf
- Upload to PyPi with twine:
python setup.py sdist && twine upload -r pypi dist/*
Debugging
- Notebooks to profile python code are in the profile folder
Support
Please open an issue for support.
Contributing
Please contribute using Github Flow. Create a branch, add commits, and open a pull request.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
korr-0.8.2.tar.gz
(1.5 MB
view hashes)