AutoStatLib - python library for automated statistical analysis

AutoStatLib picks the right statistical test for your data and runs it — automatically, or manually if you prefer to choose. It checks normality, decides between parametric and non-parametric tests, accounts for paired vs. independent groups, and (optionally) runs post-hoc pairwise comparisons. Results come back as a plain Python dictionary, and a companion plotting module turns that same dictionary straight into a publication-style figure.

Table of contents

• Install
• Quick start
• How the API works
  • 1. Creating an analysis
  • 2. Running a test
  • 3. How automatic test selection works
  • 4. Reading the results
  • 5. Warnings
  • 6. Text summary
• Plotting results

Install

pip install AutoStatLib

Requires Python ≥ 3.10.

Quick start

import numpy as np
import AutoStatLib

# two independent, normally-distributed groups
group_A = list(np.random.normal(loc=4.0, scale=1.0, size=30))
group_B = list(np.random.normal(loc=6.5, scale=2.0, size=30))

analysis = AutoStatLib.StatisticalAnalysis([group_A, group_B],
                                            groups_name=['Control', 'Drug'],
                                            paired=False, 
                                            verbose=False,
                                            posthoc=True)

analysis.RunAuto()
result = analysis.GetResult()

print(f"{result['Test_Name']} \np-value: {result['p_value']} \nSignificance: {result['Significance(p<0.05)']}")

fig = AutoStatLib.StatPlots.BarStatPlot(result['Samples'], **result)
fig.plot()
fig.show()
fig.save('result.png')

More runnable examples live in the /demo directory of this repo.

How the API works

The library is built around one class, StatisticalAnalysis. You construct it with your data and configuration, call one of its Run*() methods to actually perform a test, then pull the outcome out with GetResult().

StatisticalAnalysis(data, ...)  →  .RunAuto() / .RunTtest() / ...  →  .GetResult()
                                                                   →  .GetSummary()

1. Creating an analysis

analysis = AutoStatLib.StatisticalAnalysis(
    groups_list,
    paired=False,
    tails=2,
    posthoc=True,
    verbose=True,
    raise_errors=False,
    groups_name=None,
    subgrouping=None,
)

2. Running a test

Call RunAuto() to let AutoStatLib decide, or call one of the manual methods if you already know which test you want:

GetTestIDs() returns the list of valid IDs for RunManual(), e.g.:

analysis.GetTestIDs()
# ['anova_1w_ordinary', 'anova_1w_rm', 'friedman', 'kruskal_wallis',
#  'mann_whitney', 't_test_independent', 't_test_paired',
#  't_test_single_sample', 'wilcoxon', 'wilcoxon_single_sample']

analysis.RunManual('mann_whitney')   # equivalent to analysis.RunMannWhitney()

Calling a manual method on data it isn't suited for (e.g. a parametric test on non-normal data, or a 2-group test on 3 groups) doesn't crash — it either runs anyway and adds a warning to the result (wrong distribution / wrong pairing), or fails validation and returns an error result (wrong number of groups, mismatched paired lengths). See Warnings below.

3. How automatic test selection works

RunAuto() follows this decision logic:

1. Check normality of every group. Each group is run through up to four normality tests — Shapiro-Wilk, Lilliefors, Anderson-Darling, and D'Agostino-Pearson (the latter two are skipped for groups smaller than n=20, where they're unreliable). A group is treated as normally distributed if at least one test says so — this is intentionally lenient, since requiring unanimous agreement makes the parametric branch too hard to reach with realistic sample sizes.
2. Data is "parametric" only if every group passed. One non-normal group is enough to route everything to the non-parametric branch.
3. Branch by group count and paired:

The full normality breakdown (which test passed/failed for each group) is always printed in the verbose log, and the overall True/False verdict is available in the result dict as Data_Normaly_Distributed.

Note on one-tailed tests: when tails=1, AutoStatLib currently halves the two-tailed p-value. This is only strictly correct when the observed effect is in your hypothesized direction — if you need directional certainty, treat one-tailed results with that caveat in mind, or verify the effect direction yourself before reporting.

4. Reading the results

result = analysis.GetResult()

GetResult() returns:

• a populated dict if a test ran successfully,
• {} (empty dict) if the test failed validation (and prints an explanatory message),
• None if you haven't called any Run*() method yet.

Full key reference:

{
    'p_value':                      str,                 # human-formatted, e.g. "p=0.03" or "p<0.0001"
    'p_value_exact':                float,               # raw numeric p-value
    'Significance(p<0.05)':         bool,
    'Stars':                        int,                 # 0-4, number of significance stars
    'Stars_Printed':                str,                 # 'ns', '*', '**', '***', '****'
    'Test_Name':                    str,                 # e.g. "t-test for independent samples"
    'Groups_Compared':              int,
    'Population_Mean':              float | str,         # value of `popmean`, or 'N/A' for multi-group tests
    'Data_Normaly_Distributed':     bool,
    'Parametric_Test_Applied':      bool,
    'Paired_Test_Applied':          bool,
    'Tails':                        int,                 # 1 or 2, echoes the input

    'Groups_Name':                  list[str],
    'Groups_N':                     list[int],
    'Groups_Mean':                  list[float],
    'Groups_Median':                list[float],
    'Groups_SD':                    list[float],         # sample SD, ddof=1
    'Groups_SE':                    list[float],         # standard error of the mean
    'Samples':                      list[list[float]],   # your cleaned input data, by group

    'Posthoc_Tests_Name':           str,                 # '' if posthoc=False
    'Posthoc_Matrix':               list[list[float]],   # pairwise p-values, [] if posthoc=False
    'Posthoc_Matrix_bool':          list[list[bool]],    # pairwise significance
    'Posthoc_Matrix_printed':       list[list[str]],     # human-formatted p-values
    'Posthoc_Matrix_stars':         list[list[str]],     # star notation per pair

    'Warnings':                     list[str],           # see below
    'Successfull_Test':             bool,
    'subgrouping':                  list,                # echoes the constructor input
}

If posthoc=True was set and you ran a 3+ group test, Posthoc_Matrix is an N×N matrix (where N is the number of groups) of pairwise comparison p-values, with 1.0 on the diagonal.

5. Warnings

result['Warnings'] is a list of human-readable strings describing anything questionable about the analysis — it's empty ([]) for a clean run. Warnings are added when:

• A parametric test was manually run on data that isn't normally distributed (or vice versa)
• A paired test was manually run on data declared independent (or vice versa)
• A single-sample test was run without popmean set (defaults to 0)

These don't stop the test from running — they flag results that might be misleading so you can decide whether to trust them or re-run with the correct configuration / RunAuto().

6. Text summary

For a copy-paste-ready writeup of the whole analysis (data table, normality breakdown, test chosen, full results):

analysis.PrintSummary()        # prints directly to stdout
text = analysis.GetSummary()   # same content, as a string

If verbose=True (the default), this same summary is printed automatically every time you call a Run*() method.

Plotting results

The AutoStatLib.StatPlots module turns a finished analysis directly into a matplotlib figure. Each plot class takes your raw data plus (optionally) the exact dictionary returned by GetResult(), unpacked as keyword arguments — so the test name, p-value, and post-hoc matrix flow straight from the analysis into the plot without you re-typing anything.

from AutoStatLib.StatPlots import BarStatPlot

result = analysis.GetResult()

fig = BarStatPlot(result['Samples'], **result)
fig.plot()
fig.show()                       # or fig.save('figure.png')

Available plot types — all share the same constructor signature and workflow:

You don't need a StatisticalAnalysis result to use these — you can also build a plot directly from raw data and your own values:

from AutoStatLib.StatPlots import BoxStatPlot

fig = BoxStatPlot(
    [group_A, group_B, group_C],
    p_value_exact=0.012,
    Test_Name='Kruskal-Wallis test',
    Groups_Name=['Control', 'Drug A', 'Drug B'],
    plot_title='Effect on firing rate',
    y_label='Hz',
)
fig.plot()
fig.save('result.png', dpi=300)

Useful customization parameters (all optional):

Output methods on every plot object: .plot() to render, .show() to display, .save(path, format='png', dpi=150) to export, .close() to free the figure.

TODO

The project is in alpha dev status. Here is much work to do:

-- Anova: posthocs
-- Anova: add 2-way anova and 3-way anova
-- onevay Anova: add repeated measures (for normal dependent values) with and without Gaisser-Greenhouse correction
-- onevay Anova: add Brown-Forsithe and Welch (for normal independent values with unequal SDs between groups)
-- paired T-test: add ratio-paired t-test (ratios of paired values are consistent)
-- add Welch test (for norm data unequal variances)
-- add Kolmogorov-smirnov test (unpaired nonparametric 2 sample, compare cumulative distributions)
-- add independent t-test with Welch correction (do not assume equal SDs in groups)
-- add correlation test, correlation diagram
-- add linear regression, regression diagram
-- add QQ plot
-- n-sample tests: add onetail option

✅ done -- detailed normality test results
✅ done -- added posthoc: Kruskal-Wallis Dunn's multiple comparisons

tests check:
1-sample:
✅ok --Wilcoxon 2,1 tails
✅ok --t-tests 2,1 tails

2-sample:
✅ok --Wilcoxon 2,1 tails
✅ok --Mann-whitney 2,1 tails
✅ok --t-tests 2,1 tails

n-sample:
✅ok --Kruskal-Wallis 2 tail
✅ok --Dunn's multiple comparisons
✅ok --Friedman 2 tail
✅ok --one-way ANOVA 2-tailed
✅ok --Tukey`s multiple comparisons