permutation_test 0.18

Implementation of Fishers permutation test

What is it

Implementation of Fisher’s permutation test.

The test is described in following publications:

• Fisher, R. A. (1935). The design of experiments. 1935. Oliver and Boyd, Edinburgh.

• Ernst, M. D. (2004). Permutation methods: a basis for exact inference. Statistical Science, 19(4), 676-685

How to install it

Install with pip:

$ pip install permutation_test

Command Line Script Usage

Example:

permtest [path/to/data.csv] [groups_colname] [reference_group_name] -t [test_group_name]

Use help to get info about parameters:

$ permtest -h

usage: permtest [-h] [-t TESTGROUP]
            input_filepath treatment_column_name referencegroup

positional arguments:
  input_filepath        e.g. path/to/my/data.csv, path to csv file with data
  treatment_column_name
                        name of column in the csv table that specifies the
                        group
  referencegroup        name of the reference group as named in the csv table

optional arguments:
  -h, --help            show this help message and exit
  -t TESTGROUP, --testgroup TESTGROUP
                        name of the test group as named in th csv table. If
                        not defined, test group is determined automatically.
   -a ALPHA, --alpha ALPHA
                         significance level alpha (between 0 and 1) If not
                         defined, alpha is set to 0.05.
   -m MULTI_COMP_CORR, --multi_comp_corr MULTI_COMP_CORR
                         perform multiple comparison correction with benjamini
                         hochberg procedure yes/no, If not defined, correction
                         is performed.

Specifications of data structure in csv file

• The csv should contain comma separated values. One ore more columns should contain measurement data.

• All columns need to have a name, specified in the first row.

• One column contains names for the groups

Example my_data.csv:

experiment_1	experiment_2	experiment_3	group_names
1.4	3	2.5	condition_2
2	5	2	condition_1
5.6	3	17	condition_2
9	6.5	2	condition_1
17	5	13.0	condition_1
17	2	13.0	condition_3
12	8	18.7	condition_3

To perform tests for all experiments, where condition_1 is the reference and condition_2 is the test data, run follwoing command:

$ permtest my_data.csv group_names condition_1 -t condition_2

Often, it is convenient to save the output in a textfile:

$ permtest my_data.csv group_names condition_1 -t condition_2 > my_test_result.txt

Python Library Use Example

>>> import permutation_test as p
>>> data = [1,2,2,3,3,3,4,4,5]
>>> ref_data = [3,4,4,5,5,5,6,6,7]

>>> p_value = p.permutation_test(data, ref_data)
taking random subsample of size 20000 from 48620 possible permutations
nr of mean diffs: 20000


 Distribution of mean differences
                                       │
                                    *  ┼+1.73038
                                       │  *
                                       │
                                 *     │      *
                                       │
                              *        │         *
                                       │
                                       │
                          *            │             *
                                       │
                                       │
                       *               │                *
                                       │
                    *                  │                   *
                                       │
                *                      │                       *
         *   *                         ┼+0.037                    *   *
───┼*****─***─**─***─**─**─***─**─**─**┼**─***─**─***─**─**─***─**─***─*****┼───
    -2.38713                           │                            +2.39919
mean difference of tested dataset: -2.0
p_value: 0.00345
p_lower_than (probability that mean of test data is not lower than mean of ref data): 0.00345
p_value_greater_than (probability that mean of test data is not greater than mean of ref data): 0.9998
0.0034500000000000121

The asccii art plot shows the ditribution of mean differences for the permutations. The ascii art plot is done with [AP](https://github.com/mfouesneau/asciiplot), a plotting package by Morgan Fouesneau.

If the number of possible combinations is grater than n_combinations_max, a random subsample of size n_combinations_max is taken for histogram calculation.

If detailed is False, only (two-sided) p_value is returned, i.e. the probability that data is not different from ref_data

If detailed is True, one-sided p values and histogram data of mean differences is returned in a dict:

hist_data: distribution of mean differences for all permutations p_value: two sided p_value (the probability that data is not different from ref_data ) p_value_lower_than: the probability that mean of data is not lower than mean of ref_data p_value_greater_than: the probability that mean of data is not grater than mean of ref_data

Christoph Möhl, Image and Data Analysis Facililty/Core Faciliies, Deutsches Zentrum für Neurodegenerative Erkrankungen e. V. (DZNE) in der Helmholtz-Gemeinschaft German Center for Neurodegenerative Diseases (DZNE) within the Helmholtz Association