This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (pypi.python.org).
Help us improve Python packaging - Donate today!

Two-sample test based on model selection and with better performance than the t-test on small sample sizes.

Project Description

mtest

mtest is a Python implementation of the m-test, a two-sample test based on model selection and described in [1] and [2].

Despite their importance in supporting experimental conclusions, standard statistical tests are often inadequate for research areas, like the life sciences, where the typical sample size is small and the test assumptions difficult to verify. In such conditions, standard tests tend to be overly conservative, and fail thus to detect significant effects in the data.

The m-test is a classical statistical test in the sense of defining significance with the conventional bound on Type I errors. On the other hand, it is based on Bayesian model selection, and thus takes into account uncertainty about the model’s parameters, mitigating the problem of small samples size.

The m-test has been found to generally have a higher power (smaller fraction of Type II errors) than a t-test error for small sample sizes (3 to 100 samples).

[1] Berkes, P., Fiser, J. (2011) A frequentist two-sample test based on Bayesian model selection. arXiv:1104.2826v1

[2] Berkes, P., Orban, G., Lengyel, M., and Fiser, J. (2011). Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science, 331:6013, 83-87.

mtest tables

mtest ships caches tables of statistics to compute the p-value and power of new data in the most efficient way. The library is distributed with tables for p-values (type I error) for N=3,4,…,20 and for N=30,40,…,100. These tables cover the most common cases. New tables are computed when needed, although completion might take a few hours. Type II error tables are not included to keep the package size small.

See scriptscompute_basic_tables.py for an example script to pre-compute tables you might need. The script makes use of the joblib library to distribute the computations on multiple cores.

Dependencies

mtest requires SciPy and PyMC.

License

mtest is released under the GPL v3. See LICENSE.txt .

Copyright (c) 2011, Pietro Berkes. All rights reserved.

Release History

Release History

This version
History Node

1.0

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting