ml4qc 0.1.3

Toolkit for ML-based survey quality control

The ml4qc Python package offers a toolkit for employing machine learning technologies in survey data quality control. Among other things, it helps to extend the surveydata package and advance SurveyCTO’s machine learning roadmap.

Installation

Installing the latest version with pip:

pip install ml4qc

Overview

The ml4qc package builds on the scikit-learn toolset. It includes the following utility classes for working with survey data:

• SurveyML provides core functionality, including preprocessing and outlier detection

• SurveyMLClassifier builds on SurveyML, adding support for running classification models and reporting out results

Examples

This package is best illustrated by way of example. The following example analyses are available:

• CATI1 analysis

• CAPI1 analysis

Documentation

See the full reference documentation here:

https://ml4qc.readthedocs.io/

Project support

Dobility has generously provided financial and other support for v1 of the ml4qc package, including support for early testing and piloting.

Development

To develop locally:

1. git clone https://github.com/orangechairlabs/ml4qc.git

2. cd ml4qc

3. python -m venv venv

4. source venv/bin/activate

5. pip install -r requirements.txt

For convenience, the repo includes .idea project files for PyCharm.

To rebuild the documentation:

1. Update version number in /docs/source/conf.py

2. Update layout or options as needed in /docs/source/index.rst

3. In a terminal window, from the project directory:

   1. cd docs

   2. SPHINX_APIDOC_OPTIONS=members,show-inheritance sphinx-apidoc -o source ../src/ml4qc --separate --force

   3. make clean html

To rebuild the distribution packages:

1. For the PyPI package:

   1. Update version number (and any build options) in /setup.py

   2. Confirm credentials and settings in ~/.pypirc

   3. Run /setup.py for bdist_wheel build type (Tools… Run setup.py task… in PyCharm)

   4. Delete old builds from /dist

   5. In a terminal window:

      1. twine upload dist/* --verbose

2. For GitHub:

   1. Commit everything to GitHub and merge to main branch

   2. Add new release, linking to new tag like v#.#.# in main branch

3. For readthedocs.io:

   1. Go to https://readthedocs.org/projects/ml4qc/, log in, and click to rebuild from GitHub (only if it doesn’t automatically trigger)