django-studies 0.1.2

Installation

To use this library, install it using pip

pip install django-studies

register the Django app in your settings.py:

# project/settings.py
INSTALLED_APPS = [
    # ...
    "studies",
]

and run the migrations:

python manage.py studies

Features

• To run an experiment, instantiate the Experiment class, define the control and the candidate and conduct the experiment. For example, a simple Django class-based view with an experiment would look like (the one from the demo project):

from studies.experiments import Experiment


class ViewWithMatchingResults(View):
    def get(self, request, *args, **kwargs):
        with Experiment(
            name="ViewWithMatchingResults",
            context={"context_key": "context_value"},
            percent_enabled=100,
        ) as experiment:
            arg = "match"
            kwargs = {"extra": "value"}
            experiment.control(
                self._get_control,
                context={"strategy": "control"},
                args=[arg],
                kwargs=kwargs,
            )
            experiment.candidate(
                self._get_candidate,
                context={"strategy": "candidate"},
                args=[arg],
                kwargs=kwargs,
            )
            data = experiment.conduct()

        return JsonResponse(data)

    def _get_control(self, result, **kwargs):
        return {"result": result, **kwargs}

    def _get_candidate(self, result, **kwargs):
        return {"result": result, **kwargs}

• Adjust the percentage of users who’ll be impacted by this experiment via the admin:

• To add support for your own reporting system, whether it’s logging, statsd or something else, override the Experiment class’ publish method and make the call (another example from the demo project):

import logging
from studies.experiments import Experiment


logger = logging.getLogger()


class ExperimentWithLogging(Experiment):
"""
An override that provides logging support for demonstration
purposes.
"""

def publish(self, result):
    if result.match:
        logging.info(
            "Experiment %(name)s is a match",
            {"name": result.experiment.name},
        )
    else:
        control_observation = result.control
        candidate_observation = result.candidates[0]
        logging.info(
            json.dumps(
                control_observation.__dict__,
                cls=ExceptionalJSONEncoder,  # defined in `demo.overrides`
            )
        )
        logging.info(
            json.dumps(
                candidate_observation.__dict__,
                cls=ExceptionalJSONEncoder,
            )
        )
        logging.error(
            "Experiment %(name)s is not a match",
            {"name": result.experiment.name},
        )

• Override any method from laboratory’s Experiment class, including how you make the comparison:

from studies.experiments import Experiment


class MyExperiment(Experiment):
    def compare(self, control, candidate):
        return control.value['id'] == candidate.value['id']

Caveats

As always there are certain caveats that you should keep in mind. As stated in laboratory’s Caveats, if the control or the candidate has a side-effect like a write operation to the database or the cache, you could end up with erroneous data or similar bugs.

At the moment, this library doesn’t provide a safe write mechanism to mitigate this situation, but it may in the future.

Contributing

To contribute to this project, take a look at CONTRIBUTING.rst.