Skip to main content

A unittest runner for detecting lemon tests

Project description

🍋test /ˈlɛmən tɛst/ Noun.
 1. A test which appears to test a feature, but in fact does not test that feature at all.
 [us. I reviewed that pull request and found one 🍋test that doesn't test right.]

What?

A unittest runner for detecting 🍋tests.

Why?

Good development practice dictates that new features should be accompanied with new tests. Good tests can identify regressions and accidental removal of features.

A 🍋test, is a test which appears to test a feature, but in fact does not test that feature at all. 🍋tests can be considered worse than not having a test at all, because they give the false assumption that your new feature is tested. If a regression occurs, 🍋tests won’t let you know. The detection of 🍋tests can identify situations when the feature does not meet your requirements. Detecting 🍋tests helps ensure code quality and feature delivery.

This library identifies 🍋tests. This is done by examining two git branches: the feature, and the merge destination. The algorithm separates business logic from testing code, and then uses Git to revert business logic to it’s previous state (the same as the merge destination). A test suite is then run on the reverted business logic with the new tests. Any tests that are successful are marked as 🍋tests.

Testing for 🍋tests is a form of mutation testing. The mutation operation in this case is reverting the business logic to it’s previous state before the feature was written.

Quickstart

pip install lemontest
manage.py test --testrunner=lemontest.djangorunner.DjangoLemonTestRunner --to-branch=master --from-branch=feature/123

It must be a git repository.

Lemontest needs a git repository to be able to revert business logic.

Travis CI

Travis conveniently provides environment variables we can use.

pip install lemontest
manage.py test --testrunner=lemontest.djangorunner.DjangoLemonTestRunner --to-branch=$TRAVIS_BRANCH --from-branch=$TRAVIS_PULL_REQUEST_SHA

Algorithm

  1. Get source files that have changed (git diff)

  2. Identify test source files (use unittest discovery and intersect with step 1)

  3. Identify business logic files (set difference between step 1 and step2)

  4. Use git to revert business logic code (git checkout)

  5. Run tests that have changed

  6. Raise an exception for each test that succeeds

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

lemontest-0.4.6.tar.gz (7.4 kB view details)

Uploaded Source

Built Distribution

lemontest-0.4.6-py2.py3-none-any.whl (8.2 kB view details)

Uploaded Python 2 Python 3

File details

Details for the file lemontest-0.4.6.tar.gz.

File metadata

  • Download URL: lemontest-0.4.6.tar.gz
  • Upload date:
  • Size: 7.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: Python-urllib/2.7

File hashes

Hashes for lemontest-0.4.6.tar.gz
Algorithm Hash digest
SHA256 09cc9c88b9d0c573b9d02203683d140ff87bfa6504be2b4a00674428f3bffa91
MD5 aa362fd4f64ed3f8398cc73e57f20118
BLAKE2b-256 44e25209996b5948bc109d093e04693d17ab73e6071d24d4c33bd7daca54c356

See more details on using hashes here.

File details

Details for the file lemontest-0.4.6-py2.py3-none-any.whl.

File metadata

File hashes

Hashes for lemontest-0.4.6-py2.py3-none-any.whl
Algorithm Hash digest
SHA256 3c45028f37f9b3d7b46b58e52e182d71652eecf3986d2b3d09c018df3124235b
MD5 e5fda6d66be971dd4bf4400aa51e3463
BLAKE2b-256 21f1e124fe5b62be0e2317fc766cc178ae06eebf4a6d952590977ee04bd30872

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page