Skip to main content

Extendable mutation testing framework

Project description

Antigen

Extendable mutation testing framework

What is mutation testing?

Mutation testing provides what coverage tries to, it finds logic that is not covered by your test suite.

It finds such places by applying mutations to your code and running the modified code against your test suite. If the tests succeed with the mutated code, it means the changed expression is likely not covered ny the tests.

In comparison to coverage:

:green_heart: Checks expressions, not lines.

:green_heart: Checks whether the expression is covered, not whether it was executed.

:x: Can find irrelevant mutants (e.g. mutations in logging or performance optimizations or a mutation that does not break the code)

:x: Executes the tests many times and therefore takes much more time.

There are mitigations for these downsides:

:star: We can mutate only lines that have changed in a given PR

:star: We can show the failing mutants via comments/warnings, as opposed to failing the whole CI pipeline.

A much more in-depth explanation about the concept can be found in This blog post by Goran Petrovic

Why use Antigen?

Extendability

In my personal experience, trying to integrate mutation testing into your CI pipeline can be a bit challenging. There are a lot of features you might want to customize to mitigate some of the downsides of mutation testing, or to be able to integrate it to your project and dev environment effectively.

For example, ignoring mutations on logging logic (which depends on your logging framework and conventions), or showing the results on various platforms (e.g. github, bitbucket, gitlab).

Antigen puts extendability as a top priority so that adding mutation testing to your project is feasible.

How?

Mechine friendly

The core of antigen is a pure python package that can be used by scripts.

The actual CLI uses the core package instead of the logic being coupled to it.

Pluginable

Antigen is written as a pipeline, each stage has an interface (e.g. Mutator, MutantFilter, Runner).

Extending the logic is as simple as creating an object or function that matches that (simple) interface.

The Antigen CLI utilises entry points so that antigen plugins can be added just by installing them with pip.

Usage

Antigen is currently in development, the API might change between versions.

antigen = Antigen(
    filters=[PatchFilter.from_git_diff("develop")],
    config=Config(project_root=Path("/home/myuser/myproject/")),
)

for path in antigen.find_files("."):
    for mutation in antigen.gen_mutations(path):
        result = antigen.test_mutation(
            mutation,
            run_command="pytest",
        )
        print(result)

Roadmap

  • Add CLI
  • Use parso instead of the built-in ast for cross-version mutations.
  • Add wrapper class for remote components (i.e. RemoteFilter(hostname), RemoteRunner(hostname)).
  • Add Output component (with JunitXMLOutput, GithubOutput, BitbucketOutput builtins)
  • Add Cache component (with FileCache, MongoCache builtins)
  • Add Sorter component (for selecting the most likely to succeed mutations)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

antigen-0.2.0.tar.gz (9.4 kB view details)

Uploaded Source

Built Distribution

antigen-0.2.0-py3-none-any.whl (11.7 kB view details)

Uploaded Python 3

File details

Details for the file antigen-0.2.0.tar.gz.

File metadata

  • Download URL: antigen-0.2.0.tar.gz
  • Upload date:
  • Size: 9.4 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.12 CPython/3.8.10 Linux/5.8.0-59-generic

File hashes

Hashes for antigen-0.2.0.tar.gz
Algorithm Hash digest
SHA256 86aea57f76bcbd61def976aba03675c78a4bf97b85d9410aff07c489873b962d
MD5 4a1b249a15b6e1e55c9abacbc004037a
BLAKE2b-256 a0de72ab9be2d29da685e6572f2570575fe6b641f3cd54d7c5c592c0410b9c5f

See more details on using hashes here.

File details

Details for the file antigen-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: antigen-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 11.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.1.12 CPython/3.8.10 Linux/5.8.0-59-generic

File hashes

Hashes for antigen-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 eca9616ce3595cb94ddbc7103e1a23c79e5d53df66ef4c966de69adbbe7af3fc
MD5 904caeb0a946f198351b8b01d09cb0a4
BLAKE2b-256 dafcb5856c720d1e24f20f21cee3269529b14771a0402c431ecdfb917a623b81

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page