Extendable mutation testing framework
Project description
Antigen
Extendable mutation testing framework
What is mutation testing?
Mutation testing provides what coverage tries to, it finds logic that is not covered by your test suite.
It finds such places by applying mutations to your code and running the modified code against your test suite. If the tests succeed with the mutated code, it means the changed expression is likely not covered ny the tests.
In comparison to coverage:
:green_heart: Checks expressions, not lines.
:green_heart: Checks whether the expression is covered, not whether it was executed.
:x: Can find irrelevant mutants (e.g. mutations in logging or performance optimizations or a mutation that does not break the code)
:x: Executes the tests many times and therefore takes much more time.
There are mitigations for these downsides:
:star: We can mutate only lines that have changed in a given PR
:star: We can show the failing mutants via comments/warnings, as opposed to failing the whole CI pipeline.
A much more in-depth explanation about the concept can be found in This blog post by Goran Petrovic
Why use Antigen?
Extendability
In my personal experience, trying to integrate mutation testing into your CI pipeline can be a bit challenging. There are a lot of features you might want to customize to mitigate some of the downsides of mutation testing, or to be able to integrate it to your project and dev environment effectively.
For example, ignoring mutations on logging logic (which depends on your logging framework and conventions), or showing the results on various platforms (e.g. github, bitbucket, gitlab).
Antigen puts extendability as a top priority so that adding mutation testing to your project is feasible.
How?
Mechine friendly
The core of antigen is a pure python package that can be used by scripts.
The actual CLI uses the core package instead of the logic being coupled to it.
Pluginable
Antigen is written as a pipeline, each stage has an interface (e.g. Mutator, MutantFilter, Runner).
Extending the logic is as simple as creating an object or function that matches that (simple) interface.
The Antigen CLI utilises entry points so that antigen plugins can be added just by installing them with pip.
Usage
Antigen is currently in development, the API might change between versions.
antigen = Antigen(
filters=[PatchFilter.from_git_diff("develop")],
config=Config(project_root=Path("/home/myuser/myproject/")),
)
for path in antigen.find_files("."):
for mutation in antigen.gen_mutations(path):
result = antigen.test_mutation(
mutation,
run_command="pytest",
)
print(result)
Roadmap
- Add CLI
- Use
parso
instead of the built-inast
for cross-version mutations. - Add wrapper class for remote components (i.e.
RemoteFilter(hostname)
,RemoteRunner(hostname)
). - Add Output component (with
JunitXMLOutput
,GithubOutput
,BitbucketOutput
builtins) - Add Cache component (with
FileCache
,MongoCache
builtins) - Add Sorter component (for selecting the most likely to succeed mutations)
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file antigen-0.2.0.tar.gz
.
File metadata
- Download URL: antigen-0.2.0.tar.gz
- Upload date:
- Size: 9.4 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.1.12 CPython/3.8.10 Linux/5.8.0-59-generic
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 86aea57f76bcbd61def976aba03675c78a4bf97b85d9410aff07c489873b962d |
|
MD5 | 4a1b249a15b6e1e55c9abacbc004037a |
|
BLAKE2b-256 | a0de72ab9be2d29da685e6572f2570575fe6b641f3cd54d7c5c592c0410b9c5f |
File details
Details for the file antigen-0.2.0-py3-none-any.whl
.
File metadata
- Download URL: antigen-0.2.0-py3-none-any.whl
- Upload date:
- Size: 11.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: poetry/1.1.12 CPython/3.8.10 Linux/5.8.0-59-generic
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | eca9616ce3595cb94ddbc7103e1a23c79e5d53df66ef4c966de69adbbe7af3fc |
|
MD5 | 904caeb0a946f198351b8b01d09cb0a4 |
|
BLAKE2b-256 | dafcb5856c720d1e24f20f21cee3269529b14771a0402c431ecdfb917a623b81 |