Testuary
Testuary writes unit test suites for Python and Java code, and for C code on Linux, with a language model. It runs the generated tests, measures their line and branch coverage and how many deliberately injected faults (mutants) they detect, and asks the model to close the gaps until the goals or the limits are reached. It then removes the tests that add no coverage and catch no extra mutant, and delivers the suite with its measurements.
Language support comes from LangAdapter, a separate package
(langadapter).
Install
Testuary runs on Python 3.10 or newer. It is supported on Windows and Linux. On macOS it is expected to work, but this release has not run on a Mac: Tested environments says what is known. Install Testuary into a virtual environment of its own, outside the directory of the project you test, as getting started explains. pip installs LangAdapter with it:
python3 -m venv testuary-env
source testuary-env/bin/activate
python -m pip install testuary
On Windows, create the environment with python -m venv testuary-env and
activate it with testuary-env\Scripts\activate. On Debian and Ubuntu,
python3 -m venv needs the package python3-venv
(sudo apt install python3-venv), as Set up your machine
describes.
Testing a C file needs Linux, LangAdapter's extra for C
(python -m pip install "langadapter[c]") and the compiler tools that
LangAdapter's C guide
lists.
Testuary comes with a local web interface. Start it in the directory you work in:
testuary-web
It opens the interface in your browser, or in an SSH login prints the
ssh -L line for your own computer.
The web interface describes
its pages and, under Access,
how to start it on a machine you share.
The project you test needs its own test tools: pytest, coverage and Cosmic Ray
for Python, or a JDK and the project's Maven or Gradle build for Java.
Tested environments
lists the tested versions. For Java,
LangAdapter downloads the JUnit launcher, JaCoCo and PIT into its cache on
first use, unless LANGADAPTER_JAVA_TOOLS_OFFLINE is set to 1.
Example
Create the configuration templates in your project's directory:
cd /path/to/project
testuary init
This writes models.ini into your user directory and hints.json into the
project's directory, and names each file it writes. Then add your model to that
models.ini. The section name is the alias you pass with
--model, and provider = names the kind of model service, ollama,
openai or vllm:
[ollama_qwen]
provider = ollama
model = qwen3-coder:30b
api_url = http://localhost:11434/v1
Run the quick check in the project's directory. The check's report says what it looks at and what it leaves alone:
testuary check --model ollama_qwen --python-executable /path/to/project-env/bin/python --source-files package/module.py
Generate a suite for one target. For a first run, a few lines in the project's settings file keep it small, as getting started shows. Run it in the project's directory too, where the run reads that file:
testuary generate --model ollama_qwen --python-executable /path/to/project-env/bin/python --source-files package/module.py
The results go to .testuary/results/ollama_qwen in the project's directory.
At a paid service, model requests can cost money. To see a run of all four phases without
a model, play back the offline demonstration with testuary --demo --output-dir demo-python.
The quickstart directory of the repository, also in the source distribution, holds these first steps as scripts and commands: the demonstrations, the web interface started from the console or from Python, and two small projects, one Python file and one Java class, to run with your own model.
Documentation
- Getting started: install, check your setup, run with small limits and read the results.
- Terminal output: the progress views and the check's report.
- Results and logs: the run directory, the delivered suite, the exit status and how to continue a run.
- Offline demonstrations: recorded runs without a model.
- Web interface:
testuary-web, a local browser interface for projects, models, settings, runs and results. - Configuration: models, settings and hints.
- Running on a cluster: mutation workers, node-local scratch, proxies and resuming batch jobs.
- Models and prompt profiles
- Extension points: every way to extend Testuary, with its guide.
- Make a language ready for Testuary
- Measurement and recovery
- Result files
- Contributing
- Changelog
License
Testuary is licensed under the MIT license. A few files from other projects, in the demonstration data, keep their own licenses, and so do the installed dependencies. Third-party material lists them.
Metadata
Release files for testuary 0.2.0
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| testuary-0.2.0.tar.gz | 2.0 MB | Without provenance | Details for testuary-0.2.0.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| testuary-0.2.0-py3-none-any.whl | Python 3 | none | any | Details for testuary-0.2.0-py3-none-any.whl |
Total release size: 4.0 MB
Release files / testuary-0.2.0.tar.gz
| Download URL | testuary-0.2.0.tar.gz |
|---|---|
| Size | 2.0 MB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
bc9b879969f7ab160281ee729e99d0e7a34e3c0cc7d5b772db7b0b05e805182f
|
|
BLAKE2b-256 checksum How to use checksums |
c3d07fd1b5b3d7673e4fa0d4dd9e96ceb20354957b1a1efd699fd80d9488ac78
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.10
|
Release files / testuary-0.2.0-py3-none-any.whl
| Download URL | testuary-0.2.0-py3-none-any.whl |
|---|---|
| Size | 2.1 MB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b7384b413497003f26b4b5fa44b3e8952d9c2c351f6d0a980e523f6136c40cb9
|
|
BLAKE2b-256 checksum How to use checksums |
2df3c9065f3bd0c6b21a8da4b10f650802c393d2eb14dc19e727ae6f829ae7d7
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/7.0.0 CPython/3.12.10
|
Security for testuary 0.2.0
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.