Skip to main content

Testuary

PyPI version Python versions License: MIT Documentation

Testuary writes unit tests for Python, Java and C code with a language model. It runs the tests, measures which lines and branches they cover and how many mutants (small deliberate bugs) they catch, and asks the model for more tests wherever something is still missed, until your goals or limits are reached. It then keeps only the tests that add coverage or catch an extra mutant, and hands you the finished test suite with its measurements.

Testuary is built to be extended: your own model service, prompt profile, hints for your project or a new language each come as one file, or a few lines in one. Running and measuring the tests is the job of LangAdapter, a separate package that installs with Testuary.

Supported systems

Projects in Windows Linux macOS
Python yes yes expected to work
Java yes yes expected to work
C in WSL yes not yet supported

Testuary was tested on Windows 11, Ubuntu 24.04 under WSL and AlmaLinux 8.10. This release has not run on a Mac. On Windows, C projects run with Testuary installed inside WSL. Tested environments says what is known.

Install

Testuary runs on Python 3.10 or newer. Install Testuary into a virtual environment of its own, outside the directory of the project you test, as getting started explains:

python3 -m venv testuary-env
source testuary-env/bin/activate
python -m pip install testuary

On Windows, create the environment with python -m venv testuary-env and activate it with testuary-env\Scripts\activate in cmd or PowerShell. In PowerShell, run Set-ExecutionPolicy -Scope Process RemoteSigned first: the default policy of Windows PowerShell refuses the activation script, and this command allows local scripts in that window only. On Debian and Ubuntu, python3 -m venv needs the package python3-venv (sudo apt install python3-venv), as Set up your machine describes.

Testing a C file needs LangAdapter's extra for C (python -m pip install "langadapter[c]") and the compiler tools that LangAdapter's C guide lists.

Testuary comes with a local web interface. Start it in the directory you work in:

testuary-web

It opens the interface in your browser, or in an SSH login prints the ssh -L line for your own computer. The web interface describes its pages and, under Access, how to start it on a machine you share.

The project you test needs its own test tools: pytest, coverage and Cosmic Ray for Python, or a JDK and the project's Maven or Gradle build for Java. Tested environments lists the tested versions. For Java, LangAdapter downloads the JUnit launcher, JaCoCo and PIT into its cache on first use, unless LANGADAPTER_JAVA_TOOLS_OFFLINE is set to 1.

Example

Create the configuration templates in your project's directory:

cd /path/to/project
testuary init

This writes models.ini into your user directory and hints.json into the project's directory, and names each file it writes. Then add your model to that models.ini. The section name is the alias you pass with --model, and provider = names the kind of model service, ollama, openai or vllm:

[ollama_qwen]
provider = ollama
model = qwen3-coder:30b
api_url = http://localhost:11434/v1

Run the quick check in the project's directory. The check's report says what it looks at and what it leaves alone:

testuary check --model ollama_qwen --python-executable /path/to/project-env/bin/python --source-files package/module.py

Generate a suite for one target. For a first run, a few lines in the project's settings file keep it small, as getting started shows. Run it in the project's directory too, where the run reads that file:

testuary generate --model ollama_qwen --python-executable /path/to/project-env/bin/python --source-files package/module.py

The results go to .testuary/results/ollama_qwen in the project's directory.

At a paid service, model requests can cost money. To see a run of all four phases without a model, play back the offline demonstration with testuary --demo --output-dir demo-python.

The quickstart directory of the repository, also in the source distribution, holds these first steps as scripts and commands: the demonstrations, the web interface started from the console or from Python, and two small projects, one Python file and one Java class, to run with your own model.

Documentation

License

Testuary is licensed under the MIT license. A few files from other projects, in the demonstration data, keep their own licenses, and so do the installed dependencies. Third-party material lists them.

Metadata

Release files for testuary 0.2.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for testuary 0.2.1
File Size Uploaded Provenance Actions
testuary-0.2.1.tar.gz 2.0 MB Without provenance Details for testuary-0.2.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for testuary 0.2.1
File Interpreter ABI Platform Actions
testuary-0.2.1-py3-none-any.whl Python 3 none any Details for testuary-0.2.1-py3-none-any.whl

Total release size: 4.0 MB

Release files / testuary-0.2.1.tar.gz

Download URL testuary-0.2.1.tar.gz
Size 2.0 MB
Tags Source
SHA-256 checksum
How to use checksums
70c66fb9c834e6e7c7e8a7bfec8404bdaac508de248a2dc3ce82a7e554607dd7
BLAKE2b-256 checksum
How to use checksums
b1667474d332d4f38398638d05eb011f218c6da15299efb804a89d5ad5313978
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.10

Release files / testuary-0.2.1-py3-none-any.whl

Download URL testuary-0.2.1-py3-none-any.whl
Size 2.1 MB
Tags Python 3
SHA-256 checksum
How to use checksums
47a370ba2b365ab78dcda0eed8beaede10b7a3acfe549c257e50c9a1089c1398
BLAKE2b-256 checksum
How to use checksums
4af66de2c32543bd01595d63f89ecc4fbbf545473ccdc3c1dbe18f4d89eb26e9
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.12.10

Security for testuary 0.2.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

0.2.1 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page