Skip to main content

Common Workflow Language testing framework

Linux Build Status Code coverage

PyPI: PyPI Version PyPI Downloads Month Total PyPI Downloads

Conda: Conda Version Conda Installs

This is a testing tool for checking the output of Tools and Workflows described with the Common Workflow Language. Among other uses, it is used to run the CWL conformance tests.

This is written and tested for Python 3.10, 3.11, 3.12, 3.13, and 3.14

Install

Installing the official package from PyPi

pip install cwltest

Or from bioconda

conda install -c bioconda cwltest

Or from source

git clone https://github.com/common-workflow-language/cwltest.git
cd cwltest && pip install  .

Run on the command line

Simple command:

cwltest --test test-descriptions.yml --tool cwl-runner

Generate conformance badges using cwltest

To make badges that show the results of the conformance test, you can generate JSON files for https://badgen.net by using –badgedir option

To generate JSON files:

cwltest --test test-descriptions.yml --tool cwl-runner --badgedir badges
...
$ cat badges/command_line_tool.json | jq .
{
  "subject": "command_line_tool",
  "status": "100%",
  "color": "green"
}

Once you upload JSON file to a server, you make a badge by using a link like https://badgen.net/https/path/to/generated/json or https://flat.badgen.net/https/path/to/generated/json (for flat badges).

Here is an example of markdown to add a badge:

![test result](https://flat.badgen.net/https/path/to/generated/json?icon=commonwl)

Custom file access module

If your CWL implementation does not write output files to a local file system location but instead to some other remote storage system, you can provide an alternate implementation of the StdFsAccess object that is able to access your storage system.

Step 1:

Implement your own class with the same public interface of the StdFsAccess object in cwltest/stdfsaccess.py (as of this writing, the methods are open, size, isfile and isdir). These methods should expect to be called with URIs from the location field of the outputs of test cases.

Define a function that, when called, returns a new instance of your object.

Step 2:

Create a Python package containing your class (or add it to an existing one).

In the package metadata, add an entry point that declares the module (in this example, my_cwl_runner.fsaccess) containing the function (in this example, get_fsaccess) that cwltest will invoke to get an object implementing the StdFsAccess interface.

In setup.py this looks like:

setup(
  ...
  entry_points={"cwltest.fsaccess": ["fsaccess=my_cwl_runner.fsaccess:get_fsaccess"]}},
  ...
)

In pyproject.toml it looks like:

[project.entry-points.'cwltest.fsaccess']
fsaccess = 'my_cwl_runner.fsaccess:get_fsaccess'

Step 3:

Install your package in the same Python environemnt as the installation of cwltest. When invoked, cwltest will query Python package metadata for a package with the cwltest.fsaccess entry point and call it to get back a custom filesystem access object.

Metadata

Release files for cwltest 2.7.20261106145043

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for cwltest 2.7.20261106145043
File Size Uploaded Provenance Actions
cwltest-2.7.20261106145043.tar.gz 50.2 kB Without provenance Details for cwltest-2.7.20261106145043.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for cwltest 2.7.20261106145043
File Interpreter ABI Platform Actions
cwltest-2.7.20261106145043-py3-none-any.whl Python 3 none any Details for cwltest-2.7.20261106145043-py3-none-any.whl

Total release size: 100.0 kB

Release files / cwltest-2.7.20261106145043.tar.gz

Download URL cwltest-2.7.20261106145043.tar.gz
Size 50.2 kB
Tags Source
SHA-256 checksum
How to use checksums
af324f0414ff366639e6a295561575a50b694450f487ae55098806c73e3c7208
BLAKE2b-256 checksum
How to use checksums
ea85f59b02fdcfba4c6d628e71bf477bfe6574b93fe31c650b27f8fc7387148f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.5

Release files / cwltest-2.7.20261106145043-py3-none-any.whl

Download URL cwltest-2.7.20261106145043-py3-none-any.whl
Size 49.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
7a2c0fc1c6398bd64783ccb478f016e836ce15ee45843712dddf58a7d154849a
BLAKE2b-256 checksum
How to use checksums
2a2fe586675e6fa260e1c57bf1a675ebfcc67fd3e72e1782f30892a5dddef763
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/7.0.0 CPython/3.13.5

Security for cwltest 2.7.20261106145043

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release
Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page