stgem: System Testing Using Generative Models

stgem is a tool for runtime verification of cyber-physical systems. It supports falsification of requirements described in Signal Temporal Logic (STL) and other runtime monitors using robustness semantics. This is achieved by training a generative machine learning model online to produce system inputs that yield a low robustness.

This repository contains stgem implementing the API Version 2.

The INSTALLATION.md file describes how to set up stgem.

Documentation

Use the generate-docs.sh script to build stgem documentation, available in the docs folder.

Please refer to the demo folder for a demonstration on how to setup a system under test, use stgem in conjunction with a test generator to search for inputs that violate a specified safety requirements and subsequently how to analyze the results.

Research Articles

stgem has been used in the folling research articles:

Main algorithms

• J. Peltomäki, I. Porres. Requirement falsification for cyber-physical systems using generative models. Autom. Softw. Eng 32(33) (2025). DOI, Preprint. The code specific for this paper is available here.

• J. Peltomäki, I. Porres. Learning test generators for cyber-physical systems (2024), submitted. Preprint

Benchmarks

• T. Khandait, F. Formica, P. Arcaini, S. Chotaliya, G. Fainekos, A. Hekal, A. Kundu, E. Lew, M. Loreti, C. Menghi, L. Nenzi, G. Pedrielli, J. Peltomäki, I. Porres, R. Ray, V. Soloviev, E. Visconti, M. Waga, Z. Zhang. ARCH-COMP 2024 category report: Falsification. Proceedings of 11th International Workshop on Applied Verification of Continuous and Hybrid Systems, ARCH24. EPiC Series in Computing, Vol. 103, 122-144 (2024). DOI. The code specific for this paper is available here.

• C. Menghi, P. Arcaini, W. Baptista, G. Ernst, G. Fainekos, F. Formica, S. Gon, T. Khandait, A. Kundu, G. Pedrielli, J. Peltomäki, I. Porres, R. Ray, M. Waga, Z. Zhang. ARCH-COMP 2023 category report: Falsification. Proceedings of 10th International Workshop on Applied Verification of Continuous and Hybrid Systems, ARCH23. EPiC Series in Computing, Vol. 96, 151-169 (2023). DOI. The code specific for this paper is available here.

Applications

• J. Winsten, I. Porres. Unified Search for Multi-requirement Falsification for Cyber-Physical Systems. Proceedings of International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2025 (2025). DOI. The code specific for this paper is available here.

• J. Winsten, V. Soloviev, J. Peltomäki, I. Porres. Adaptive test generation for unmanned aerial vehicles using WOGAN-UAV. The 17th Intl. Workshop on Search-Based and Fuzz Testing, SBFT 2024 (2024). DOI. The code specific for this paper is available here.

• J. Peltomäki, J. Winsten, M. Methais, I. Porres. Testing cyber-physical systems with explicit output coverage. Proceedings of International Conference on Software Testing, Verification and Validation Workshops, ICSTW 2024 (2024). DOI. The code specific for this paper is available here.

• J. Winsten, I. Porres. WOGAN at the SBFT 2023 tool competition - Cyber-physical systems track. The 16th Intl. Workshop on Search-Based and Fuzz Testing, SBFT 2023 (2023). DOI, Preprint.

  • Related presentation video here (slides)

• J. Peltomäki, F. Spencer, I. Porres. Wasserstein generative adversarial networks for online test generation for cyber physical systems. The 15th Intl. Workshop on Search-Based Software Testing, SBST 2022 (2022). DOI, Preprint.

• J. Peltomäki, F. Spencer, I. Porres. WOGAN at the SBST 2022 CPS tool competition. The 15th Intl. Workshop on Search-Based Software Testing, SBST 2022 (2022). DOI, Preprint.

  • Related presentation videos here (slides) and here (slides).

• J. Peltomäki, I. Porres. Falsification of multiple requirements for cyber-physical systems using online generative adversarial networks and multi-armed bandits. The 6th. Intl. Workshop on Testing Extra-Functional Properties and Quality Characteristics of Software Systems, ITEQS 2022 (2022). DOI, Preprint.

Contact us

stgem is developed at Åbo Akademi University. Contact Ivan Porres ivan.porres@abo.fi for more information.