Signal Feature Coverage and Testing for CPS Dataflow Models

Abstract

Design of cyber-physical systems (CPS) typically involves dataflow modelling. The structure of dataflow models differs from the traditional software, making standard coverage metrics not appropriate for measuring the thoroughness of testing. To address this limitation, this paper proposes signal feature coverage as a new coverage metric for systematically testing CPS dataflow models. We derive signal feature coverage by leveraging signal features. We developed a testing framework in Simulink®, a popular dataflow modelling and simulation environment, that automates the generation and execution of test cases based on the defined coverage metric. We evaluated the effectiveness of our approach by carrying out experiments on five Simulink®models tested against ten Signal Temporal Logic specifications. We compared our coverage-based testing approach to adaptive random testing, falsification testing, output diversity-based approaches, and testing using MathWorks’ Simulink® Design Verifier™. The results demonstrate that our coverage-based testing approach outperforms the conventional techniques regarding fault detection capability.