Securing the Testing Process for Industrial Automation Software

Abstract

The testing of automation applications has become a crucial pillar of every production systems engineering (PSE) project with the proliferation of cyber-physical systems (CPSs). In light of new attack vectors against CPSs, caused, inter alia, by increased connectivity, security aspects must be considered throughout the PSE process. In this context, software testing represents a critical activity, as a lack of adequate security mechanisms puts a variety of valuable assets (e.g., system configurations and production details) at risk of information theft and sabotage. Thus, organizations must analyze the security of their software testing process on a regular basis in order to counter these threats. Yet, due to the required security knowledge or budget constraints for security-related expenses, these undertakings may be destined to fail. In this work, we present a framework that supports the semi-automated security analysis of an organization’s software testing process for industrial automation software. This framework is based on the VDI/VDE 2182 guideline and integrates an ontological approach to model the necessary background knowledge, including, e.g., data flows, assets, entities, threats, and countermeasures. The framework comprises a default model of the testing process, which users can adapt so that the target of inspection accurately reflects their software testing environment. In particular, the testing process considered for creating the default model is based on best practices observed at a major system integrator, aligned with the ISO/IEC/IEEE 29119 series of software testing standards. Moreover, we developed a tool that enables the automatic generation of attack–defense trees from such formal models of the organization’s software testing process. We demonstrate how the proposed framework can be applied to a generic software testing process to answer essential questions in conducting a security risk analysis. The results of the exemplary security analysis provide guidance, should raise awareness in the industrial domain, and support effective, yet cost- and time-efficient security analyses. Finally, we evaluate the presented framework by performing a comprehensive comparison of suitable security analysis tools.