Federated Learning Systems in the Industrial Internet of Things

Abstract

In the Industrial Internet of Things, a multitude of sensors and devices deliver data from machines and production processes. Applying data analysis and trained Machine Learning models on the data provides insightful information for optimizing the processes and therefore increasing productivity. With the rise of Federated Learning decentralized devices are enabled to collaboratively train ML models without sharing data. For this, a server aggregates (i.e., averages) the model updates delivered by devices. This approach preserves privacy of the local clients and offers the potential to boost the performance of ML models, since knowledge is shared between involved FL participants without revealing raw data. The application of FL in industry and the development of holistic FL systems for industrial clients still face open issues. In particular, there is a need for providing tools and services to develop, deploy, and run FL solutions and integrate them into industry applications. Due to the diverse setup of individual devices and variations in underlying processes and monitored machines, the collected data often follows heterogeneous data distributions as well. Consequently, the training of a global model through averaging can result in insufficient model quality. Another problem is the optimal deployment of clients and their selection for FL runs. Essentially, it is crucial for efficient FL systems to minimize the response time and boost the performance of the model. Furthermore, there is still a gap to integrate FL solutions into business and manufacturing processes, since there is only limited information on practical implementation guidelines available for industry. This thesis presents systems and concepts that provide FL services to clients in the IIoT considering strategies for optimizing models towards individual data distributions. We support FL activities along the whole lifecycle from development to application integration. To optimize deployments of FL solutions, we present approaches for client placement and selection for FL clients by the server. For this, we consider edge, fog, and cloud platforms. Finally, we survey the prerequisites and requirements for FL solutions in practice and collect respective pain points to derive suitable FL blueprints for industrial collaboration. We evaluate the FL system(s) and proposed algorithms on industrial data from decentralized data sources and show improvements in the performances of the trained FL models as compared to isolated local ML and plain FL. The results of our deployment and client selection optimization demonstrate flexibility and convergence to the relatively best platform for FL clients aiming for optimizing e.g., model performance and response time. Our FL blueprints address the surveyed pain points of 13 companies and provide architectures for solving identified AI problems with FL.