Collaborative inference for edge intelligence - Impacts on performance and privacy of partition points

Abstract

Edge computing combined with Artificial Intelligence allows for a powerful new paradigm called Edge Intelligence, pushing the execution of AI-based applications towards the edge of the network. However, the heterogeneous infrastructure of edge computing poses challenges to performance and privacy of these applications, such as Video Analysis Pipelines (VAPs).VAPs utilize state-of-the-art machine learning models to analyze video streams, but their execution is limited at the edge, due to the insufficient performance characteristics of edge devices. Furthermore, they process data potentially containing personally identifiable information. Collaborative Inference (Co-Inference) poses a novel technique to alleviate both performance and privacy concerns, by distributing the computational workload of inference. Concretely, machine learning models are split at a partition point, resulting in a head and tail model, which are then deployed on different compute nodes. After performing inference with the head model on one device, resulting intermediate features are transmitted to another device, which finishes inference with the tail model. This collaborative approach can accelerate execution of the analysis, and improve data protection aspects, as intermediate features are transmitted and processed instead of raw (video) data. Determining an optimal partition point for Co-Inference is of paramount importance in order to maximize performance and privacy aspects of a VAP. This thesis analyzes the impact different partition points have on the performance and privacy of Co-Inference systems. To evaluate the impact on performance as close to real life scenarios as possible, a prototypical Co-Inference VAP for object detection was implemented on a heterogeneous hardware test environment typical for edge computing. Additionally, an image reconstruction attack was performed on the different partition points, to evaluate the reconstructability of original images from intermediate features and related privacy concerns. The performance benchmarks have shown how the different partition points, nine in total, of the implemented Co-Inference VAP impact throughput, resource utilization, and required bandwidth for the various devices of the testbed. Images exposing personally identifiable information were generated when applying the image reconstruction attack to six partition points. The generated images for the remaining three partition points did not expose such information, therefore protecting the Co-Inference VAP against the applied image reconstruction attack.